Artificial Mysterious Intelligence

Dude. Dude. No wonder you've been so emphatic in your denunciations of mysterious answers to mysterious questions.

The universe doesn't have to be kind and make all problems amenable to insight....

There are only a certain number of short programs, and once a program gets above a certain length it is hard to compress (I can't remember the reference for this, so it may be wrong, can anyone help?). We can of course reorder things, but then we have to make things currently simple complex.

That said I do think insight will play some small part in the development of AI, but that there may well be a hell a lot of parameter tweaking that we don't understand or know why they are so.

The arguments Eliezer describes are made, and his reactions are fair. But really the actual research community "grew out" of most of this stuff a while back. CYC and the "common sense" efforts were always a sideshow (in terms of research money and staff, not to mention results). Neural networks were a metonym for statistical learning for a while, then serious researchers figured out they needed to address statistical learning explicitly. Etc.

Admittedly there's always excessive enthusiasm for the current hot thing. A few years ago it was support vector machines, I'm not sure what now.

I recognize there's some need to deflate popular misconceptions, but there's also a need to move on and look at current work.

Eliezer, I'd be very interested in your comments on (what I regard as) the best current work. Examples for you to consider would be Sebastian Thrun, Andrew Ng (both in robotics at Stanford), Chris Manning (linguistics at Stanford), and the papers in the last couple of NIPS conferences (the word "Neural" in the conference title is just a fossil, don't have an allergic reaction).

As an entertaining side note, here's an abstract for a poster for NIPS '08 (happening tomorrow) that addresses the crossover between AI and ems:

A Bayesian Approach for Extracting State Transition Dynamics from Multiple Spike Trains

Neural activity is non-stationary and varies across time. Hidden Markov Models (HMMs) have been used to track the state transition among quasi-stationary discrete neural states. Within this context, an independent Poisson model has been used for the output distribution of HMMs; hence, the model is incapable of tracking the change in correlation without modulating the firing rate. To achieve this, we applied a multivariate Poisson distribution with a correlation term for the output distribution of HMMs. We formulated a Variational Bayes (VB) inference for the model. The VB could automatically determine the appropriate number of hidden states and correlation types while avoiding the overlearning problem. We developed an efficient algorithm for computing posteriors using the recursive relationship of a multivariate Poisson distribution. We demonstrated the performance of our method on synthetic data and a real spike train recorded from a songbird.This is a pretty good example of what I meant by "solving engineering problems" and it should help the ems program "cut corners".

Will Pearson: are you suggesting that the simplest algorithm for intelligence is too large to fit in human memory?

I should mention that the NIPS '08 papers aren't on line yet, but all previous conferences do have the papers, tutorials, slides, background material, etc. on line. For example here's last year.

4 seems important to me. I wouldn't expect intelligence to come via that route, but that route does seem to put a fairly credible (e.g. I would bet 4:1 on claims that credible and expect to win in the long term), though high, soft upper bound to how long we can go on with roughly current rate scientific progress without achieving AI. I'd say that it suggests such a soft uupper bound in the 2070s. That said, I wouldn't be at all surprised by science ceasing to advance at something like the current rate long before then, accelerating or decelerating a lot even without a singularity.

We shouldn't under-rate the power of insight, but we shouldn't over-rate it either; some systems can just be a mass of details, and to master such systems you must master those details. And if you pin your hopes for AI progress on powerful future insights, you have to ask how often such insights occur, and how many we would need. The track record so far doesn't look especially encouraging.

A question about Andrew Ng, who was mentined in this thread. Is that his real name?

A question about Andrew Ng, who was mentined in this thread. Is that his real name?

Here's his homepage: http://robotics.stanford.edu/~ang/

Looks like he's one of the people who worked on the LittleDog robot: http://cs.stanford.edu/groups/littledog/

Robin, the question of whether compact insights exist and whether they are likely to be obtained in reasonable time (and by how large a group, etc) are very different questions and should be considered separately, in order.

Jed, I've met Sebastian Thrun, he's smart but not quite Bayesian-to-the-uttermost-core. (I.e., he once gave an anecdote involving a randomized algorithm and it took me a while to figure out how to derandomize it.) Haven't met the others, but I agree with you that a whole lot of AI has moved on far ahead from the old days. Artificial Intelligence: A Modern Approach is a gorgeous textbook; e.g., they give some idea of what logic is good for and what it's not. Statistical learning is cool (I'm a Bayesian, what do you expect me to say?) and again the great marker is that often they can say, in an abstract sense, what kind of regularity the algorithm assumes and exploits.

But for some odd reason, a lot of the people working in the field of Artificial General Intelligence - "true AI" startups and projects - still seem to adhere to older ways, and to believe that mysterious algorithms can work... an obvious selection effect once you realize it.

But if the brain does not work by magic (of course), then insight does not either. Genius is 99% perspiration, 10,000 failed lightbulbs and all that...

I think the kind of experimental approach Jed Harris was talking about yesterday is where AI will eventually come from. Some researcher who has 10,000 failed AI programs on his hard drive will then have the insight, but not before. The trick is, once he has the insight, to not implement it right away but stop and think about friendliness! But after so many failures how could you resist...

Eliezer: Sorry to harp on something tangential to your main point, but you keep repeating the same mistake and it's bugging me. Evolution is not as slow as you think it is.

In an addendum to this post you mention that you tried a little genetic algorithm in Python, and it didn't do as badly as you would have expected from the math. There is a reason for this. You have the math completely wrong. Or rather, you have it correct for asexual reproduction, and then wrongly assume the limit still applies when you add in sex. As has been pointed out before, genetic algorithms with sex are much, much, much faster than asexual algorithms. Not faster by a constant factor; faster in proportion to the square root of the genome length, which can be pretty damn big.

The essential difference is that as a genome gets more fit, the odds of a mutation hurting rather than helping fitness go up, which limits your acceptable mutation rate, which limits your search rate for asexual reproduction. But if you rely on crossover (sex) to produce new individuals, (1) the expected fitness of the child is equal to the average fitness of the parents, even if the parents are already very fit; and (2) mutation isn't the main thing driving the search anyway, so even if your mutation rate is very low you can still search a large space quickly.

Once again I'll point to MacKay's Information Theory, Learning, and Inference Algorithms for a much better explanation of the math (from an information theory standpoint) than I'm capable of.

And just for fun, but here's another way to look at it that I haven't seen described before. Asexual evolution, as you've pointed out a few times, relies on mutations to generate new candidates, so it's just doing a random walk to adjacent positions in search space. But if you take two parent genomes and do a random crossover, you're effectively taking the hypercube (in the search space of genome strings) whose corners are the two parents, and randomly picking a different corner. So you're taking a much larger step. Mutations are now only necessary to prevent the population from collapsing into a lower-dimensional search space; they aren't the driver of innovation any more.

Jeff - thanks for your comment on evolutionary algorithms. Gave me a completely new perspective on these.

Real robots are hardly going to be the route to AI - too slow a build-test cycle.

Tim Tyler:

As much as I like your posts, one formal note:

If you are responding to somebody else, it is always a good idea to put his name at the beginning of post.

Jeff, if you search for my pseudonym in the comments of the "Natural Selection's Speed Limit and Complexity Bound" post, you will see that I have already brought MacKay's work to Eliezer's attention. Whatever conclusions he's come to have already factored MacKay in.

I'd previously read some of MacKay's book but not that particular section; I actually still have the information-gain problem marked as "pending". Anyone who can't distinguish between 1s gained in a bitstring, and negentropy gained in allele frequencies, is politely invited not to try to solve this particular problem.

How is this different from saying,

"For a long time, many different parties and factions in AI, adherent to more than one ideology, have been trying to build AI without understanding consciousness. Unfortunate habits of thought will already begin to arise, as soon as you start thinking of ways to create Artificial Intelligence without having to penetrate the mystery of consciousness. Instead of all this mucking about with neurons and neuroanatomy and population encoding and spike trains, we should be facing up to the hard problem of understanding what consciousness is."

I would consider that a perfectly legitimate remark if you were trying to build an Artificial Consciousness.

Eliezer: I was making a parallel. I didn't mean "how are these different"; I really meant, "This statement below about consciousness is wrong; yet it seems very similar to Eliezer's post. What is different about Eliezer's post that would make it not be wrong in the same way?"

That said, we don't know what consciousness is, and we don't know what intelligence is; and both occur in every instance of intelligence that we know of; and it would be surprising to find one without the other even in an AI; so I don't think we can distinguish between them.

Eliezer: "Anyone who can't distinguish between 1s gained in a bitstring, and negentropy gained in allele frequencies, is politely invited not to try to solve this particular problem."

Ok, here's the argument translated into allele frequencies. With sexual selection, mutations spread rapidly through the population, so we assume that each individual gets a random sample from the set of alleles for each gene. This means that some poor bastards will get more than their share of the bad ones and few of the good ones (for the current environment), while luckier individuals gets lots of good ones and few bad ones. When the unlucky individuals fail to reproduce, they're going to eliminate bad genes at a higher-than-average rate, and good genes at a lower-than-average rate.

"On average, one detrimental mutation leads to one death" does not hold with sexual selection.

Also, just in case I'm giving the wrong impression--I'm not trying to argue that genetic algorithms are some kind of secret sauce that has special relevance for AI. They just aren't as slow as you keep saying.

Eliezer, MacKay's math isn't very difficult. I think it will take you at most a couple of hours to go through how he derived his equations, understand what they mean, and verify that they are correct. (If I knew you were going to put this off for a year, I'd mentioned that during the original discussion.) After doing that, the idea that sexual reproduction speeds up evolution by gathering multiple bad mutations together to be disposed of at once will become pretty obvious in retrospect.

Jeff, I agree with what you are saying, but you're using the phrase "sexual selection" incorrectly, which might cause confusion to others. I think what you mean is "natural selection in a species with sexual reproduction". "Sexual selection" actually means "struggle between the individuals of one sex, generally the males, for the possession of the other sex".

Wei, I already understand the idea that sexual selection gathers multiple mutations to dispose - i.e., the rule is still "one mutation, one death" but not "one death, one mutation". I can even accept that eliminating half the population applies more than one bit of selection pressure, because the halves are not randomly assigned. But it's not a simple matter of reading MacKay and accepting everything he says - I have to reconcile with Worden's speed limit and with Kimura, and try to get a grasp on the negentropy produced rather than the number of bits in a bitstring.

Jeff, my estimate above on how slow evolution is, was based on the historical evolution time of Earth, rather than any numerical calculation.