Surprised by Brains

post by Eliezer Yudkowsky (Eliezer_Yudkowsky) · 2008-11-23T07:26:41.000Z · LW · GW · Legacy · 28 comments

Contents

28 comments

Followup to:  Life's Story Continues

Imagine two agents who've never seen an intelligence - including, somehow, themselves - but who've seen the rest of the universe up until now, arguing about what these newfangled "humans" with their "language" might be able to do...

Believer:  Previously, evolution has taken hundreds of thousands of years to create new complex adaptations with many working parts.  I believe that, thanks to brains and language, we may see a new era, an era of intelligent design. In this era, complex causal systems - with many interdependent parts that collectively serve a definite function - will be created by the cumulative work of many brains building upon each others' efforts.

Skeptic:  I see - you think that brains might have something like a 50% speed advantage over natural selection?  So it might take a while for brains to catch up, but after another eight billion years, brains will be in the lead.  But this planet's Sun will swell up by then, so -

Believer:  Thirty percent?  I was thinking more like three orders of magnitude. With thousands of brains working together and building on each others' efforts, whole complex machines will be designed on the timescale of mere millennia - no, centuries!

Skeptic:  What?

Believer:  You heard me.

Skeptic:  Oh, come on!  There's absolutely no empirical evidence for an assertion like that!  Animal brains have been around for hundreds of millions of years without doing anything like what you're saying.  I see no reason to think that life-as-we-know-it will end, just because these hominid brains have learned to send low-bandwidth signals over their vocal cords.  Nothing like what you're saying has happened before in my experience -

Believer:  That's kind of the point, isn't it?  That nothing like this has happened before?  And besides, there is precedent for that kind of Black Swan - namely, the first replicator.

Skeptic:  Yes, there is precedent in the replicators.  Thanks to our observations of evolution, we have extensive knowledge and many examples of how optimization works.  We know, in particular, that optimization isn't easy - it takes millions of years to climb up through the search space.  Why should "brains", even if they optimize, produce such different results?

Believer:  Well, natural selection is just the very first optimization process that got started accidentally.   These newfangled brains were designed by evolution, rather than, like evolution itself, being a natural process that got started by accident.  So "brains" are far more sophisticated - why, just look at them.  Once they get started on cumulative optimization - FOOM!

Skeptic:  So far, brains are a lot less impressive than natural selection.  These "hominids" you're so interested in - can these creatures' handaxes really be compared to the majesty of a dividing cell?

Believer:  That's because they only just got started on language and cumulative optimization.

Skeptic:  Really?  Maybe it's because the principles of natural selection are simple and elegant for creating complex designs, and all the convolutions of brains are only good for chipping handaxes in a hurry.  Maybe brains simply don't scale to detail work.  Even if we grant the highly dubious assertion that brains are more efficient than natural selection - which you seem to believe on the basis of just looking at brains and seeing the convoluted folds - well, there still has to be a law of diminishing returns.

Believer:  Then why have brains been getting steadily larger over time?  That doesn't look to me like evolution is running into diminishing returns.  If anything, the recent example of hominids suggests that once brains get large and complicated enough, the fitness advantage for further improvements is even greater -

Skeptic:  Oh, that's probably just sexual selection!  I mean, if you think that a bunch of brains will produce new complex machinery in just a hundred years, then why not suppose that a brain the size of a whole planet could produce a de novo complex causal system with many interdependent elements in a single day?

Believer:  You're attacking a strawman here - I never said anything like that.

Skeptic:  Yeah?  Let's hear you assign a probability that a brain the size of a planet could produce a new complex design in a single day.

Believer:  The size of a planet?  (Thinks.)  Um... ten percent.

Skeptic:  (Muffled choking sounds.)

Believer:  Look, brains are fast.  I can't rule it out in principle -

Skeptic:  Do you understand how long a day is?  It's the amount of time for the Earth to spin on its own axis, once.  One sunlit period, one dark period.  There are 365,242 of them in a single millennium.

Believer:  Do you understand how long a second is?  That's how long it takes a brain to see a fly coming in, target it in the air, and eat it.  There's 86,400 of them in a day.

Skeptic:  Pffft, and chemical interactions in cells happen in nanoseconds.  Speaking of which, how are these brains going to build any sort of complex machinery without access to ribosomes?  They're just going to run around on the grassy plains in really optimized patterns until they get tired and fall over.  There's nothing they can use to build proteins or even control tissue structure.

Believer:  Well, life didn't always have ribosomes, right?  The first replicator didn't.

Skeptic:  So brains will evolve their own ribosomes?

Believer:  Not necessarily ribosomes.  Just some way of making things.

Skeptic:  Great, so call me in another hundred million years when that evolves, and I'll start worrying about brains.

Believer:  No, the brains will think of a way to get their own ribosome-analogues.

Skeptic:  No matter what they think, how are they going to make anything without ribosomes?

Believer:  They'll think of a way.

Skeptic:  Now you're just treating brains as magic fairy dust.

Believer:  The first replicator would have been magic fairy dust by comparison with anything that came before it -

Skeptic:  That doesn't license throwing common sense out the window.

Believer:  What you call "common sense" is exactly what would have caused you to assign negligible probability to the actual outcome of the first replicator.  Ergo, not so sensible as it seems, if you want to get your predictions actually right, instead of sounding reasonable.

Skeptic:  And your belief that in the Future it will only take a hundred years to optimize a complex causal system with dozens of interdependent parts - you think this is how you get it right?

Believer:  Yes!  Sometimes, in the pursuit of truth, you have to be courageous - to stop worrying about how you sound in front of your friends - to think outside the box - to imagine futures fully as absurd as the Present would seem without benefit of hindsight - and even, yes, say things that sound completely ridiculous and outrageous by comparison with the Past.  That is why I boldly dare to say - pushing out my guesses to the limits of where Truth drives me, without fear of sounding silly - that in the far future, a billion years from now when brains are more highly evolved, they will find it possible to design a complete machine with a thousand parts in as little as one decade!

Skeptic:  You're just digging yourself deeper.  I don't even understand how brains are supposed to optimize so much faster.  To find out the fitness of a mutation, you've got to run millions of real-world tests, right?  And even then, an environmental shift can make all your optimization worse than nothing, and there's no way to predict  that no matter how much you test -

Believer:  Well, a brain is complicated, right?  I've been looking at them for a while and even I'm not totally sure I understand what goes on in there.

Skeptic:  Pffft!  What a ridiculous excuse. 

Believer:  I'm sorry, but it's the truth - brains are harder to understand.

Skeptic:  Oh, and I suppose evolution is trivial?

Believer:  By comparison... yeah, actually.

Skeptic:  Name me one factor that explains why you think brains will run so fast.

Believer:  Abstraction.

Skeptic:  Eh?   Abstrah-shun?

Believer:  It... um... lets you know about parts of the search space you haven't actually searched yet, so you can... sort of... skip right to where you need to be -

Skeptic:  I see.  And does this power work by clairvoyance, or by precognition?  Also, do you get it from a potion or an amulet?

Believer:  The brain looks at the fitness of just a few points in the search space - does some complicated processing - and voila, it leaps to a much higher point!

Skeptic:  Of course.  I knew teleportation had to fit in here somewhere.

Believer:  See, the fitness of one point tells you something about other points -

Skeptic:  Eh?  I don't see how that's possible without running another million tests.

Believer:  You just look at it, dammit!

Skeptic:  With what kind of sensor?  It's a search space, not a bug to eat!

Believer:  The search space is compressible -

Skeptic:  Whaa?  This is a design space of possible genes we're talking about, not a folding bed -

Believer:  Would you stop talking about genes already!  Genes are on the way out!  The future belongs to ideas!

Skeptic:  Give. Me. A. Break.

Believer:  Hominids alone shall carry the burden of destiny!

Skeptic:  They'd die off in a week without plants to eat.  You probably don't know this, because you haven't studied ecology, but ecologies are complicated - no single species ever "carries the burden of destiny" by itself.  But that's another thing - why are you postulating that it's just the hominids who go FOOM?  What about the other primates?  These chimpanzees are practically their cousins - why wouldn't they go FOOM too?

Believer:  Because it's all going to shift to the level of ideas, and the hominids will build on each other's ideas without the chimpanzees participating -

Skeptic:  You're begging the question.  Why won't chimpanzees be part of the economy of ideas?  Are you familiar with Ricardo's Law of Comparative Advantage?  Even if chimpanzees are worse at everything than hominids, the hominids will still trade with them and all the other brainy animals.

Believer:  The cost of explaining an idea to a chimpanzee will exceed any benefit the chimpanzee can provide.

Skeptic:  But why should that be true?  Chimpanzees only forked off from hominids a few million years ago.  They have 95% of their genome in common with the hominids.  The vast majority of optimization that went into producing hominid brains also went into producing chimpanzee brains.  If hominids are good at trading ideas, chimpanzees will be 95% as good at trading ideas.  Not to mention that all of your ideas belong to the far future, so that both hominids, and chimpanzees, and many other species will have evolved much more complex brains before anyone starts building their own cells -

Believer:  I think we could see as little as a million years pass between when these creatures first invent a means of storing information with persistent digital accuracy - their equivalent of DNA - and when they build machines as complicated as cells.

Skeptic:  Too many assumptions... I don't even know where to start...  Look, right now brains are nowhere near building cells.  It's going to take a lot more evolution to get to that point, and many other species will be much further along the way by the time hominids get there.  Chimpanzees, for example, will have learned to talk -

Believer:  It's the ideas that will accumulate optimization, not the brains.

Skeptic:  Then I say again that if hominids can do it, chimpanzees will do it 95% as well.

Believer:  You might get discontinuous returns on brain complexity.  Like... even though the hominid lineage split off from chimpanzees very recently, and only a few million years of evolution have occurred since then, the chimpanzees won't be able to keep up.

Skeptic:  Why?

Believer:  Good question.

Skeptic:  Does it have a good answer?

Believer:  Well, there might be compound interest on learning during the maturational period... or something about the way a mind flies through the search space, so that slightly more powerful abstracting-machinery can create abstractions that correspond to much faster travel... or some kind of feedback loop involving a brain powerful enough to control itself... or some kind of critical threshold built into the nature of cognition as a problem, so that a single missing gear spells the difference between walking and flying... or the hominids get started down some kind of sharp slope in the genetic fitness landscape, involving many changes in sequence, and the chimpanzees haven't gotten started down it yet... or all these statements are true and interact multiplicatively... I know that a few million years doesn't seem like much time, but really, quite a lot can happen.  It's hard to untangle.

Skeptic:  I'd say it's hard to believe.

Believer:  Sometimes it seems that way to me too!  But I think that in a mere ten or twenty million years, we won't have a choice.

28 comments

Comments sorted by oldest first, as this post is from before comment nesting was available (around 2009-02-27).