Failure By Analogy

post by Eliezer Yudkowsky (Eliezer_Yudkowsky) · 2008-11-18T02:54:06.000Z · LW · GW · Legacy · 13 comments

Contents

13 comments

Previously in series:  Logical or Connectionist AI?
Followup to:  Surface Analogies and Deep Causes

"One of [the Middle Ages'] characteristics was that 'reasoning by analogy' was rampant; another characteristic was almost total intellectual stagnation, and we now see why the two go together. A reason for mentioning this is to point out that, by developing a keen ear for unwarranted analogies, one can detect a lot of medieval thinking today."
        -- Edsger W. Dijkstra

<geoff> neural nets are over-rated
<starglider> Their potential is overrated.
<geoff> their potential is us
        -- #sl4

Wasn't it in some sense reasonable to have high hopes of neural networks?  After all, they're just like the human brain, which is also massively parallel, distributed, asynchronous, and -

Hold on.  Why not analogize to an earthworm's brain, instead of a human's?

A backprop network with sigmoid units... actually doesn't much resemble biology at all.  Around as much as a voodoo doll resembles its victim.  The surface shape may look vaguely similar in extremely superficial aspects at a first glance.  But the interiors and behaviors, and basically the whole thing apart from the surface, are nothing at all alike.  All that biological neurons have in common with gradient-optimization ANNs is... the spiderwebby look.

And who says that the spiderwebby look is the important fact about biology?  Maybe the performance of biological brains has nothing to do with being made out of neurons, and everything to do with the cumulative selection pressure put into the design.  Just like how the performance of biological brains has little to do with proteins being held together by van der Waals forces, instead of the much stronger covalent bonds that hold together silicon.  Sometimes evolution gets stuck with poor building material, and it can't refactor because it can't execute simultaneous changes to migrate the design.  If biology does some neat tricks with chemistry, it's because of the greater design pressures exerted by natural selection, not because the building materials are so wonderful.

Maybe neurons are just what brains happen to be made out of, because the blind idiot god is too stupid to sit down and invent transistors.  All the modules get made out of neurons because that's all there is, even if the cognitive work would be much better-suited to a 2GHz CPU.

"Early attempts to make flying machines often did things like attaching beak onto the front, or trying to make a wing which would flap like a bird's wing.  (This extraordinary persistent idea is found in Leonardo's notebooks and in a textbook on airplane design published in 1911.)  It is easy for us to smile at such naivete, but one should realize that it made good sense at the time.  What birds did was incredible, and nobody really knew how they did it.  It always seemed to involve feathers and flapping.  Maybe the beak was critical for stability..."
        -- Hayes and Ford, "Turing Test Considered Harmful"

So... why didn't the flapping-wing designs work?  Birds flap wings and they fly.  The flying machine flaps its wings.  Why, oh why, doesn't it fly?

Because... well... it just doesn't.  This kind of analogic reasoning is not binding on Reality.

One of the basic tests to apply to reasoning that sounds kinda-good is "How shocked can I justifiably be if Reality comes back and says 'So what'?"

For example:  Suppose that, after keeping track of the motions of the planets for centuries, and after confirming the underlying theory (General Relativity) to 14 decimal places, we predict where Mercury will be on July 1st, 2009.  So we have a prediction, but that's not the same thing as a fact, right?  Anyway, we look up in the sky on July 1st, 2009, and Reality says "So what!" - the planet Mercury has shifted outward to the same orbit as Mars.

In a case like this, I would be highly indignant and would probably sue Reality for breach of contract.

But suppose alternatively that, in the last twenty years, real estate prices have never gone down.  You say, "Real estate prices have never gone down - therefore, they won't go down next year!"  And next year, Reality says "So what?"  It seems to me that you have no right to be shocked.  You have used an argument to which Reality can easily say "So what?"

"Nature is the ultimate bigot, because it is obstinately and intolerantly devoted to its own prejudices and absolutely refuses to yield to the most persuasive rationalizations of humans."
        -- J. R. Molloy

It's actually pretty hard to find arguments so persuasive that even Reality finds them binding.  This is why Science is more difficult - why it's harder to successfully predict reality - than medieval scholars once thought.

One class of persuasive arguments that Reality quite often ignores is the Law of Similarity - that is, the argument that things which look similar ought to behave similarly.

A medieval alchemist puts lemon glazing onto a lump of lead.  The lemon glazing is yellow, and gold is yellow.  It seems like it ought to work... but the lead obstinately refuses to turn into gold.  Reality just comes back and says, "So what?  Things can be similar in some aspects without being similar in other aspects."

You should be especially suspicious when someone says, "I am building X, which will do P, because it is similar to Y, which also does P."

An abacus performs addition; and the beads of solder on a circuit board bear a certain surface resemblance to the beads on an abacus. Nonetheless, the circuit board does not perform addition because we can find a surface similarity to the abacus. The Law of Similarity and Contagion is not relevant.   The circuit board would work in just the same fashion if every abacus upon Earth vanished in a puff of smoke, or if the beads of an abacus looked nothing like solder.  A computer chip is not powered by its similarity to anything else, it just is.  It exists in its own right, for its own reasons.

The Wright Brothers calculated that their plane would fly - before it ever flew - using reasoning that took no account whatsoever of their aircraft's similarity to a bird.  They did look at birds (and I have looked at neuroscience) but the final calculations did not mention birds (I am fairly confident in asserting).  A working airplane does not fly because it has wings "just like a bird".  An airplane flies because it is an airplane, a thing that exists in its own right; and it would fly just as high, no more and no less, if no bird had ever existed.

The general form of failing-by-analogy runs something like this:

• You want property P.
• X has property P.
• You build Y, which has one or two surface similarities S to X.
• You argue that Y resembles X and should also P.
• Yet there is no reasoning which you can do on Y as a thing-in-itself to show that it will have property P, regardless of whether or not X had ever existed.

Analogical reasoning of this type is a very weak form of understanding.  Reality often says "So what?" and ignores the argument.

The one comes to us and says:  "Calculate how many synaptic transmissions per second take place in the human brain.  This is the computing power required for human-equivalent intelligence.  Raise enough venture capital to buy a supercomputer which performs the same number of floating-point operations per second.  Intelligence is bound to emerge from a machine that powerful."

So you reply:  "I'm sorry, I've never seen a human brain and I don't know anything about them.  So, without talking about a human brain, can you explain how you calculated that 10^17 floating-point operations per second is the exact amount necessary and sufficient to yield human-equivalent intelligence?"

And the one says:  "..."

You ask:  "Say, what is this property of 'human-equivalent intelligence' which you expect to get?  Can you explain it to me without pointing to a human?"

And the one says:  "..."

You ask:  "What makes you think that large amounts of computing power have something to do with 'intelligence', anyway?  Can you answer without pointing to the example of the human brain?  Pretend that I've never seen an 'intelligence' and that I have no reason as yet to believe any such thing can exist."

But you get the idea.

Now imagine that you go to the Wright Brothers and say:  "I've never seen a bird.  Why does your aircraft have 'wings'?  And what is it you mean by 'flying'?"

And the Wright Brothers respond:  "Well, by flying, we mean that this big heavy object is going to rise off the ground and move through the air without being supported.  Once the plane is moving forward, the wings accelerate air downward, which generates lift that keeps the plane aloft."

If two processes have forms that are nearly identical, including internal structure that is similar to as many decimal places as you care to reason about, then you may be able to almost-prove results from one to the other.  But if there is even one difference in the internal structure, then any number of other similarities may be rendered void.  Two deterministic computations with identical data and identical rules will yield identical outputs.  But if a single input bit is flipped from zero to one, the outputs are no longer required to have anything in common.  The strength of analogical reasoning can be destroyed by a single perturbation.

Yes, sometimes analogy works.  But the more complex and dissimilar the objects are, the less likely it is to work.  The narrower the conditions required for success, the less likely it is to work.  The more complex the machinery doing the job, the less likely it is to work.  The more shallow your understanding of the object of the analogy, the more you are looking at its surface characteristics rather than its deep mechanisms, the less likely analogy is to work.

Analogy might work for something on the order of:  "I crossed this river using a fallen log last time, so if I push another log across it, I might be able to cross it again."  It doesn't work for creating objects of the order of complexity of, say, a toaster oven.  (And hunter-gatherer bands face many rivers to cross, but not many toaster ovens to rewire.)

Admittedly, analogy often works in mathematics - much better than it does in science, in fact.  In mathematics you can go back and prove the idea which analogy originally suggested.  In mathematics, you get quick feedback about which analogies worked and which analogies didn't, and soon you pick up the pattern.  And in mathematics you can always see the entire insides of things; you are not stuck examining the surface of an opaque mystery.  Mathematical proposition A may be analogous to mathematical proposition B, which suggests the method; but afterward you can go back and prove A in its own right, regardless of whether or not B is true.  In some cases you may need proposition B as a lemma, but certainly not all cases.

Which is to say: despite the misleading surface similarity, the "analogies" which mathematicians use are not analogous to the "analogies" of alchemists, and you cannot reason from the success of one to the success of the other.

13 comments

Comments sorted by oldest first, as this post is from before comment nesting was available (around 2009-02-27).