Protein Reinforcement and DNA Consequentialism

I think you are making the same sort of error here that you frequently admonish us against. In particular it seems to me that your argument here is tacitly assuming that somehow the experience of imagining is necessery to efficently predict the future. As you yourself have pointed out we shouldn't ascribe our own ends and desires to the genes that 'act' through us. Thus the fact that we propogate our genes by explicit verbal reasoning and imagination doesn't give us any particular reason to believe that our genetic computers are any more predictively accurate than those of other animals.

I mean suppose you asked the best and brightest geneticists and computer scientists to design a set of genes (starting from scratch) that would best enable the organism containing them to quickly and accurately adapt to new circumstances. Do you really think we could do better than the genome of a fish? If not then in what sense isn't it true that the genome of a fish is a more efficient calculating device for how to respond to the environment?

Reinforcement learning isn't trivial, but it might be kind of modular. Evolution doesn't have to create reinforcement learning from scratch every time, it can re-use the existing structures and just hook in new inputs.

Learning takes a lot of overhead. Much less overhead if you can simply be born knowing what you need to know, instinctively. Konrad Lorenz believed that instincts somehow develop out of learned behavior. He looked at related species where one had an instinctual behavior and the other had to learn the same behavior. It takes so long to learn instincts by mutation and selection, it's intuitively obvious that learning should come first and then be replaced by instinct over time. But Lorenz didn't propose a mechanism for that to happen and I don't know a mechanism either.

We do so much less instinct than other animals that I'm tempted to think we haven't spent much time in consistent predictable environments. Maybe we've always lived by disrupting ecosystems and surviving in the chaos -- a role we would manage better than animals that do less flexible thinking.

To put my point a bit differently I think your argument would disappear if you tried to define imagine/reason more precisely.

Humans certainly aren't perfect at imagining. In fact if you ask most people to imagine a heavy object and a much heavier object falling they will predict the much heavier object hits first and I can give a host of other examples of the same thing. So certainly we can't require perfection to count as imagining or reasoning.

Neither do we want to define reasoning in terms of speaking english sentences or anything similar. I mean we can imagine the possibility of intelligent alien blobs of gas that have no language at all (or at least that we can understand). Besides using this definition would just make the claims trivial.

The best approach I can think of to making these terms more precisce is by defining reasoning ability in terms of ability to respond in a survival enhancing way to a diverse set of conditions. However, when we try to define it this way it is far from clear that complexes of genes aren't just as good at reasoning as people are.

Sure it's weird that something like this might be the case but it's damn weird that induction works at all so why not believe it can be captured in a large number of relatively simple heuristics.

"How the heck does a double-stranded molecule that fits inside a cell nucleus, come to embody truths that baffle a whole damn squirrel brain?"

This kind of thing happens all the time, even in distantly related fields like physics. Observe: a dropped brick perfectly obeys the laws for movement within a curved Lorentzian manifold, without a brain or even a computer, while it would take a mere human half an hour to do out the calculations! Surely we should appoint bricks as pre-eminent physics professors.

Surely we should appoint bricks as pre-eminent physics professors.

Well, Isaac Newton did crib his theory from an apple.

logicnazi, he is making progress ;-)

Humans certainly aren't perfect at imagining. In fact if you ask most people to imagine a heavy object and a much heavier object falling they will predict the much heavier object hits first and I can give a host of other examples of the same thing.

When you ask someone to imagine something he is controlling his imagination which is equivalent to conscious though. What one can think of however is controlled by ones beliefs - what is skewed in humans is their beliefs, not their imagination. Once beliefs are being controlled by science one will imagine the falling consistent with the scientific theory forming ones beliefs. I would reword your sentence to: humans not usually form their beliefs on the basis of science.

A conscious mind on a protein-based computer? You expect me to believe in sentient MEAT!?

As an interesting aside, I believe calorie-counting isn't done in the stomach. However, protein content is measured directly at the taste buds. Which explains why MSG tastes so good, and makes a light meal seem more filling. At least until digestion has started and the expected barrage of amino acids is somehwat lacking.

Surely we should appoint bricks as pre-eminent physics professors.

This was the insight that started the scientific method. If you want to know how things fall, don't read Aristotle, drop things and watch them.

Empiricism, trial and error - these are the most powerful forces for knowledge there are. Reason is excellent, but overrated.

A conscious mind on a protein-based computer? You expect me to believe in sentient MEAT!? "Yes, thinking meat! Conscious meat! Loving meat. Dreaming meat. The meat is the whole deal! Are you beginning to get the picture or do I have to start all over?"

"If we want to know how objects fall, though, we shouldn't ask people with the socially-awarded status of authority, we should look at falling things - like bricks."

But now we don't do much of that, we read physics books and work out the homework. There isn't time to repeat all the experiments. When it comes to supercolliders etc it would take years to work out the math and deal with the confounding variables and such.

And no one could do it. It's not just the time needed. The original work was done by many people over decades of time. NO ONE could repeat it on their own.

Billswift, after thinking it over some I'm surprised how much one person could do, if they knew not to follow any blind alleys. Like, you could do all the experiments needed to support Maxwell's equations in a reasonably short time if someone helped you set up the equipment.

It took a lot of people a long time to do it, but that's because they didn't know what they were doing ahead of time.

I don't know where the limits would come but they might be a lot broader than I first thought.

IMO a fun project (for those like me who like this but are clearly not smart enough to be part of a Singularity developing team): create an object-based environment with maybe rule-based reproductive agents, with customizable explicit world rules (as in a computer game, not as in physics) and let them evolve. Maybe users across the world can add new magical artifacts and watch the creatures fail hilariously...

On a more related note, the post sounds ominous for any hope of a general AI. There may be no clear distinction between protein computer and just protein, between learning and blindly acting. If we and our desires are in such a position, aren't any AI we can make indirectly also blind? Or as I understand, Eliezer seems to think that we can (or had better) bootstrap, both for intelligence/computation and morality. For him, this bootstrapping, this understanding/theory of the generality of our own intelligence (as step one of the bootstrapping) seems to be the Central Truth of Life (tm). Maybe he's right, but for me with less insight into intelligence, that's not self-evident. And he didn't explain this crucial point anywhere clearly, only advocated it. Come on, it's not as if enough people can be a seed AI programmer even armed with that Central Truth. (But who knows.)

Your general point is valid. But you say "no non-primate species", and I know of some non-primate species that can do this task. Namely, cetaceans, corvids, and octopus. Slight nitpick, as these are all very smart animals.