Humans are stunningly rational and stunningly irrational

post by Stuart_Armstrong · 2020-10-23T14:13:59.956Z · LW · GW · 4 comments

Contents

  Humans are stunningly irrational
  Humans are stunningly rational
  In conclusion
None
4 comments

Are humans mainly rational or mainly irrational? This is an old debate, with some political implications.

But the question is ill posed, because it elides the important detail: compared with what?

Humans are stunningly irrational

Let $V$ be some reward/utility function that encodes human preferences - including a mix of wealth, security, status, happiness, pleasant lives, good friends and experiences, romantic connections, sex, etc... It doesn't matter too much which of the adequate [LW · GW] $V$'s this is.

Then it's clear that humans are stunningly irrational as compared with a fully rational $V$-maximiser. As I mentioned before, one of the reasons that we think we're rational is because we only contemplate a tiny slice of the action space [LW · GW].

But if I were unboundedly rational, I could do many things. I could re-engineer biology from physics and some research articles, and create cures for multiple diseases, becoming incredibly rich and respected. I could launch a media campaign to make myself very popular. I could create tailored drugs to make myself maximally happy and extremely productive.

Hell, I could just program a superintelligence to optimise the world and maximise $V$. Unboundedly rational means that I'd be more effective than any such superintelligence, so I'd achieve a lot.

So it seems that, if a random agent gets $V=0$, and an unboundedly rational $V$-maximiser would expect to get $V=1$, then a human would get a value of $V$ that is utterly tiny - maybe $1/{10}^{100}$ or smaller. On the scale of randomness to full rationality, humans are almost indistinguishable from the randomness. We're stunningly irrational.

Humans are stunningly rational

To see how rational humans are, let's zoom in on that difference between humans and randomness. Suppose that it takes twenty actions to walk to a door (it actually takes far more). Then every time a human walks to a door, they add at least twenty bits of information as compared with a random agent. Add in the movements that humans do every day, the planning, the eating, the conversations, and we have thousands of bits of information; we occupy a tiny slice of possibility space, and have extreme optimisation powers.

And that degree of optimisation is similar to that of animals. Humans have longer term planning, rather extreme social skills, the ability to integrate vast amount of new information and come up with new ideas and designs. We direct the world to utterly tiny areas of outcome space.

Maybe we've directed the world to the top $1/{10}^{100}$ of outcome space, or an even smaller space. We're stunningly rational.

In conclusion

So, are humans rational? Well, we deploy a stunningly high amount of optimisation power to achieve our goals. On the other hand, we fall stunningly short of those goals.

So our rationality depends on what we're being compared with (eg random policy vs unbounded rationality) and what measure is being used (eg optimisation power vs expected utility).

There is no simple intrinsic measure as to "how rational" humans are. In questions where this is relevant, the comparison and measure have to be specified.

4 comments

Comments sorted by top scores.