Rationality Toughness Tests

This seems like a very general and useful idiom - I propose calling them hot/cold tests, if the term isn't already taken.

This concept comes up among poker players. Smarts corresponds to an ability to talk about the correct play in a hand in theory. Toughness corresponds to a player's ability to continue to play well in a downswing. There are a lot of correct plays that can lead to bad outcomes with high frequencies. Sometimes a player encounters so many bad outcomes that they begin to doubt whether the play is correct.

There are many issues that can make thinking rationally harder, some of which are very hard to reproduce in controlled circumstances, and being better at dealing with one issue does not necessarily make you any better at dealing with the others. Almost anything that we would describe as stressful impairs rationality. To your list of circumstances, I would add:

• Distractions: noisy environments, recent events
• Altered mental states: fatigue, intoxication
• Time pressure: either a hard limit on time available to decide, or thinking longer imposes a cost large enough to matter.

A few of these are easy to test; for example, you could take a written test once while rested, once while tired, once in a noisy room, once with a short time limit, etc. However, it doesn't seem likely that these tests would generalize well, and many of the obvious strategies for training seem unlikely to generalize well either.

I was very interested to read your views on why humans choke. I was thinking about this the other day in relation to a question posed by a professor. What I came up with at the time was that choking is caused by sympathetic fight or flight response (adaptive in less cognitively complex organisms but not in our human world where finesse is necessary). In other words, I was suggesting it is a misfire that is so deeply rooted in our mental architecture we haven't managed to evolve out of it yet. Also, I wonder, are there examples of non-human animals "choking"?

Edit: I meant "choking" in the sense of failing during high pressure situations. I was referring to Robin Hanson's second link ( to an OB post). Sorry for any confusion.

The distinction is one between competence (smarts) and performance (toughness).

"Measuring" either is a problem in ability and personality testing: find the underlying factors and the combinatorial rules; whether the constructs exist as consistent dimensions shouldn't be taken for granted.

Some findings will probably surprise you. The cognitive performance of the most able degrades the most under pressure for intellectually demanding tasks.

I read this post while in the middle writing my post on heuristic, and I suspect that smarts and toughness apply to both the heuristic/creative components of rationality, and to the critical/skeptical components of rationality.

I wonder which degrades more under pressure: the ability to formulate new productive ideas, or the ability to check the ideas you currently have.