Wits and Wagers

It is actually mostly about social inference; first everyone makes a guess and then everyone can make bets - so you can see everyone's guesses when you make your bets. It is a good simple fast test in who you should listen to when.

I have played it, or something very much like it.

My answer is that people are shockingly bad at some aspects. Just knowing the world population, the population of the US, and the difference between million and billion is enough to do significantly better than average for many questions.

It's not good training, though: the people who did poorly didn't really seem to learn. It also is a "which answer is the closest without going over" system, which makes for interesting game play but is not a useful constraint for many practical questions. It also asks things for which you have no information at all, and you just pick based on only the available choices (much like guessing on a multiple-choice test). That's not really a useful skill outside of taking tests. (i.e., in real life you should get more information if possible instead of guessing)

I've played it and enjoyed it quite a bit. One interesting aspect was that you're penalized for guessing "too high," so you have to be careful.

Another is how you can manipulate peoples' biases - place your estimate at one point and then bet on someone else's. A perfectly rational player wouldn't fall for this, but some might think you wouldn't make a bad estimate for fear of looking ignorant.

I (an electrical engineer) played with an economist, two psychologists, and a musician. I was way behind until one question was for the speed of light in miles per second. I bet with high confidence, and all followed my bets, but the winnings were substantial.

I would recommend getting a good group of people to play it, and to change the rules around and see how they affect play. It's a good time.

Robin Hanson introduced me to it. It does ask about things for which you have no information, which is why it is about calibration. You draw a card with a question with a numeric answer. Everyone publicly guesses at the answer. You then wager on whose answer is closest without going over. It's a good demonstration of the effectiveness of recalibration. It's also about expected value, because your wager gains you more points if you wager on an answer far from the median, so you need to trade off confidence vs. expected returns.

I tried it out and it is much easier to play than my game (see http://lesswrong.com/lw/ilw/games_for_rationalists/ ) and thus is somewhat more fun. But it also has less insights.

My experience is as follows:

• The trivia questions are not difficult enough. It is very seldom that values lie outside a times 2-range. And 'surprises' are rare.
• The questions have an american cultural bias (no wonder)
• The 'going over' rule is simple but totally skews the betting and guessing.
• The simple payout-rules cause gaming for higher payouts thus mixing confidence and probability in non-trivial ways.
• The two-phase setup where you can look where the 'experts' bet is interesting but doesn't help with confidence calibration.

It really is optimized for playability. I think it does some calibration of (over)confidence and it builds intuition for probability and risk-trade-offs.

But - and that is my main point - it doesn't have clear concepts. The concepts are all mingled up, skewed, hidden. You may gain intuition but it will not help you toward overcoming e.g. overconfidence bias or egocentric bias.

I still think for that the concept must be sufficiently present to be able to reflect and consciously use it.