Real World Solutions to Prisoners' Dilemmas

Yay Dr. Zany! And a good post in general.

However, Western behavior in the Ultimatum Game seems to be a cultural, not biological, phenomenon.

By the mid‐1990s researchers were arguing that a set of robust experimental findings from behavioral economics were evidence for set of evolved universal motivations (Fehr & Gächter 1998, Hoffman et al. 1998). Foremost among these experiments, the Ultimatum Game, provides a pair of anonymous subjects with a sum of real money for a one‐shot interaction. One of the pair—the proposer—can offer a portion of this sum to a second subject, the responder. Responders must decide whether to accept or reject the offer. If a responder accepts, she gets the amount of the offer and the proposer takes the remainder; if she rejects both players get zero. If subjects are motivated purely by self‐interest, responders should always accept any positive offer; knowing this, a self‐interested proposer should offer the smallest non‐zero amount. Among subjects from industrialized populations—mostly undergraduates from the U.S., Europe, and Asia—proposers typically offer an amount between 40% and 50% of the total, with a modal offer of usually 50% (Camerer 2003). Offers below about 30% are often rejected.

With this seemingly robust empirical finding in their sights, Nowak, Page and Sigmund (2000) constructed an evolutionary analysis of the Ultimatum Game. When they modeled the Ultimatum Game exactly as played, they did not get results matching the undergraduate findings. However, if they added reputational information, such that players could know what their partners did with others on previous rounds of play, the analysis predicted offers and rejections in the range of typical undergraduate responses. They concluded that the Ultimatum Game reveals humans’ species‐specific evolved capacity for fair and punishing behavior in situations with substantial reputation influence. But, since the Ultimatum Game is typically done one‐shot without reputational information, they argued that people make fair offers and reject unfair offers because their motivations evolved in a world where such interactions were not fitness relevant—thus, we are not evolved to fully incorporate the possibility of non‐reputational action in our decision‐making, at least in such artificial experimental contexts.

Recent comparative work has dramatically altered this initial picture. Two unified projects (which we call Phase 1 and Phase 2) have deployed the Ultimatum Game and other related experimental tools across thousands of subjects randomly sampled from 23 small‐scale human societies, including foragers, horticulturalists, pastoralists, and subsistence farmers, drawn from Africa, Amazonia, Oceania, Siberia and New Guinea (Henrich et al. 2005, Henrich et al. 2006). Three different experimental measures show that the people in industrialized societies consistently occupy the extreme end of the human distribution. Notably, small‐scale societies with only face‐to‐face interaction behaved in a manner reminiscent of Nowak et. al.’s analysis before they added the reputational information. That is, these populations made low offers and did not reject. [...]

Analyses of these data show that a population’s degree of market integration and its participation in a world religion both independently predict higher offers, and account for much of the variation between populations. Community size positively predicts greater punishment (Henrich et al. n.d.). The authors suggest that norms and institutions for exchange in ephemeral interactions culturally coevolved with markets and expanding larger‐scale sedentary populations. In some cases, at least in their most efficient forms, neither markets nor large population were feasible before such norms and institutions emerged. That is, it may be that what behavioral economists have been measuring in such games is a specific set of social norms, evolved for dealing with money and strangers, that have emerged since the origins of agriculture and the rise of complex societies.

There are many problems here.

At the end of paragraph 2 and the other examples, you say

This exactly mirrors the Prisoner's Dilemma.

But it doesn't, as you point out later in the post, because the payoff matrix isn't D-C > C-C > D-D, as you explain, but rather C-C > D-C > C-D, because of reputational effects, which is not a prisoner's dilemma. "Prisoner's dilemma" is a very specific term, and you are inflating it.

evolution is also strongly motivated [...] evolution will certainly take note.

I doubt that quite strongly!

The evolutionarily dominant strategy is commonly called “Tit-for-tat” - basically, cooperate if and only if you expect your opponent to do so.

That is not tit-for-tat! Tit-for-tat is start with cooperate and then parrot the opponent's previous move. It does not do what it "expects" the opponent to do. Furthermore, if you categorically expect your opponent to cooperate, you should defect (just like you should if you expect him to defect). You only cooperate if you expect your opponent to cooperate if he expects you to cooperate ad nauseum.

This so-called "superrationality” appears even more [...]

That is not superrationality! Superrationality achieves cooperation by reasoning that you and your opponent will get the same result for the same reasons, so you should cooperate in order to logically bind your result to C-C (since C-C and D-D are the only two options). What is with all this misuse of terminology? You write like the agents in the examples of this game are using causal decision theory (which defects all the time no matter what) and then bring up elements that cannot possibly be implemented in causal decision theory, and it grinds my gears!

And if two people with these sorts of emotional hangups play the Prisoner's Dilemma together, they'll end up cooperating on all hundred crimes, getting out of jail in a mere century and leaving rational utility maximizers to sit back and wonder how they did it.

This is in direct violation of one of the themes of Less Wrong. If "rational expected utility maximizers" are doing worse than "irrational emotional hangups", then you're using a wrong definition of "rational". You do this throughout the post, and it's especially jarring because you are or were one of the best writers for this website.

playing as a "rational economic agent" gets you a bad result

9_9

[...] anger makes us irrational. But this is the good kind of irrationality [...]

"The good kind of irrationality" is like "the good kind of bad thing". An oxymoron, by definition.

[...] if we're playing an Ultimatum Game against a human, and that human precommits to rejecting any offer less than 50-50, we're much more likely to believe her than if we were playing against a rational utility-maximizing agent

Bullshit. A rational agent is going to do what works. We know this because we stipulated that it was rational. If you mean to say a "stupid number crunching robot that misses obvious details like how to play ultimatum games" then sure it might do as you describe. But don't call it "rational".

It is distasteful and a little bit contradictory to the spirit of rationality to believe it should lose out so badly to simple emotion, and the problem might be correctable.

You think?

Downvoted.

This sequence is really great - thank you for writing it!

Game theory is particularly interesting because it adds up to normalcy so fast - very simple math on very simple situations very rapidly describes real life, macro-level behaviour.

Tom Siegfried has a powerful quote: Game theory captures something about how the world works. (note: bizarre HTML book setup, read pg 73-5)

Tangentially, I now understand exactly what I don't like about Eric S. Raymond's morality:

I am among those who fear... that the U.S. response to 9/11 was not nearly as violent and brutal as it needed to be. To prevent future acts of this kind, it is probably necessary that those who consider them should shit their pants with fear at the mere thought of the U.S.’s reaction.

... the correct response to a person who says “You do not own yourself, but are owned by society (or the state), and I am society (or the state) speaking.” is to injure him as gravely as you think you can get away with ... In fact, I think if you do not do violence in that situation you are failing in a significant ethical duty.

A precommitment to retribution is effective when dealing with "rational" agents or CDT agents. In fact a self-interested TDT agent in a world of CDT agents would do well by retaliating against all injuries with disproportionate force. (And also issuing extortionate threats; to be fair, ESR doesn't advocate this.) If you buy Gary Drescher's reduction of morality to decision theory, this is where the moral duty of revenge comes from. But a superrational agent in a world of superrational agents will rarely need to exercise retribution at all.

Human beings might be thought of as superrational agents who make mistakes (moral errors). I don't know of a good technical model for this, but I feel like one recommendation that would come out of it is to not punish people disproportionately, because what if others did that to you when you make a mistake?

The evolutionarily dominant strategy is commonly called “Tit-for-tat” - basically, cooperate if and only if you expect your opponent to do so.

No, Tit-for-Tat co-operates if and only if the other player co-operated last time. It works only in an iterated Prisoner's Dilemma, where you have multiple interactions with the same player.

Co-operate if and only if you expect the other player to co-operate (because of reputation, emotional behaviour etc.) is a quite different strategy. Strategies with some reputational or prediction element like this will work in cases where you have a one-shot interaction with each player, but multiple interactions with a community of players.

Strategies where you "commit" yourself to co-operating but only with players who have similarly committed themselves are different yet again, and work on true one-shot dilemmas. These are the "super-rational" strategies. Variants of these (TDT, UDT etc.) avoid the need for explicit commitments, since they always do what they wish they'd committed to doing.

Basically, I think you are mixing up quite different solutions here to the Prisoner's Dilemma.

EDIT: I see that Wedifrid made much the same points already. Sorry for the repetition.

The evolutionarily dominant strategy is commonly called “Tit-for-tat” - basically, cooperate if and only if you expect your opponent to do so.

That strategy is neither evolutionarily dominant nor "tit-for-tat". Tit-for-tat is applicable in the Iterated Prisoner's Dilemma with unknown duration and involves cooperating on the first round thereafter doing whatever the opponent did in the round before the current round. As the name implies it is somewhat like a specific implementation of "eye for an eye".

As for evolutionary dominance the strategy "cooperate if and only if you expect your opponent to do so" is strictly worse than "If cooperating gives the expected result of your opponent also cooperating then do so unless they will cooperate anyway". ie. Your proposed "completely-not-tit-for-tat" strategy cooperates with CooperateBot which is a terrible move. ("CooperateBot" may represent a much smaller or lower status monkey, for example.)

I like it, but suggest that you link back to the previous entry in the sequence and/or the sequence index.

You may want to be more careful about using game theory on real-world problems. Game theory makes a lot of assumptions (some explizit, others implizit) that most of the time are not given in real life.

You will even have a hard time to find good examples for real life prisoners who are in a game theoretic PD. In reality, most of the times the prisoners dilemma looks rather like this: Same payoff matrix as the classical PD, BUT both prisoners may chose to break their silence any time. Once a prisoner has confessed, there is no more going back to silence. This situation will probably yield cooperation (unless the prisoners cannot get accurate information).

• Most real-world situations do neither fit one-shot games nor iterated games. Real life rarely has discrete game turns.

• Many real-world situations are about behaviours that have some duration and can be stopped/changed.

• Many real- life situations permit/enable the player to change his decision from cooperation to defection if he learns about the defection of the other player. In many of these situations the betrayed player is able to change his own action fast enough to deny the defector any advantage from defecting.

Many uses of the word "rational" here were fine ("rational economic agent" is understandable), but others really bothered me ("It is distasteful and a little bit contradictory to the spirit of rationality to believe it should lose out so badly to simple emotion" -- why perpetuate the Spock myth? I want to show this to my friends!). I have no specific suggestion at hand, but circumlocuting around the word in some of the cases above would bring the article from excellent to perfection.

Excellent article, but the image is too wide so it gets cropped by LW's fixed-width content section. Looks like this. Using Firefox 9.0.

We all enjoy defecting of a salesman, who doesn't cooperate holding a price high, but defect and lower it to have a gain.

The defection in economy has its implication in this mechanism of pricing.

The defecting is just as crucial!

Prisoners' Dilemmas even come up in nature. In baboon tribes, when a female is in “heat”, males often compete for the chance to woo her. The most successful males are those who can get a friend to help fight off the other monkeys, and who then helps that friend find his own monkey loving. But these monkeys are tempted to take their friend's female as well. Two males who cooperate each seduce one female. If one cooperates and the other defects, he has a good chance at both females. But if the two can't cooperate at all, then they will be beaten off by other monkey alliances and won't get to have sex with anyone. Still a Prisoner's Dilemma!

If anything this is closer to Parfit's Hitchiker. While both are game theory problems and a good decision theory will result in cooperation on both of them (assuming adequate prediction capabilities) the non-sequential nature of the game matters and some decision making strategies will handle them differently.

Reputation is a way to change many one-time Prisonner's Dilemmas into one big Iterated Prisonner's Dilemma, where mutual cooperation is the best strategy for rational players. But how exactly does it work in real life?

I guess it works better when a small group of people interact again and again; and it works worse in a large group of people where many interactions are with strangers. So we should expect more cooperation in a village than in a big city.

Even in big cities people can create smaller units and interact more frequently within these units. So they would trust more their neighbors, coworkers, etc. But here is an opportunity for exploitation by people who don't mind frequent migration or changing jobs -- they often reset their social karma, so they should be trusted less. We should be also suspicious about other karma resetting moves, such as when a company changes their name.

Even if we don't know someone in person, we can do some probabilistic reasoning by thinking about their reference class: "Do I have an experience of people with traits X, Y, Z cooperating or defecting?" However, this kind of reasoning may be frowned upon socially, sometimes even illegal. It is said that people should not be punished for having a bad reference class because of a trait they didn't choose voluntarily and cannot change. Sometimes it is considered wrong to judge people by changeable traits, for example how they are dressed. On the other hand, reference classes like "people having a university diploma" are socially allowed.

Here is an interesting (potentially mindkilling) prediction: When it is legally forbidden to use reference classes and other forms of evaluating prestige, the rate of defection increases. (An extreme situation would be when you are legally required to cooperate regardless of what your opponent does.)

Another, mostly unrelated comment: the ultimatum game can actually tell you two different things. First, what divisions do people propose, and second, what divisions do people accept?

Presumably, everyone accepts fair divisions. Different groups of people have different percentages that reject unfair divisions, and different percentages that offer unfair divisions (a simplification, since the degree of fairness can also be varied). There are four potential clusters: groups that propose fair and accept unfair, groups that propose fair and reject unfair, groups that propose unfair and accept unfair, and groups that propose unfair and reject unfair. (Empirically, I believe only three of these clusters show up, but it'd take a lit search to verify that. The first two groups can be differentiated by having confederates propose unfair divisions)

These differences between people groups have real-world implications.

[edit] Note that Kaj_Sotala has found at least one paper on the subject.

If I defect but you cooperate, then I get to spend all day on the beach and still get a good grade - the best outcome for me, the worst for you.

! No, it's not. The "you" in this example prefers getting a good grade and privately fuming about having to do the work themselves to failing and not having to do the work. (And actually, for many of the academic group projects I've been involved in, it's less and happier work for the responsible member to do everything themselves, because there's too little work for too many people otherwise.)

The basic solution to the prisoner's dilemma: the only winning move is not to play. When you find a payoff matrix that looks like the prisoner's dilemma when denominated in years, add other considerations to the game until it is no longer the prisoner's dilemma when denominated in utility.

What's neat about that statement of the solution is that it naturally extends in every direction. Precommitments are an attempt to add another consideration- and their credibility is how much utility it looks like you can bestow to them. "My utility for punishing my daughter's defiler is higher than my utility for having clean carpets" is easy to believe, "my utility for keeping my commitments is higher than my utility for being $20,000 richer" is difficult to believe. (Scale should impact willingness to accept on an ultimatum game.)

Most people, when they hear the rational arguments in favor of defecting every single time on the iterated 100-crime Prisoner's Dilemma, will feel some kind of emotional resistance. Thoughts like “Well, maybe I'll try cooperating anyway a few times, see if it works”, or “If I promised to cooperate with my opponent, then it would be dishonorable for me to defect on the last turn, even if it helps me out., or even “Bob is my friend! Think of all the good times we've had together, robbing banks and running straight into waiting police cordons. I could never betray him!”

I don't want to start a discussion here, I just want to clarify your position on this: Are you implying that it is perfectly rational to always assume that your opponent is a perfect rationalist?

though not quite as good as me cooperating against everyone else's defection.

Shouldn't it be the other way around? (you defecting while everyone else cooperates)

ETA: liking this sequence so far, feels like I'm getting the concepts better now.

What is your source for your baboon anecdote? it is contrary to what I have read, eg, in Baboon Metaphysics. Or here:

Lower-ranking males form alliances and can harass newly immigrated but dominant males and protect adult females with which they have bonds. Overall, though, dominance rank of a male indicates reproductive success so that high-ranking males have more mating opportunities, more offspring, and increased fitness compared to lower-ranking males

So far as I can analyse, isn't a hostage negotiation a Prisoners Dilemma too? (Terrorists can spare (C) or kill (D), Government can pay (C) or raid (D))

Most people, when they hear the rational arguments in favor of defecting every single time on the iterated 100-crime Prisoner's Dilemma, will feel some kind of emotional resistance.

The rationalist strategy is not to defect from the beginning, but to cooperate till somewhere into the 90s