Prisoner's Dilemma vs the Afterlife

Ah, now that's a lovely wrinkle I hadn't thought of, which could easily overwhelm any of the effect described in the root post here; and would at least need to seriously think about.

(My not having thought of it may from be the same sort of blind spot that arises from psychological studies tending to be performed on Amero/European college students...)

I would love to learn what your undergrad course taught you; if you'd rather point me at a resource to read than explain it, that would be fine.

The 5% thing could be closely approximated by taking a million-turn game, averaging the score to a typical per-round value, and comparing it to other 'infinite' games, to see which has the best long-term life enjoyment.

What you called "behave morally", I tend to think of in PD terms as 'being nice': not being the first to defect.

As a first thought, using honor as a virtue seems to be a way of replacing the ordinary set of rewards with a new scoring system - that is, valuing the honor of not being a thief over stealing a bunch of gold coins from an unlocked chest. I'm not entirely sure how to look at that in the evo-psych manner, how such an idea would arise, spread, and develop over time; but it seems like a workable alternative, for whatever portion of the population can be convinced being honorable is more important than the rewards from dishonorable behaviour.

A lot, yes; but a lot don't, either.

Also, present-day society is the result of millennia of previous societies which all included, at least implicitly, a supernatural afterlife-belief; likely going all the way back to Neanderthals burying their dead with red ochre. This leads to a somewhat different environment in which people develop their preferences than the environment in which an afterlife-belief originally formed, or developed its complications.

Suicide attacks are an extreme end of a spectrum, which also includes 'brave young warriors posturing against the tribal enemy to show how fierce they are, and make them back down' and 'going off to war'. Young men are, in many respects and to a certain degree, a disposable resource - you don't /need/ that many men to keep your in-group's population high (especially if your culture explicitly allows for polygyny), and your overall in-group can gain certain benefits from spending that resource. Should those young men believe in an afterlife, they would be more willing to be spent, thus potentially increasing their in-group's overall success.

Re well-trodden ground: Well, yes, but as far as I can find, mainly in a vague, emperical, qualitative way. I'm looking for the /numbers/.

The second: that afterlife-memes allowed better cooperation; then, after becoming sufficiently widespread to be taken for granted, variants that depended on the basic idea but allowed even greater cooperation (eg, that He'll punished defectors) would similarly take root, spread, and become a new foundation.

Funny, when talking to Patrick at the workshop I made pretty much the opposite point. Maybe worth spelling it out here, since I came up with Lobian cooperation in the first place:

The PD over modal agents is just another game-theoretic problem. As the zoo of proposed modal agents grows, our failure to find a unique "rational" modal agent is a reflection of our inability to find a unique "rational" strategy in an arbitrary game. Waging war on an established result is typically a bad idea, we probably won't roll back the clock on game theory and reduce n-player to 1-player. This particular game is still worth investigating, but I don't hope to find any unique notion of rationality in there.

Without a unique notion of rationality, it seems premature to say that rational agents won't play a game in a certain way. Who knows what limitations they might have? For example, PrudentBot based on PA will defect against PrudentBot based on PA+1.

I've only had time to read the introduction so far; but if it's not mentioned in the paper itself, it seems that PrudentBot should not only be "correct" if it defects against CooperateBot, it should also defect against DefectBot. In fact, in a one-shot PD, it seems as if it should defect against any Bot which is unable to analyze its own source code to see how it will react.

It seems as if there's an important parallel between the Iterated Prisoner's Dilemma and the One-Shot Prisoner's Dilemma With Access To Source Code: both versions of the PD provide a set of evidence which each side can use to attempt to predict the other's behaviour. And since the PD-with-source is, according to the paper, equivalent to Newcomb's Problem, this suggests that the Iterated-PD is equivalent to a variant of Newcomb's based on reasonably-available historical evidence rather than Omega-level omniscience about the other player.

This also suggests that an important dividing line between algorithms one should defect against, and algorithms one should cooperate with, is somewhere around "complicated enough to be able to take my own actions into account when deciding its own actions". For PD-with-source, that means being complicated enough to analyze source code; Iterated-PD's structure puts that line at tit-for-tat.

This is also implying a certain intuitive leap to me, involving species with complicated enough social interactions to need to think about others' minds (parrots, dolphins, apes); that the runaway evolutionary process that led to our own species perhaps has to do with such mind-modeling finally becoming complicated enough to model one's own mind for higher-level social plots... but that's more likely than not just some college-freshmen-level "say, what if..." musing. It could be just as likely that the big step forward was minds becoming complicated enough to become less-predictable black-boxes than simple predictable "if I cheat on him and he catches me, he'll peck me painfully" call-and-responses.

Actually, Prisoner's Dilemma between programs with each other source code (aka the program equilibrium setting) is a much different problem than both oneshot and iterated Prisoner's Dilemma.

The main attractiveness of both oneshot and iterated PD is that, despite their extreme simplicity, they provide a suprisingly good model of many common problems humans face in real-world social interactions.
On the other hand, Program PD is a much more artificial scenario, even in a speculative world of robots, or world of software agents. It is theoretical interesting to analyze that scenario, but even if an unabiguously satisfactory solution for it was found (and I think your paper doesn't, but I'll leave that for another post * ), it would be far fetched to claim that it would essentially solve all practical instances of PD.

( * ) Short story: PrudentBot cooperates too much. (PrudentBot, PrudentBot) is an unstable payoff-dominant Nash equilibrium. It's not even a Nash equilibrium under a small modification of the game that penalizes program complexity.
CliqueBots and generalized CliqueBots (bots that recognize each other with a criterion less strict than perfect textual equivalence but strict enough to guarantee functional equivalence) are better since they are stable payoff-dominant Nash equilibria and they never fail to exploit an exploitable opponent.

That's the same as just tit-for-tat.

It depends on what you know about the person you are playing with. You do know something, even if it is just a probability distribution. Different distributions give you different strategies.

A transhumanist, singulatarian, or cryonicist might be able to say that they do have /something/ which is close enough to fit into the appropriate slot, thus raising their standing from 'untrustworthy barbarian' to 'semi-trustworthy member of not-my-religion'.

But aren't cryonicists engaging in precisely the sort of defection that we are proposing that belief in the afterlife combats? At the end of their natural lives, they're using their power to pool resources unto themselves, rather than passing it on to the unambiguously living. Most people donate everything to children or charity after death. Cryonics is hardly effecting altruism.

Note - The above statement is not to be interpreted as opposition to cryonics.

Perhaps; but few people seriously believe that they're going to live to be 150 years old; and as long as there is /some/ maximum length to a PD tourney, the reasoning from the last-possible-move would still seem to apply.

I think you've probably influenced the experiment by making that comment.

This is the reasoning behind tit-for-tat, and which allows it to succeed - when it does succeed.

However, in a crowd of always-defectors (such as a set created by reasoning from a known length-of-game), tit-for-tat's initial cooperation means it will have a slightly lower score than the crowd which always defects; tit-for-tat needs the cooperation of one or more other not-always-defectors to build up enough of an advantage amongst themselves to overcome that.