The Blackmail Equation

What's to stop the Countess from having precommitted to never respond to blackmail?

Or to have precommitted to act as though having precommitted to the course of action having precommitted to in retrospect seems the most beneficial (including meta-precommittments, meta-meta-precommitments, meta^meta^meta precommitments etc up to the highest level she can model)?

Which would presumably include not being blackmailable to agents who would not try to blackmail if she absolutely committed to not be blackmailable, but being blackmailable to agents who would try blackmail even if she absolutely committed to not be blackmailable, except agents who would not have modified themselves into such agents were in not for such exceptions. Or in short: Being blackmailable only to irrationally blackmailing agents who were never deliberately modified into such by anyone.

The fifth fact is a consequence of the previous ones.

Um, no. Again, I think you may have misunderstood that point there. The point is not that all Countesses can inevitably and inescapably be blackmailed. It is just that a Countess designed a particular way can be blackmailed. The notion of a superior epistemic vantage point is not that there is some way for the Baron to always get it, but that if the Baron happens to have it, the Baron wins.

Could the countess plausibly raise herself to a superior epistemic vantage over the baron, and get out from under his thumb? Alas no.

Again, this just wasn't a conclusion of the workshop. A certain fixed equation occupies a lower epistemic vantage. Nothing was said about being unable to raise yourself up.

Alas no. Once the countess allows herself to use tactics conditional on the baron's actions, the whole set-up falls apart: the two start modelling the other's actions based on their own actions which are based on the other's actions, and so on. The baron can no longer assume that the countess has no influence on his decision, as now she does, so the loop never terminated.

Or the Countess just decides not to pay, unconditional on anything the Baron does. Also, if the Baron ends up in an infinite loop or failing to resolve the way the Baron wants to, that is not really the Countess's problem.

As I did say at the decision workshop, the resolution that seems most likely is "respond to offers, not to threats".

If the Baron can commit to counterfactually doing Z, then he never has to do Z, so it doesn't matter how horrible the consequences of Z are to himself.

This is true, but you've neutered the prisoner's dilemma. One of the central problems one faces in game theory is that it is extremely hard to credibly precommit to do something that you'd clearly rather not do. Your point is valid, but you've assumed away almost all of the difficult parts of the problem. This is even more of a problem in your subsequent post on nukes.

(A few notes, with no particular point.)

The players should be modeled as algorithms that respond not to each other's moves, but as algorithms responding to information about the other player's algorithm (by constructing a strategy for responding to the other player's moves). In particular, saying that a certain player is a "rational agent" with certain payoff in the game might sometimes fix what that player's algorithm (of responding to info about the other player's algorithm) is. "It's a rational agent", if given as info about the player's algorithm to the other player, in the right games, may be taken as equivalent to the complete specification of this player's source code (modulo logical uncertainty; it's actually easier if we say "it's a rational agent" than if we give the source code, since in the latter case we may need to also make the inference -- hence you don't even need to be Omega to read "it's a rational agent" as exact specification of the algorithm, if the game is right). For this reason, it's unnecessary to actually communicate the source code, if it's a given that the players are "rational agents" and this knowledge can, by some magic, be communicated.

The game of considering counterfactuals has the goal of revealing semantics of the other player's algorithm: player A doesn't care what the source code of player B looks like, it only cares of what that source code does, that is how it reacts to possible players A (to what possible players A do, not to what possible players' source code is -- nobody is caring about the coding; and player A doesn't even know what its own code does -- this is what it's currently deciding; the coding only reflects logical uncertainty).

Note that each player has to maximize the payoff across all possible opponents, and not just the "rational agents" (particularly given that the coding for the "rational agent" may be unknown, even if the game has the "rational agent"). The other player may well be a random knot of wires (that still somehow processes the info about the other player's algorithm, maybe punishes the player if it cooperates with "rational agents"; no relation to the payoff matrix is possible at this point). The conserved resource here is expressive power: each player can't have an arbitrary table of answers to each of the possible other player's algorithms (the number of possible reactions to elements of set S is bigger than set S), so it's not possible in general to "step back" and improve on one's algorithm pointwise, for each of the possible other players (even if it were possible computationally). This suggests that many games won't have the unique "rational player" optimum, and it won't be enough to communicate "it's a rational player" to fix the algorithm or facilitate effective cooperation (for example, the ultimatum game, obviously).

Compare the notion of "rational agent" (as applied to more general games than PD) with stating that it's known about the other player that it's in your interest to make move X in response to that player.

I'm staying out of this discussion mainly because I'm incredibly confused about acausal/timeless/counterfactual trade/blackmail. Eliezer gave a small presentation at the recent decision theory mini-workshop on his ideas but unlike Stuart I'm pretty sure I don't understand it. I've been told there are also some very rough notes/drafts on related ideas written by a couple of individuals floating around SIAI and FHI, but so far I have been unsuccessful in getting access to them.

ETA: I should mention that the workshop was very enjoyable and I greatly appreciate SIAI's efforts in setting it up, even though I came out more confused than I did going in. That just means I wasn't confused nearly enough previously. :)

Sure to all that; but what I want to see is an explanation of why I should consider the notion of "superior epistemic vantage" a useful idea. It seems like a fantasy that has no bearing on real life. Why shouldn't I dismiss Newcomb's dilemma as philosophical masturbation the moment someone says that it depends on superior epistemic vantage?

Parfit does a good job of covering this ground in Reasons and Persons in his discussion of threat-fulfillers and threat-ignorers. Let's see... you can read most of that section on Google Books, starting on p. 20 (section 1 part 8). A "threat" is a claim by someone that they will do X if you do Y, where doing X would make both them and you worse-off. In a world of transparency (shared source codes), Parfit comes out in favor of threat-ignoring (as well as promise-keeping).

Sorry for such a late reply... Just so I understand the point, the point is "whoever can better model their opponent while themselves being able to then precommit to an action is the one in a superior epistemic vantage, whoever can go 'deeper' in the counterfactuals wins"?

ie, just so I understand, is the only thing stopping the countess from doing the same sort of counterfactual modelling on the baron, from having strategies that are functions of which strategy the baron chooses being that simply "by assumption, the countess's code/computational resources are lesser"? Or is there something about the situation that inherently leads to the countess having fewer options, however sophisticated she is and however much computational resources available to her?

(first, sorry for replying to this thing so late. I was a bit ill for a while, wasn't really up to tackling this/thinking about it)

Near as I can make out, one ends up with a "precommitment arms race"

ie, what stops the countess from credibly precommiting to "If you blackmail me, I will not only not pay, but will immediately and publicly reveal all myself"? And then the baron will have a way of precommiting to "no matter what, I'm going to blackmail you", and basically one ends up with a precommitment race until one side or the other blinks or they both make each other miserable. (or did I completely miss the point?)

The most plausible version would go: "If you reveal the affair, I will shoot you."

This is of course contingent on the Countess being aware of this limitation on the Baron (i.e. she knows he will not reveal, forcing him to choose either not-a-word or anti-blackmail, as these are the only precommitments that would result in her not forcing him to reveal). I am therefore fairly certain the Archduke does not need an epistemic advantage; he merely needs the ability to make one outcome unacceptable to the party with an epistemic advantage, and make this fact known to both parties.

Granted, if the Archduke must keep his intervention unknown to his sister, your solution seems like the simplest one that would work. In my solution, the Countess must know that the payoff of [Reveal] has changed for the Baron, or he would choose blackmail, she would pay him, and the Archduke would not shoot him.

In the world of perfect decision makers, there is no risk to doing so, because the Countess will hand over the money, so the Baron will not take the hit from the revelation.

Since the payoffs are symmetrical (in sense that both baron and countess lose the same amount of utility if the secret is revealed), in the world of perfect decision markers the winner is clearly the person who decides to blackmail. But the countess can retaliate easily: after she pays, she can blackmail the baron, and get the money back with certainty, even with some bonus. So, say that the baron has Xb money, and prefers having at least Lb money to keeping the secret (this is the lower bound; it can be 0 if the baron is willing to pay anything he has). For the countess, the analogous quantities are Xc and Lc.

1st step: The baron blackmails, and rationally demands Xc-Lc money. Countess pays. After that, the baron has Xb+Xc-Lc, the countess has Lc. Now, the baron knows that the countess will reject any further threat. 2nd step: The countes blackmails the baron, and demands Xb+Xc-Lc-Lb money. Baron, obviously, pays. After that, the baron is at Lb, while the countess is at Xb+Xc-Lb. 3rd step is obvious.

Now, both the countess and the baron are reasonable enough that they can predict the outcome being an infinite oscillation of the bank account, and that the result is only loss of time. (Assume that it is impossible for the countess to trick the baron by spending the money somehow to get some utility while lowering her accessible possessions to Lc, or vice versa). So, what is the solution in the timeless decision theory? How would the ideally rational baron and countess behave?