Explicit Optimization of Global Strategy (Fixing a Bug in UDT1)

What about when there are agents with difference source codes and different preferences?

Set aside different preferences, because I have no idea how to deal with that. It seems like just having different source codes makes things very difficult.

You used "1" and "2" as your indexical labels, which come with a very obvious order. But suppose that Omega labeled one agent ♠ and the other agent ♣. If you and your opposite have different source codes, you have no idea how your opposite internally represents the symbol that they received. For example, you certainly have no guarantee that the two of you order these in lexicographically the same way. So how could you possibly coordinate on a tie-breaking strategy?

You want a general algorithm that plays arbitrary games against other copies of itself "fairly". If utilities are transferable, it's easy, but note that this solution wasn't derived from consideration of logical correlations alone. If utilities are non-transferable, I've given up on the problem because the literature offers bewilderingly many different solutions and no clear winner.

You could try a simpler problem as a stepping stone: blame assignment. (I'm currently trying to solve this myself for a future post but what the hell, let a million flowers bloom.) In a given play of a multiplayer game, how much credit/blame should we assign to player A for the payoff that player B got? Concrete example: how many people did one voter personally kill by voting for Hitler and how is this responsibility shared between them and Hitler? I believe this should be easier than NTU fairness but it's still too tricky for me.

This is essentially PD (in the aspects relevant to this post), but without magical identification of Player1.Cooperate and Player2.Cooperate by virtue of using the same label "Cooperate". Consider what happens with your thought experiment if the rule given by Omega is "You should give the same answer", given that they already have differentiating info (assigned number). This distinction shouldn't matter.

Recursion through decision-making of all relevant agents seems conceptually indispensable. When Player1 sees itself again through the eyes of Player2, symmetry or asymmetry between the players (as opposed to identity of the recursive copies of each player) becomes irrelevant.

Consider: if players are different, then Player1 knows about Player2 that knows about Player1 (recursion, a site of TDT-like acausal control); if players are the same, then Player1 knows about identical Player2 (recursion at the first step). If we go the first road, not much is lost, but we get more generality.

This was my thought process: To get the $10, my copy and I have to choose differently. I am 1, he is 2. I have to choose A or B... At some point I thought of the mapping A=1, B=2, implicitly as part of the bigger mapping (A...Z)=(1...26) I suppose. I noticed that this was a particular mapping which had spontaneously presented itself to me. So it must be a natural one for me to think of; so there is a good chance my copy will think of it as well. So I select A, hoping my copy went through an analogous process, arrived at the same mapping and selected B.

If only Omega let you use quantum pseudo-telepathy.

What about when there are agents with difference source codes and different preferences? The result here suggests that one of our big unsolved problems, that of generally deriving a "good and fair" global outcome from agents optimizing their own preferences while taking logical correlations into consideration, may be unsolvable, since consideration of logical correlations does not seem powerful enough to always obtain a "good and fair" global outcome even in the single-player case.

I don't understand this statement. What do you mean by "logical correlations", and how does this post demonstrate that they are insufficient for getting the right solution?

Suppose you're choosing a strategy S for a cooperation game with some other entity X, which you are told nothing about. Then U(S) = .5 (S(1)!=X(2)) + .5 (S(2)!=X(1)) In this case, you have to choose a probability distribution over other entities X, and choose S to optimize the utility function based on that. There's no way around that. If we're told that X was given the same utility function, and is trying to optimize over it, then that greatly narrows down the possibilities for what X is. We assume that X is chosen, by some unspecified but intelligent process, to also optimize U. Fortunately, English culture provides a standard mapping between numbers and letters (A=1, B=2, C=3, ...); so if we assume X has some probability of coming from a similar culture and choosing that mapping for that reason, and will choose an arbitrary random mapping otherwise, then we're better off with the standard mapping.

If the other agent has a different utility function, then that changes your probability distribution over what that agent is. If we're told that the other agent is supposed to implement the utility function "1 if it chooses A, 0 if it chooses B", then its implementation is probably going to be to just return A, so we should always return B.

Now assume that when we enter into the coordination game, we're told something about A, and A is told something about us. Then our utility function is U(S) = .5(S(1,X)!=X(2,S)) + .5(S(2,X)!=X(1,S)) We still need a probability distribution over Xs, but this time the distribution includes Xs that model S. If we're also told that X is supposed to be optimizing the same utility function, then we can assign some probability to it modeling S with each of various techniques, and to it being model-able with each of various techniques. Not all modeling techniques will work on all functions - some of them lead to infinite regress, some are just bad designs that can't model anything accurately, etc - so to maximize the probability of successful coordination we should both make S easy for X to model, and make S try to model X.

Different kinds of games lead to different kinds of likely opponents, hence the field of game theory. A nash equilibrium is any pair of strategies that optimize utility under the assumption that the other is their opponent.

Does UDT not allow randomization, or is Omega just disallowing it in this case?

EDIT: Or does it turn out to be irrelevant?