Strongmanning Pascal's Mugging

The scariest version of Pascal's mugging is the mugger-less one.

Very many hypotheses -- arguably infinitely many -- can be formed about how the world works. In particular, some of these hypotheses imply that by doing something counter-intuitive in following those hypothesis, you get ridiculously awesome outcomes. For example, even in advance of me posting this comment, you could form the hypothesis "if I send Kindly $5 by Paypal, he or she will refrain from torturing 3^^^3 people in the matrix and instead give them candy."

Now, usually all such hypotheses are low-probability and that decreases the expected benefit from performing these counter-intuitive actions. But how can you show that in all cases this expected benefit is sufficiently low to justify ignoring it?

p(d(N)) * U(UTM(d(N))

It is considered a good form to summarize your main point in a reasonably plain English, even for a complicated research paper, let alone a forum post. Give it a try, you might gain some broader audience.

Moreover, have you tried to simplify your problem, distill it to its essence? Find the extra piece that makes it non-trivial?

Another standard step is looking up the existing literature and referencing it. The issues of Pascal Mugging, unbounded utility functions and largest computable integers have been discussed here recently, are none of them relevant to your problem?

Isn't that steel-man, rather than strong-man?

This steelman argument doesn't address the main issue with Pascal's mugging - anyone in real life who makes you such an offer is virtually certain to be lying or deluded.

The mugger should have some non-zero probability, p, for which zhe is indifferent between p"have $10 after fulfilling the deal" and (1-p)"have $5 now".

There is no reason to suppose this, and in fact it's unlikely, since the mugger probably isn't motivated by money (since he surely has better ways of obtaining that). In the least convenient world, he's probably just curious to know how you'll answer.

The standard solution is to have a bounded utility function, and it seems like it fully solves this reformulation as well. There may also be some other solutions that work for this, but I'm sufficiently unsure about all the notation that I'm not very confident in them.

Also, just to check, have you seen the Lifespan Dilemma?

A mugger appears and says "For $5 I'll offer you a set of deals from which you can pick any one. Each deal, d(N), will be N bits in length and I guarantee that if you accept d(N) I will run UTM(d(N)) on my hypercomputer, where UTM() is a function implementing a Universal Turing Machine. If UTM(d(N)) halts you will increase your utility by the number of bits written to the tape by UTM(d(N)). If UTM(d(N)) does not halt, I'll just keep your $5. Which deal would you like to accept?"

That's unclear. Does the mugger show you the set of deals before you accept the offer or later? If he shows you later, he can trivially win by presenting you the set { 'while (true) {printf("I Go Chop Your Dollar\n");}' }

where p(d(N)) is the probability of accepting d(N)

d(N) either halts or it doesn't. There is nothing probabilistic about it. Are you talking about the prior probability about its termination before you see d(N), or after you see it (since you can't compute its termination in the general case)?

p(UTM(d(N)) = BB(N)) * BB(N) > 0. To paraphrase: Even though the likelihood of being offered a deal that actually yields BB(N) utilons is incredibly small, the fact that BB(X) grows at least as fast as any function of length X means that, at minimum, an agent that can be emulated on a UTM by a program of M bits cannot provide a non-zero probability of d(M) such that the expected utility of accepting d(M) is negative.

To paraphrase nothing.
p(UTM(d(N)) = BB(N)) BB(N) > 0 is trivially true because the product of two positive numbers is always positive. But this tells you nothing about the magnitude of the value. In particular, it doesn't follow that
p(UTM(d(N)) = BB(N)) BB(N) >= U($5)

Since p("UTM(d(X)) = BB(X)") >= 2^-X for d(X) with bits selected at random it doesn't make sense for the agent to assign p(d(X))=0 unless the agent has other reasons to absolutely distrust the mugger.

I wonder what that reasons to distrust the mugger may be... Perhaps that he wants to win your $5 rather than giving away utility?

Question: Is Pascal's Mugging Isomorphic to the Two General's Problem, or am I confused?

I tried to start making a comparison between them, since each additional Messenger should grant you Utility, with Busy Beaver levels of Utility being Busy Beaver levels of Messengers, but the conclusion I came to is even if you trust the other person 100%, can never actually be safe on the on the attack/mugging unless the other person says "I will attack/pay with Certainty, regardless of any replies you send." and then does.

At that point, the worst thing that can happen to you is that the other general gets slaughtered because you didn't hear or chose not to trust the other general. The best possible result for you is still for them to be the one that takes the risk of choosing to move first with certainty so either way you get as good a result as you can.

The Pascal's Mugging equivalent of this would seem to be for the mugger to appear and say "I am going to take a chance such that the first thing I do is that I give you a small, small chance of Fabulously large utility, and I'm going to do that regardless of whether you pay me or not. But after I DO that... I really need you to send me 5 dollars."

But that doesn't seem like a mugging anymore!

I guess that means a possible reply is essentially "If this offer is so good, then pay me first, and then we'll talk."

If they resist or say no, then yes, you can just reply "But there's got to be SOME payment I can offer to you so that you move first, right?" But that's basically the offer they made to you initially!

If they are isomorphic, it makes sense that we would have trouble solving it, since there IS no solution to the two generals problem, according to the wiki:

http://en.wikipedia.org/wiki/Two_Generals%27_Problem

If they aren't isomorphic, that's weird and I am confused, because they have a lot of similarities when I look at them.

Facts not yet presented: How many utilions are lost by giving away $5? How quickly does the UTM operate (if it operates at ~10^10 operations per second, but the normal interest on $5 provides 10^100 utilions per second, you lose even if you get your money back at the end; if it operates at 1 operation per year, you presumably lose if you die before it halts.)