Pascal's Mugging for bounded utility functions

The information required to describe your body is about an exabyte. Once you have a simulated body, getting answers out is trivial, so we'll call an exabyte an upper limit on what information you could tell someone. 10^18 ish. This means that if you have a utility function, you aren't able to imagine situations complicated enough to have a simplicity prior below 10^-10^18. That is, one part in 1 followed by 10^18 zeroes.

Hm, that's an interesting point. On the other hand, "Robin Hanson has suggested penalizing the prior probability of hypotheses which argue that we are in a surprisingly unique position to affect large numbers of other people who cannot symmetrically affect us. Since only one in 3^^^^3 people can be in a unique position to ordain the existence of at least 3^^^^3 other people who can't have a symmetrical effect on this one person, the prior probability would be penalized by a factor on the same order as the utility." (source: LW wiki; I couldn't find where Robin actually said that)

In other words, you can represent the hypothesis with so little information because you can cheat by referring to yourself with a small amount of information, no matter how much information it would take to specify you objectively.

That runs into problems - like you'd dump toxic waste in your house as long as you only got sick far in the future.

Why?

An exabyte? Really? 8e18 bits?

(Most values to one sig fig)

Estimate 4e28 total nucleons in the human body (60Kg of nucleons) it takes about 100 bits to describe the number of nucleons. each nucleon takes about two bits of information to specify type (proton, neutron, antiproton, antineutron) Figure that a human is about 6'x2'x1'; that's 1e35x4e34x2e34 plank units. With 8e108 unique locations within that rectangle, each nucleon needs 362 bits to describe location.

Without discussing momentum, it already takes 10^41 bits to describe the location of every nucleon. Any benefit you gain from the distribution of matter not being uniform you lose when describing what the distribution of matter is in a human.

With a discussion of momentum, there is no way to describe an arbitrary nucleon in fixed space, since there is no upper limit to the amount of momentum of a nucleon.

How did you estimate an exabyte for the information content of a human?

Not necessarily. Future!you doesn't have much control over events that happen in its past, so you don't have much incentive to ensure that it cares about them. Also, you know that yourself 4^^^^4 years from now will not actually be only a billion years old.

I realized that after thinking a little longer. There's nothing from preventing me from believing "There's an Omega who will give me 4^^^^4 years of fun if I decrease my utility by X right now."

One way to approach it is by the likelihood that such a hypothesis could pay rent, in which case the possibility of trade with the Omega is a way to test the truth of the statement. If there's an entity that can dramatically improve my life I would be willing to pay for it with reasonable economic means. If that entity refuses all sane economic means it dramatically lowers the probability that it is a true offer.

Another approach is to consider the opportunity cost of paying this particular entity some of my negative utility when some other entity may offer a better deal. I assume there are an uncountable amount of Omegas who might offer me deals for every possible future utility if I give up some utility now. I only have so much utility that I can reasonably give up right now. Which Omega(s) do I pick to give up my utility to? I certainly shouldn't choose the most expensive Omega that yields the lowest expected return; I should pick the most efficient Omega for my cost in negative utility. What does that Omega look like? Certainly not one that offers 4^^^^4 years of fun for $5, but one that offers 4^^^^4 years of fun for less than $5 (or even pays me to experience it). Perhaps I only have $5, and spending it now would prevent me from ever accepting another deal from a different Omega. Ultimately what this hypothetical choice reduces to is choosing the earliest likely payoff so that I have more money in the future to consider more deals, e.g. skip out on paying Omega $5 today when I could invest it in the local economy and earn $6 tomorrow and have even more utility to pay potential Omegas with. I have to assume that any particular instance of Omega is roughly as likely as any other; a mugger appearing and offering me a purported deal from a particular Omega is no proof that the particular Omega exists unless there is additional proof such as being able to take out a loan repayable in the distant future or a verifiable insurance policy against the failure of Omega to deliver on the promise.

EDIT: I should probably try to quantify this. U(deal_N)*P(deal_N_is_true) - U(cost_N) is the expected utility of giving up cost_N utility in exchange for deal_N utility from a potential entity Omega_N. Unless I have some evidence that P(deal_X_is_true) is greater than P(deal_N_is_true) for some X and every other N, I should simply pick the most efficient deal. If P(deal_X_is_true | evidence_X_is_true)*U(deal_X) - U(cost_X) > U(deal_N)*P(deal_N_is_true) - U(cost_N) for some X and all other N, then I should pick X. Is a mugger appearing and offering me a particular deal, X, evidence that X is true? No, unless the mugger (or another entity I can trade with, including myself if I find actual evidence that X is true) also believes the deal sufficiently strongly to sell me insurance. In other words the mugger has to believe the deal will pay rent in order for me to believe it will pay rent.

Just to confirm... I take it from this that you consider a large universe intrinsically more valuable than a small one, and therefore any value system that leads one to choose a small universe doesn't capture your values. Have I understood you?

It can only apply significant marginal weight for so long. Just weight until it gets to 100-epsilon utils, and then mug.

Yes, that was my point (maybe should have been more explicit about it).

Though it's also worth pointing out that with a utility function like Carl is alluding to (where utilities are significantly different if the lifespans are noticeably different to humans), if you've lived for 3^^^3 years and the universe looks capable of supporting 2^(3^^^3) years of fun but not more than that, you will worry about the tiny probability of TREE(100) more than about realizing the 2^(3^^^3) that are actually possible.

Presumably the idea is that 3^^^3 is computable by a fairly short program

Irrelevant.

Solomonoff induction evaluates programs that generate strings of possible future observations conditioned on all your past observations (or equivalently, programs that generate strings of past and future observations, and counts only those that generate a string whose prefix matches your previous observations).
These programs correspond to models of the universe (observed from your particular point of view).

Models of the universe where life goes on as we know it until at a certain point and then suddenly you get 4^^^^4 years of fun are presumbly much more complex than models of the universe where life keeps going on as we know it.

In particular, if Solomonoff induction assigns to models where starting from now you get X years of fun a total probabilty p(X) that asymptotically decreases fast enough with X so that U(X) * p(X) also decreases, it will not be subject to Pascal's mugging.
All reasonable light-tailed probability distribution would have that property w.r.t. any reasonable utility function.
I see no reason to believe that the Solomonoff distribution would behave otherwise.