The I-Less Eye

After step one, you have a 50% chance of finding yourself the original; there is nothing controversial about this much.

That's not the way my subjective anticipation works, so the assertion of uncontroversialness is premature. I anticipate that after step one I have a 100% chance of being the copy, and a 100% chance of being the original. (Which is to say, both of those individuals will remember my anticipation.)

Here’s another, possibly more general, argument against subjective anticipation.

Consider the following thought experiment. You’re told that you will be copied once and then the two copies will be randomly labeled A and B. Copy A will be given a button with a choice: either push the button, in which case A will be tortured, or don’t push it, in which case copy B will be tortured instead, but for a longer period of time.

From your current perspective (before you’ve been copied), you would prefer that copy A push the button. But if A anticipates any subjective experiences, clearly it must anticipate that it would experience being tortured if and only if it were to push the button. Human nature is such that a copy of you would probably not push the button regardless of any arguments given here, but let’s put that aside and consider what ideal rationality says. I think it says that A should push the button, because to do otherwise would be to violate time consistency.

If we agree that the correct decision is to push the button, then to reach that decision A must (dis)value any copy of you being tortured the same as any other copy, and its subjective anticipation of experiencing torture ends up playing no role in the decision.

Eliezer wrote that we should make beliefs pay rent in anticipated experiences, and I think we should also make anticipation pay rent in correct decisions. It seems that is only possible under a limited set of circumstances (namely, with no mind copying).

Are you convinced yet there is something wrong with this whole business of subjective anticipation?

I'm not sure what this "whole business of ... anticipation" has to do with subjective experience.

Suppose that, a la Jaynes, we programmed a robot with the rules of probability, the flexibility of recognizing various predicates about reality, and the means to apply the rules of probability when choosing between courses of action to maximize a utility function. Let's assume this utility function is implemented as an internal register H which is incremented or decremented according to whether the various predicates are satisfied.

This robot could conceivably be equipped with predicates that allow for the contingency of having copies made of itself, copies which we'll assume to include complete records of the robot's internal state up to the moment of copying, including register H.

The question then becomes one of specifying what, precisely, is meant by maximizing the expected value of H, given the possibility of copying.

Suppose we want to know what the robot would decide given a copy-and-torture scenario as suggested by Wei Dai. The question of "what the robot would do" surely does not depend on whether the robot thinks of itself as rational, whether it can be said to have subjective anticipation, whether time consistency is important to it, and so on. These considerations are irrelevant to predicting the robot's behaviour.

The question of "what the robot would do" depends solely on what it formally means to have the robot maximize the expected value of H, since "the value of H" becomes an ambiguous specification from the moment we allow for copying.

(On the other hand, was that specification ever unambiguous to begin with?)

If the robot is programmed to construe "the value of H" as meaning what we might call the "indexical value" of H, that is, the value-held-by-the-present-copy, then it (or rather its A copy) would presumably act in the torture scenario as Wei Day claims most humans would act, and refuse to press the button. But since the "indexical value of H" is ill defined from the perspective of the pre-copying robot, with respect to the situation after the copy, the robot would err when making this decision prior to copying, and would therefore predictably exhibit what we'd call a time inconsistency.

If the robot is programmed to construe "the value of H" as the sum (or the average) of the indexical values of H for all copies of its state which are descendants of the state which is making the decision, then - regardless of when it makes the choice and regardless of which copy is A or B - it would decide as I have claimed a one-boxer would decide.(Though, working out these implications of the "choice machine" frame, I'm less sure than before of the relation between Wei Dai's scenario and Newcomb's problem.)

While writing the above, I realized - this is where I'm driving at with the parenthetical comment about ambiguity - that even in a world without copying you get to make plenty of non-trivial decisions about what it means, formally, to maximize the value of H. In particular, you could be faced with decisions you must make now but which will have an effect in the future and whose effect on H may depend on the value of H at that time. (There are plenty of real-life examples, which I'll leave as exercise for the reader.) Just how you program the robot to deal with those seems (as far as I can tell) underspecified by the laws of probability alone.

A shorter way of saying all the above is that if we taboo "anticipation", when predicting what a certain class of agent will do, we don't necessarily find that there is anything particularly strange about a predicate saying "the present state of the robot is copy N of M". What we find is that we might want to program the robot differently if we want it to deal in a certain way with the contingency of copying; that is unsurprising. We also find that subjective experience needn't enter the picture at all.

For me to wake up as Britney Spears, would mean the atoms in her brain were rearranged to encode my memories and personality... If that isn't what we mean, then we are presumably referring to a counterfactual world in which every atom is in exactly the same location as in the actual world. That means it is the same world. To claim there is or could be any difference is equivalent to claiming the existence of p-zombies.

I know p-zombies are unpopular around here, so maybe by 'equivalent' you merely meant 'equivalently wacky', but it's worth noting that the view you rightly dismiss here is quite different (and arguably more radical) than Chalmersian property dualism. After all, phenomenal 'qualia' are still qualitative (descriptive) features of a world, whereas you're imagining someone who thinks that there can be differences in numerical identity even between qualitatively identical worlds. (In academic circles this view is called 'haecceitism'.)

Anyway, as a property dualist who shares your anti-haecceitism, I just found it a bit strange to see you describe haecceitism as 'equivalent' to property dualism, and so wanted to clarify that the two views are actually quite independent.

Upvoted, but the Boltzmann problem is that it casually looks like the vast majority of subpatterns that match a given description ARE Boltzmann Brains. After all, maxentropy is forever.

After step one, you have a 50% chance of finding yourself the original; there is nothing controversial about this much. If you are the original, you have a 50% chance of finding yourself still so after step two, and so on. That means after step 99, your subjective probability of still being the original is 0.5^99, in other words as close to zero as makes no difference.

The way subjective probability is usually modeled, there is this huge space of possibilities. And there is a measure defined over it. (I'm not a mathematician, so I may be using the wrong terminology, but what I mean is that every 'sufficiently nice' subset of this set of possibilities has a number attached which behaves something like an area for that subset of the space.)

And then, in this model, the probability of some proposition is the measure of the subset where the proposition is true divided by the measure of the whole set. Numerator and denominator. And then each time you learn something, you throw away all of the points in that space that are no longer possible. So, you have typically decreased (never increased) both numerator and denominator. Do the division again and get the new updated probabilities. The space of all possibilities only loses points and measure, it never gains.

But I am not so sure this rule still applies when copying is involved. I think that each time you copy, you need to duplicate the subjective space of possibilities. The original space covered the measures of possibilities from one subjective viewpoint. At the point of copying, that space is duplicated because you now have two viewpoints. Initially, both original and copy are unsure which half of the space is theirs. But when they find out, they each throw out half of the doubled space. And then, as they learn more, possibilities are thrown away from one or the other of the spaces and each one updates to his own subjective probabilities.

So how does this apply to the copying scenario above? Start with one universe. Copy it when you copy the person. Produce a second copy when you produce the second copy of the person. Produce the 99th copy of subjective reality when you produce the 99th copy of the person. If at any stage, one of these persons learns for sure which copy is his, then he can prune his own subjective universe back to the original size.

So, if the protocol is that after each copying, the copy is told that he is a copy and the original is told that he is the original, then before any copying, the person should anticipate being told "You are original" N times, where N is between 0 and 99 inclusive. And he should attach equal probability to each of those events. That is, he should be 99 to 1 sure he will be the original the first time, 98 to one sure the second time, etc.

Forgive me if this is already known as one of the standard approaches to the problem.

This is really well written. I hope you post more.

Again, probability doesn't work with indexical uncertainty.

I wrote a response to this post here [LW · GW]:

In order to solve this riddle, we only have to figure out what happens when you've been cloned twice and whether the answer to this should be 1/3 or 1/4. The first step is correct, the subjective probability of being the original should be 1/2 after you've pressed the cloning button once. However, after we've pressed the cloning button twice, in addition to the agent's who existed after that first button press, we now have an agent that falsely remembers existing at that point in time.

Distributing the probability evenly between the agent's who either had that experience or remember it: we get a 1/3 chance of being a false memory and a 2/3 chance of it being a real memory. If it is a real memory, then half of that - that is a 1/3 - is the probability of being the original and the other half - also 1/3 - is the chance of being the first clone.

So, the answer at the second step should be 1/3 instead of 1/4. Continued application will provide the answer for 100 copies.

Can you win the lottery by methods such as "Program your computational environment to, if you win, make a trillion copies of yourself, and wake them up for ten seconds, long enough to experience winning the lottery. Then suspend the programs, merge them again, and start the result"?

I would much rather make a billion clones of myself whenever I experience great sex with a highly desirable partner. Point: making the clones to experiencing the lottery is about the experience and not the lottery. I'm not sure I particularly want to have orgasmic-clone-mes but if for whatever arbitrary reason I value having wireheaded simulations of myself experiencing the positive experiences I have had then I may as well do it well. I'm tempted to make a word play here on 'getting lucky' but the point is it is not about increasing your luck at all, in either case. It's about experience cloning.

How I Learned to Stop Worrying and Love the Anthropic Trilemma

My impression was that the Anthropic Trilemma was Eliezer uncharacteristically confusing himself when reality itself didn't need to be.

After step one, you have a 50% chance of finding yourself the original; there is nothing controversial about this much. If you are the original, you have a 50% chance of finding yourself still so after step two, and so on. That means after step 99, your subjective probability of still being the original is 0.5^99, in other words as close to zero as makes no difference.

Joe is just confused about the math. Multiplying the 0.5s together like that is the wrong thing to do. The subjective expectation I would have after being copied for the 99th time is that I have 50% chance of being told I am a copy.

Perhaps Joe's confusion is not fully specifying what his expectation is before the process starts. He expects that after he is cloned:

• p(there is an original) = 1
• p(there is a clone) = 1
• p(after one clone is made the original will expect to be told that he is a clone) = 0.5
• p(after one clone is made that clone will expect to be told that he is a clone) = 0.5
• p(after two clones are made there will be an original) = 1
• p(after two clones are made there will be a first clone) = 1
• p(after two clones are made there will be a second clone) = 1
• p(after two clones have been made the original clone will expect to be told that he is the second clone) = 0.5
• p(after two clones have been made the first clone will expect to be told that he is the second clone) = 0 (The first clone doesn't expect to be told anything more.)
• p(after two clones have been made the second clone will expect to be told that he is the second clone) = 0.5

Now, consider multiplying

• p(after two clones have been made the second clone will expect to be told that he is the second clone) * p(after one clone is made that clone will expect to be told that he is a clone).

Why would I do that? That makes no sense. That would give me 0.25 but so what? It certainly doesn't mean that p(after two clones have been made the second clone will expect to be told that he is the second clone | after one clone is made that clone will expect to be told that he is a clone) = 0.25. If Joe thinks you can do that he is just wrong and so will end up making a decision that doesn't optimally reflect his preferences.

Can I be sure I will not wake up as Britney Spears tomorrow?

Yes. For me to wake up as Britney Spears, would mean the atoms in her brain were rearranged to encode my memories and personality. The probability of this occurring is negligible.

It seems like you're discussing two types of copying procedure, or could be. The Ebborian copying seems to strongly imply the possibility of waking up as the copy or original (I have no statement on the probabilities), but a "teleporting Kirk" style of copying doesn't seem to imply this. You're presumably not Ebborian copied into Britney Spears, but if you're talking about Joe making 99 copies through Kirk copying, it seems like he'd have an equal chance to be any of them at any time, assuming he has any chance not to be the original.

I suppose this may not be the case if you view the subjective experience as being equally likely to be in either Joe until the first moment of differing input, here being the vocal information on what copy he is.

If you aren't discussing Ebborian copying, why does the creation of a physically identical version of oneself imply a possibility of being them? This seems like a widespread view, and I've long been confused by it, so I'd enjoy illumination by anyone on this point.

My probability of ending up the original couldn't have been 0.5^99, that's effectively impossible, less than the probability of hallucinating this whole conversation.

Does anyone have a sense of what the lower limit is on meaningful probability estimates for individual anticipation? Right, like there should be some probability p(E) where, upon experiencing E, even a relatively sane and well-balanced person ought to predict that the actual state of the world is ~E, because p(I'm Crazy or I've Misunderstood) >> p(E).

More to the point, p(E) should be roughly constant across apparently sane people; I would guess that the probability of hallucination doesn't vary by much more than a factor of 10 among people who have no reason to expect that they are hallucinating. 10^1 might be small relative to whatever the Minimum Probability turns out to be.

rwallace, nice reductio ad adsurdum of what I will call the Subjective Probability Anticipation Fallacy (SPAF). It is somewhat important because the SPAF seems much like, and may be the cause of, the Quantum Immortality Fallacy (QIF).

You are on the right track. What you are missing though is an account of how to deal properly with anthropic reasoning, probability, and decisions. For that see my paper on the 'Quantum Immortality' fallacy. I also explain it concisely on on my blog on Meaning of Probability in an MWI.

Basically, personal identity is not fundamental. For practical purposes, there are various kinds of effective probabilities. There is no actual randomness involved.

It is a mistake to work with 'probabilities' directly. Because the sum is always normalized to 1, 'probabilities' deal (in part) with global information, but people easily forget that and think of them as local. The proper quantity to use is measure, which is the amount of consciousness that each type of observer has, such that effective probability is proportional to measure (by summing over the branches and normalizing). It is important to remember that total measure need not be conserved as a function of time.

As for the bottom line: If there are 100 copies, they all have equal measure, and for all practical purposes have equal effective probability.

Interesting one. 100 hundred Joes, one 'original', some 'copies'.

If we copy Joe once, and let him be, he's 50% certainly original. If we copy the copy, Joe remains 50% certainly original, while status of copies does not change.

After the first copying process, we ended up with the original and a copy. 50% of the resulting sentient beings were original. If we do that again, again, we have two sentient beings, original and a new copy. Again, 50% chance for a random sentient byproduct of this copying process to be the original.

But there's something you didn't take into account. You can't just multiply the 50% of the first and second process, as your Joe wasn't chosen randomly after every copying process. After every single process, you took the original and copied only him again. Math should reflect this.

So let's look what happens in this process.

t=1: You know that you are the original t=2: We create a clone in such a way that you don't know whether you are a clone or not. At this time you have a subjective probability of 50% of being a clone. t=3: We tell clone 1 that they are a clone. Your subjective probability of being a clone is now 0% since you were not informed that you were a clone. t=4: We create another clone that provides you with a subjective probability of being a clone of 50% t=5: Clone 2 finds out that they are a clone. Since you weren't told you were a clone, you know you aren't a clone, so your subjective probability of being you goes back up to 100%.

Let's now imagine that we want no-one to know if they are clones or not. We will imagine that people initially know that they are not the new clone, but this information is erased.

t=1: We copy the original person so that we have two clones. We erase any information that would indicate who is original. t=2: We create a third clone, but allow the first two people to know they aren't the third clone t=3: We erase information from the first two people about whether or not they are the third clone.

At t=1, you have a 50% chance of being a clone and a 50% chance of being the original. At t=2, you still have a 50% chance as you know you aren't the third clone At t=3, you have lost information about whether you are the third clone. You can now be any of the clones and there is no distinguishing information, so the probability becomes 1/3 of being the original. Probability mass isn't just redistributed from the chance of you being the original but also from the chance of you being the first clone.

When we have created n clones, your odds of being the original will be 1/n.

It makes no difference whether the steps at t=2 and t=3 occur separately or together, I simply separately them to show that it was the loss of information about identity, not the cloning that changed the probability.

So if the clones weren't informed about their number after cloning, we would get the same result whether we produced 99 clones at once or one at a time.

Lastly, let's suppose that the clone is told that they are a clone, but the original doesn't know they won't be told. This won't affect the subjective probabilities of the original, only that of the clones, so again there isn't a paradox.

This paradox is based upon a misunderstanding of how cloning actually works. Once this is modelled as information loss, the solution is straightforward.

I don't really get what you're saying.

The normal way of looking at it is that you are only going to be you in the future. The better way of looking at it is that an unknown person is equally likely any person during any period of a given length.

The results of the former don't work well. They lead to people preferentially doing things to help their future selves, rather than helping others. This is rather silly. Future you isn't you either.

I don't believe in the Tegmark multiverse. ;)

I'm sorry, I didn't read the rest of your post after seeing the 0.5^99 estimate of the probability of being the original because the math looked very wrong to me, but I didn't know why. While I agree there is nothing controversial about saying that after one step you have a 50% chance of being the original, I'm pretty sure it is not true that you only have a 25% chance after two steps. Yes, if you are the original after step one, you have a 50% chance probability of still being the original after step two. So, I Oi is the probability of being the original after step i, then P(O2|O1) = 0.5, P(01) = 0.5, P(O2|~O1) = 0, so P(O2) = P(O2|O1) * P(O1). BUT, after step 2, O1 is not 0.5, since there are 3 copies of you in existence, and 2 of those WERE the original after step 1, so P(O1) after step 2 is 2/3, so P(O2) = 2/6 = 1/3. Don't think it affects the substance of anything else in your argument, though, but the math bothered me since intuitively the expected value of originals after any number of steps should be 1, which should be equal to the number of copies in existence times the number of copies. If I'm missing something I'd welcome an explanation of what.

I believe in continuity of substance, not similarity of pattern, as the basis of identity. If you are the original, that is what you are for all time. You cannot wake up as the copy. At best, a new mind can be created with false beliefs (such as false memories, of experiences which did not happen to it). Do I still face a problem of "subjective anticipation"?

ETA: Eliezer said of the original problem, "If you can't do the merge without killing people, then the trilemma is dissolved." Under a criterion of physical continuity, you cannot go from two objects to one object without at least one of them ceasing to be. So the original problem also appears to be a non-problem for me.

This 0.5^99 figure only appears if each copy bifurcates iteratively.

Rather than

1 becoming 2, becoming 3, becoming 4, ... becoming 100

We'd have

1 becoming 2, becoming 4, becoming 8, ... becoming 2^99