Paradoxes in all anthropic probabilities

Does your post also show that selfish preferences are incoherent, because any selfish preference must rely on a weighting of your copies and every such weighting has weird properties?

As reductios of anthropic views go, these are all pretty mild. Abandoning conservation of expected evidence isn't exactly an un-biteable bullet. And "Violating causality" is particularly mild, especially for those of us who like non-causal decision theories. As a one-boxer I've been accused of believing in retrocausality dozens of times... sticks and stones, you know. This sort of "causality violation" seems similarly frivolous. Oh, and the SSA reference class arbitrariness thing can be avoided by steelmanning SSA to make it more elegant--just get rid of the reference class idea and do it with centered worlds. SSA is what you get if you just do ordinary Bayesian conditionalization on centered worlds instead of on possible worlds. (Which is actually the more elegant and natural way of doing it, since possible worlds are a weird restriction on the sorts of sentences we use. Centered worlds, by contrast, are simply maximally consistent sets of sentences, full stop.) As for changing the probability of past events... this isn't mysterious in principle. We change the probability of past events all the time. Probabilities are just our credences in things! More seriously though, let A be the hypothetical state of the past light-cone that would result in your choosing to stretch your arm ten minutes from now, and B be the hypothetical state of the past light-cone that would result in your choosing to not stretch your arm. A and B are past events, but you should be uncertain about which one obtained until about ten minutes from now, at which point (depending on what you choose!) the probability of A will increase or decrease.

There are strong reductios in the vicinity though, if I recall correctly. (I did my MA on this stuff, but it was a while ago so I'm a little rusty.)

FNC-type views have the result that (a) we almost instantly become convinced, no matter what we experience, that the universe is an infinite soup of random noise occasionally coalescing to form Boltzmann Brains, because this is the simplest hypothesis that assigns probability 1 to the data; (b) we stay in this state forever and act accordingly--which means thinking happy thoughts, or something like that, whether we are average utilitarians or total utilitarians or egoists.

SIA-type views are as far as I can tell incoherent, in the following sense: The population size of universes grows much faster than their probability can shrink. So if you want to say that their probability is proportional to their population size... how? (Flag: I notice I am confused about this part.) A more down-to-earth way of putting this problem is that the hypothesis in which there is one universe is dominated by the hypothesis in which there are 3^^^^3 copies of that universe in parallel dimensions, which in turn is dominated by the hypothesis in which there are 4^^^^^4...

SSA-type views are the only game in town, as far as I'm concerned--except for the "Let's abandon probability entirely and just do decision theory" idea you favor. I'm not sure what to make of it yet. Anyhow, the big problem I see for SSA-type views is the one you mention about using the ability to create tons of copies of yourself to influence the world. That seems weird all right. I'd like to avoid that consequence if possible. But it doesn't seem worse than weird to me yet. It doesn't seem... un-biteable.

EDIT: I should add that I think your conclusion is probably right--I think your move away from probability and towards decision theory seems very promising. As we went updateless in decision theory, so too should we go updateless in probability. Something like that (I have to think & read about it more). I'm just objecting to the strong wording in your arguments to get there. :)

I'd also love some clarifications:

For instance, if we expect that there will be a single gender in the future, then if I'm in the reference class of males, I expect that single gender will be female - and the opposite will be expected for someone in the reference class of females

Further:

It violates causality: it assigns different probabilities to an event, purely depending on the future consequence of that event.

Unfortunately the link doesn't work.

Why shouldn't the probability predictably update for the Self-Indicative Assumption? This version of probability isn't supposed to refer to a property of the world, but some combination of the state of the world and the number of agents in various scenarios.

Regarding the self-sampling assumption, if you knew ahead of time that the lever would be pulled, then you could have updated before it was pulled. If you didn't know that the lever would be pulled, then you gained information: specifically about the number of copies in the heads case. It's not that you knew there was only one copy in the heads world, it's just that you thought there was only one copy, because you didn't think a mechanism like the lever would exist and be pulled. In fact, if you knew that the probability of the lever existing and then being pulled was 0, then the scenario would contradict itself.

As Charlie Steiner notes, it looks like you've lost information, but that's only because you've gained information that your information is inaccurate. Here's an analogy: suppose I give you a scientific paper that contains what appears to be lots of valuable information. Next I tell you that the paper is a fraud. It seems like you've lost information as you are less certain about the world, but you've actually gained it. I've written about this before for the Dr Evil problem [LW · GW].

To show a problem with SIA, assume there is one copy of you, that we flip a coin, and, if comes out tails, we will immediately duplicate you (putting the duplicate in a separate room). If it comes out heads, we will wait a minute before duplicating you.

We could make it simpler: flip a coin and duplicate you in a minute if it comes up heads. A.k.a. sleeping beauty. We already know that SIA forces you to update when you go through an anthropic "split" (like waking up in sleeping beauty), not just when you learn something.

The probability is only different when you think the world is in a different state. This no more violates conservation of expected evidence than putting the kettle on violates conservation of expected evidence by predictably changing the probability of hot water. The weird part is which part of the world is different.

With regards to your SIA objection, I think it is important to clarify exactly what we mean by evidence conservation here. The usual formulation is something like "If I expect to assign credence X to proposition P at future time T, then I should assign credence X to proposition P right now, unless by time T I expect to have lost information in a predictable way". Now if you are going to be duplicated, then it is not exactly clear what you mean by "I expect to assign ... at future time T", since there will be multiple copies of you that exist at time T. So, maybe you want to get around this by saying that you are referring to the "original" version of you that exists at time T, rather than any duplicates. But then the problem seems to be that by waiting, you will actually lose information in a predictable way! Namely, right now you know that you are not a duplicate, but the future version of you will not know that it is not a duplicate. Since you are losing information, it is not surprising that your probability will predictably change. So, I don't think SIA violates evidence conservation.

Incidentally, here is an intuition pump that I think supports SIA: suppose I flip a coin and if it is heads then I kill you, tails I keep you alive. Then if you are alive at the end of the experiment, surely you should assign 100% probability to tails (discounting model uncertainty of course). But you could easily reason that this violates evidence conservation: you predictably know that all future agents descended from you will assign 100% probability to tails, while you currently only assign 50% to tails. This points to the importance of precisely defining and analyzing evidence conservation as I have done in the previous paragraph. Additionally, if we generalize to the setting where I make/keep X copies of you if the coin lands heads and Y copies if tails, then SIA gives the elegant formula X/(X+Y) as the probability for heads after the experiment, and it is nice that our straightforward intuitions about the cases X=0 and Y=0 provide a double-check for this formula.

Does MWI imply SIA?

Here’s a model of Sleeping Beauty under MWI. The universe has two apartments with multiple rooms. Each apartment has a room containing a copy of you. You have 50:50 beliefs about which apartment you’re in. One minute from now, a new copy of you will be created in the first apartment (in another identical room). At that moment, despite getting no new information, should you change your belief about which apartment you’re in? Obviously yes.

So it seems like your "conservation of expected evidence" argument has a mistake somewhere, and SIA is actually fine.

My preferred way of doing anthropics while keeping probabilities around is to update your probabilities according to the chance that at least one of the decision making agents that your decision is logically linked to exists, and then prioritise the worlds where there are more of those agents by acknowledging that you're making the decision for all of them. This yields the same (correct) conclusions as SIA when you're only making decisions for yourself, and FNC when you're making decisions for all of your identical copies, but it avoids the paradoxes brought up in this article and it allows you to take into account that you're making decisions for all of your similar copies, which you want to have for newcombs problem like situations.

However, I think it's possible to construct even more contorted scenarios where conservation of expected evidence is violated for this as well. If there are 2 copies of you, a coin is flipped, and:

• If it's heads the copies are presented with two different choices.
• If it's tails the copies are presented with the same choice.

then you know that you will update towards heads when you're presented with a choice after a minute, since heads make it twice as likely that anyone would be presented with that specific choice. I don't know if there's any way around this. Maybe if you update your probabilities according to the chance that someone following your decision theory is around, rather than someone making your exact choice, or something like that?

Is there a strong argument as to why we should care about the conservation of expected evidence? My belief at the moment is that something very close to SIA is true, and that conservation of expected evidence is a principle which simply doesn't hold in scenarios of multiple observers.