Anthropically Blind: the anthropic shadow is reflectively inconsistent

post by Christopher King (christopher-king) · 2023-06-29T02:36:26.347Z · LW · GW · 40 comments

Contents

  Anthropically undead: ghosts are as good as gone
  A worked example: calibrated forecasts of the deadly coin
  Anthropic Angels and Lucky Fedoras
  What would convince me that the anthropic shadow is real?
  Applications
    LHC failures, Nuclear close calls, Strange anti-correlations, etc...
    The Hero With A Thousand Chances
    The Doomsday argument and Grabby Aliens
  Conclusion
None
40 comments

For the purposes of this post, the anthropic shadow is the type of inference found in How Many LHC Failures Is Too Many? [LW · GW].

"Anthropic principle! If the LHC had worked, it would have produced a black hole or strangelet or vacuum failure, and we wouldn't be here!"

In other words, since we are "blind" to situations in which we don't exist, we must adjust how we do bayesian updating. Although it has many bizarre conclusions, it is more intuitive than you think and quite useful!

There are many similar applications of anthropics, such as Nuclear close calls and Anthropic signature: strange anti-correlations [LW · GW].

This actually has implications for effective altruism. Since we are so early into humanity's existence, we can infer from the anthropic shadow that humans will probably soon die out. Also see The Hero With A Thousand Chances [LW · GW].

More practically [? · GW], the anthropic shadow should give us useful advice on how to reason about personally risky activities like driving or perhaps even aging. I have not actually seen any advice based on this principle, but theoretically there should be some conclusions you could draw.

The problem, as you probably deduced from the title, is that it is reflectively inconsistent.

Central Claim: Someone using the anthropic shadow should update their decision making to no longer use it. This can be justified with their current decision making procedure.

(This also suggests that if you used it in the past, that was probably the wrong thing to do [? · GW].)

A weaker (and obvious) claim that is also sometimes called the anthropic shadow is that we do not have experience with situations in which we have died. I agree with this version, but isn't what I will be arguing against.

Note that I am not the first to notice paradoxes with the anthropic shadow. See No Anthropic Evidence [LW · GW] for example. I have not yet seen the result about reflective inconsistency though, hence why I am making this post.

I also introduce the concepts of "Anthropic undeath", "Anthropic angel" (how you would explain an absurdly large number of weird coincidences having to do with death), "Fedora shadow", and apply the central claim to a couple examples. To my knowledge, these contributions are novel.

Anthropically undead: ghosts are as good as gone

This section contains the most general form of the argument. (This could be mathematically formalized; I just haven't gotten around to doing it.) If it seems strange to you, a generalized version of this section [LW · GW] might also work.

First, we establish the basic framing of how we will check if something is reflectively consistent. Imagine yourself before a catastrophe potentially happens. You are an expected utility maximizer (as all good agents should be, although this assumption can probably be weakened). You are trying to come up with a policy for your future-self to follow.

Consider the following scenario: in any situation that you would die, imagine instead that you become an agent with only one choice each time it takes an action: "do nothing". This state is still given the same utility as before (including the utility change from physically dying (a reinforcement learner would stop getting rewards, for example)), but as an agent you never stop existing. Call this unreal scenario "anthropic undeath".

Optimizing the utility of the real scenario is the same as optimizing utility in anthropic undeath, because the agent choosing "do nothing" in the anthropic undeath scenario has the same physical effect as what actually happens in the real scenario when the agent is dead. I call this the "ghosts are as good as gone" principle.

The anthropic undeath scenario has no anthropic shadow, because the agent never stops existing. Thus, the optimal policy never uses anthropic shadow in its reasoning. The optimal policy in the real scenario is the same by the principle in the previous paragraph.

(Also see this comment [LW(p) · GW(p)] for an explanation of why we can consider a physically dead entity to be an agent.)

Q.E.D.

A worked example: calibrated forecasts of the deadly coin

Imagine a coin with a side labelled "zero" and a side labelled "one". It will get flipped a bunch of times. Before each flip, you will give your credence p that the coin will come up one. Letting x be the result of the coin flip, you will be given 1000 (1-(p-x)²) utils. (We can imagine the utils representing cookies being given to your friend Timmy.) Notice that this is a proper scoring rule. Also, if the coin comes up one, the game ends and you die. The utils still count if you die (because Timmy still gets the cookies). (Notice how the "you die" part has no effect on the game since the game is over anyways. We could stop here using the same argument as the previous section, but we will work it out to illustrate why the anthropic shadow fails.)

You now must come up with a policy that you will follow during the game.

You have two hypotheses about the coin, both with 50% credence [? · GW] apriori:

1. The coin always comes up zero.
2. The coin has an independent 50% chance of coming up zero each time.

Consider a situation in which you have observed 7 zeros in a row. What p should you choose? The anthropic shadow suggests you have gotten no information, and thus the chance of one is 25%.

However, this is incorrect as a policy. Before the game begins, you reason as follows about the state after 7 coin flips:

1. There is a 50% chance you are in scenario 1. This contributes 1000 (0.5 (1-p²)) to the expected value.
2. There is a 50% chance you are in scenario 2.
   1. There is a 1 - 2⁻⁷ chance that one of the coin flips resulted in one. In this case, the game is over, you are dead, and nothing is contributed to the expected utility.
   2. There is a 2⁻⁷ chance that you get 7 zeros in a row. There is a 50% chance that the next flip is a zero, and a 50% chance that it is a one. Since scenario 2 is itself a 50% chance, this contributes 1000 (2⁻⁸ (1-(0.5 p² + 0.5 (p-1)²))) to the expected utility.

Maximizing your expected utility of 1000 (0.5 (1-p²) + 2⁻⁸ (1-(0.5 p² + 0.5 (p-1)²))), you find the optimal p is 1/258, equivalent to odds of 1 to 257 (about 1/85 of the anthropic shadow estimate). This is exactly the same as the probability you get by doing normal bayesian updating!

To summarize, the anthropic shadow would have you say:

"Anthropic principle! If the coin came up one, I would have died, and I wouldn't be here!"

And would lose you about 30 expected utils on the 7th round alone! At the beginning of the game when you are setting your policy, don't do that!

You might say, "but what I really want is to not die, I don't care about maximizing my calibration!". If so, you lack faith in the Litany of Tarski [? · GW]. Vladimir Nesov has an example in No Anthropic Evidence [LW · GW] where the only goal is survival. Again, the optimal policy agrees with not using anthropic shadow. If you have a clear view of your utility function (including in states where you no longer exist), it is best for your credences to calibrated!

Anthropic Angels and Lucky Fedoras

Okay, but what should we do if we observed a huge amount of evidence that weird coincidences happen around deadly things, like a zillion LHC accidents or perfect anti-correlation between pandemics and recessions in Anthropic signature: strange anti-correlations [LW · GW]. Surely at some point I must relent and go "okay, the anthropic shadow is real". And if that is so, than even a little bit of evidence should at least make us a little worried about the anthropic shadow.

No.

Treating ideal reasoning as an approximation to Solomonoff Induction [? · GW], we find that there is no anthropic shadow hypothesis. However, there are what I call anthropic angel hypotheses. These are hypotheses that posit that there is some mysterious force that protects you from death, perhaps via rejection sampling. One such hypothesis is quantum immortality [? · GW]^[1].

An important thing to understand about anthropic angels though is that they typically don't stop on the next observation. If the LHC would destroy humanity but accidents keep happening, will I protest to stop the LHC? No, because there is no reason to think that the accidents will stop.

Of course, if you are worried that the anthropic angel might fail in the future, you still might be cautious. However, the more times you get saved, the more you can trust the angel. This is the exact opposite of the anthropic shadow!

Keep in mind also that the type of reasoning behind anthropic angels also applies to things other than death. Death isn't special in this context! Suppose that you are a forgetful person, but you have a lucky fedora. You notice that there are weird coincidences that prevent you from losing your fedora. Is there a "fedora shadow", whereby the version of you currently wearing the fedora can't observe scenarios where the fedora is missing, and thus you must adjust your bayesian updating? No. Given enough evidence, you would need to conclude that there is a "fedora angel" that influences events to save your lucky fedora for some reason, instead of a fedora shadow whereby from your luck you make fearful inferences.

What would convince me that the anthropic shadow is real?

So - taking into account the previous cancellation of the Superconducting Supercollider (SSC) - how many times does the LHC have to fail before you'll start considering an anthropic explanation? 10? 20? 50? - How Many LHC Failures Is Too Many? [LW · GW]

Since the anthropic shadow is not reflectively consistent, I am convinced that there is no object-level evidence that would persuade me. No amount of LHC weirdness, Nuclear close calls, strange anti-correlations, etc... would change my mind. The evidence that is normally presented for the anthropic shadow is instead (extremely weak thus far) evidence for an anthropic angel.

However, to make my belief pay rent [? · GW], I should specify what it excludes. Here is what I would count as evidence for the anthropic shadow: if people applying the concept of anthropic shadow to personal risk of death, such as car crashes, consistently make better decisions than those who do not. Note that to be persuasive, there shouldn't be simpler explanations (like it cancelling out some other bias, or in an extreme case them actually using the anthropic angel).

Curiously, I have not seen anyone apply the anthropic shadow in this way (except ironically). If anyone tries, I strongly anticipate it will be systematically worse.

Applications

LHC failures [LW · GW], Nuclear close calls, Strange anti-correlations [LW · GW], etc...

These are basically the same as the worked out example above [LW(p) · GW(p)]. A string of identical coin flips might seem unusual, but they do not mean we should use the anthropic shadow.

For example, for the strange anti-correlations, the probability mass of "x-risk and we are alive and unexplained anti-correlation" is the same as "no x-risk and unexplained anti-correlation", so good policy does not use anthropic shadow.

Also see this comment [LW(p) · GW(p)] (and the ones around it) for a more indepth discussion of how it applies to things like the LHC (with many independent parts that could fail).

The Hero With A Thousand Chances [LW · GW]

"Allow me to make sure I have this straight," the hero said. "The previous hero said I would definitely fail for what reason?"

"Shades of Ahntharhapik, very serious!" said Aerhien.

"And how did he fare?" asked the hero?

"Pretty typically. He tried to destroy the Dust with something called a 'Nuclear bomb'. Didn't get very far though. In the uranium mine we discovered a magical artifact that turned half of the miners into zombies who started eating the Dust." replied Aerhien.

Ghandhol chipped in "The hero then said that the shades of Ahntharhapik saved us, but it wouldn't next time, and thus we should summon a hero from the same world to continue the advanced weaponry program he started."

"🤦 so the previous hero was trying to imply that each time you survived, that was evidence that you were bad at survival." sighed the hero.

"Yes, there is no other explanation!" exclaimed Aerhien.

The hero mocked "Just like if you see a thousand coin flips come up heads, the only explanation is that you got really really lucky (instead of checking if both sides of the coin are heads)?"

The whole council went silent.

"Look, out of all the possible worlds that could've summoned me, the ones that weren't good at surviving passed long ago. And the worlds that already succeeded wouldn't be summoning me to fight the Dust either. Your world genuinely does have a kind of luck. We can't agree to disagree." said the hero.

"What kind of cruel luck is this? Is it the Counter-Force?" replied Aerhien.

"So to speak. Hmm. If your world was generated by resampling, you would've defeated the Dust long ago (or if that also caused a resample, a more sensible stalemate). If your world branches off at the moment of death, the anomalies would happen much later. If someone tried to shoot you, the gun would go off, but the bullet would bounce." thought the hero.

The hero then had an insight, saying "its simple really. We are in a fantasy novel of some sort (or maybe a fanfic? something in this vicinity). Normally fictional characters don't figure this out, but as a good bayesian reasoner, even these kinds of conclusions can't escape me! Especially when the alternative is believing in a thousand identical coin flips."

The Doomsday argument and Grabby Aliens [? · GW]

When I was first writing this, I thought that my argument ruled out the Doomsday argument, but permitted Grabby aliens. Turns out, it vaguely argues against both, but not in a strong enough way to conclude they are reflectively inconsistent. It is quite similar to the Self-Indication Assumption Doomsday argument rebuttal (the main difference being that the likelihood of being born is a sublinear function of the number of humans under my argument).

Let p be that probability that a randomly generated human will be me, Christopher King (as defined by my experiences). Let q be the probability that a randomly generated civilization in the universe will contain Christopher King. What policy should I choose so that Christopher King is well calibrated?

For the doomsday argument, the example hypotheses are:

1. There will be 120 billion humans.
2. There will be 10ˡ⁰⁰ humans.

And for grabby aliens:

3. The universe will be taken over by grabby aliens.
4. The universe is and will be filled with quiet aliens (for simplicity, we will say that it has the same frequency f of intelligent civilizations per year per meter cubed of unconquered space as in hypothesis 3).

Both arguments posit that we need to explain earliness. However, hypotheses 2 and 4 also have early humans. For the doomsday argument, the probability of Christopher King being among the first ~60 billion humans is (1-(1-p)^(60 billion)) under both hypotheses. So being early is not evidence either way for 1 or 2. For grabby aliens, the probability of Christopher King being present in the first ~14 billion years of the universe is (1-(1-q)^(f * 14 billion years * (4 x 10⁸⁰ m³))) under hypothese 4, and slightly less under hypothese 3 because some space is already conquered by grabby aliens. So the likelihood ratio [? · GW] favors hypothesis 4.

The problem is that this technically isn't a case of reflective inconsistency, because I wouldn't be able to remember and reflect before the universe started, of course. I worry in particular that there is no reason for "pre-existence" Christopher King to have the same priors as "embodied" Christopher King.^[2] See also Where Recursive Justification Hits Bottom [LW · GW].

However, also see SSA rejects anthropic shadow, too [LW · GW] for ways in which popular anthropic theories handle the pre-existence anthropic shadow.

Conclusion

So in summary, we can update on the fact that we have survived. Counter-intuitively, we should treat the fact that it is "humanity 1, x-risk 0" or "survive 1, death 0" the same as we would treat any other statistic of that form. For example, we can update normally on the fact that we survived the Cold War, the fact that nothing has randomly killed humanity (like pandemics) yet, and on a personal level the fact that we survive things like car crashes. This was shown on the basis of reflective consistency.

---

1. As far as I know, our current Everett branch [? · GW] is best explained by Born's rule without quantum immortality (QI). There are infinitely many branches where, in the future, we start seeing evidence of it, but in each such branch Solomonoff induction (SI) will only require a finite amount of evidence to update (bounded by a constant determined by how complicated it is to implement QI in the programming language). That is what it means for SI to be universal: it works in every Everett branch, not just the typical ones [LW · GW].

   On the other hand, QI can have infinitely bad calibration in finite time. If they are in a normal Born's rule branch and they die, their prediction for if they would die (and any consequences thereof) would have infinitely many bits of surprisal. This could be quite bad if you cared about the state of the world in the Born's rule branch after your death! ↩︎

2. My argument works fine under SIA [? · GW] and SSA [? · GW] (assuming that physicists are correct about the universe being infinite), but there are more exotic sampling assumptions like "SSA but only in the observable universe and also no Boltzmann brains" where it can fail. This hypothesis would have positive probability under UDASSA [? · GW], for example. Even though its weird, I don't see a reflective inconsistency. ↩︎

40 comments

Comments sorted by top scores.