Past LHC failures are just civilization-level QI. (BTW, there are real things like this related to the history of earth atmosphere, in which CO2 content was anti-correlated with Sun's luminosity which result in stable temperatures). But it is not clear to me, what are other anthropic effects, which are not QI – what do you mean here? Can you provide one more example?

I meant that by creating and openly putting my copies I increase the number of my copies, and that diluting is not just an ethical judgement, but the real effect, similar to self-sampling assumption, in which I am less likely to be a copy-in-pain, if there are many my happy copies. Moreover, this effect may be so strong that my copies will "jump" from unhappy world to happy one. I explored it here. 

Thanks. It is a good point that. I should add this. 

consent to sideloading should be conditional instead of general

Unfortunately, as a person in pain will not have time to remember a lot details about their past,  a very short list of facts can be enough to recreate "me in pain". May be less than 100. 

Instead of deleting, I suggest diluting: generate many fake facts about yourself and inject them into the forum. Thus chances to get recreate you will be slim. 

Anyway, I bet on idea that it is better to have orders of magnitude more happy copies, than fight to prevent one in pain. Here I dilute not information, but pain with happiness. 

I understood your argument as following; anything which is an argument for QI, can also be argument for alien saving us. Thus, nothing is evidence for QI. 
However, apriory probabilities of QI and alien are not mutually independent. QI increases chances of alien with every round. We can't observe QI directly. But we will observe the alien and this is what is predicted by QI. 

• We care still alive
• No GPT-5 yet
• Rumors of hitting the wall

Thanks. By the way, the "chatification" of the mind is a real problem. It's an example of reverse alignment: humans are more alignable than AI (we are gullible), so during interactions with AI, human goals will drift more quickly than AI goals. In the end, we get perfect alignment: humans will want paperclips.

For the outside view: Imagine that an outside observer uses a fair coin to observe one of two rooms (assuming merging in the red room has happened). They will observe either a red room or a green room, with a copy in each. However, the observer who was copied has different chances of observing the green and red rooms. Even if the outside observer has access to the entire current state of the world (but not the character of mixing of the paths in the past), they can't determine the copied observer's subjective chances. This implies that subjective unmeasurable probabilities are real.

Even without merging, an outside observer will observe three rooms with equal 1/3 probability for each, while an insider will observe room 1 with 1/2 probability. In cases of multiple sequential copying events, the subjective probability for the last copy becomes extremely small, making the difference between outside and inside perspectives significant.

When I spoke about the similarity with the Sleeping Beauty problem, I meant its typical interpretation. It's an important contribution to recognize that Monday-tails and Tuesday-tails are not independent events.

However, I have an impression that this may result in a paradoxical two-thirder solution: In it, Sleeping Beauty updates only once – recognizing that there are two more chances to be in tails. But she doesn't update again upon knowing it's Monday, as Monday-tails and Tuesday-tails are the same event. In that case, despite knowing it's Monday, she maintains a 2/3 credence that she's in the tails world. This is technically equivalent to the 'future anthropic shadow' or anti-doomsday argument – the belief that one is now in the world with the longest possible survival.

Thanks for your thoughtful answer.

To achieve magic, we need the ability to merge minds, which can be easily done with programs and doesn't require anything quantum. If we merge 21 and 1, both will be in the same red room after awakening. If awakening in the red room means getting 100 USD, and the green room means losing it, then the machine will be profitable from the subjective point of view of the mind which enters it. Or we can just turn off 21 without awakening, in which case we will get 1/3 and 2/3 chances for green and red.

The interesting question here is whether this can be replicated at the quantum level (we know there is a way to get quantum magic in MWI, and it is quantum suicide with money prizes, but I am interested in a more subtle probability shift where all variants remain). If yes, such ability may naturally evolve via quantum Darwinism because it would give an enormous fitness advantage – I will write a separate post about this.

Now the next interesting thing: If I look at the experiment from outside, I will give all three variants 1/3, but from inside it will be 1/4, 1/4, and 1/2. The probability distribution is exactly the same as in Sleeping Beauty, and likely both experiments are isomorphic. In the SB experiment, there are two different ways of "copying": first is the coin and second is awakenings with amnesia, which complicates things.

Identity is indeed confusing. Interestingly, in the art world, path-based identity is used to define identity, that is, the provenance of artworks = history of ownership. Blockchain is also an example of path-based identity. Also, in path-based identity, the Ship of Theseus remains the same.

There is a strange correlation between paradox of young Sun (it had lower luminosity) and stable Earth temperature which was provided by higher greenhouse effect. As sun goes brighter, CO2 declined. It was even analyses as evidence of anthropic effects. 

 

In his article "The Anthropic Principle in Cosmology and Geology" [Shcherbitsky, 1999], A. S. Shcherbakov thoroughly examines the anthropic principle's effect using the historical dynamics of Earth's atmosphere as an example. He writes: "It is known that geological evolution proceeds within an oscillatory regime. Its extreme points correspond to two states, known as the 'hot planet' and 'white planet'... The 'hot planet' situation occurs when large volumes of gaseous components, primarily carbon dioxide, are released from Earth's mantle...

As calculations show, the gradual evaporation of ocean water just 10 meters deep can create such greenhouse conditions that water begins to boil. This process continues without additional heat input. The endpoint of this process is the boiling away of the oceans, with near-surface temperatures and pressures rising to hundreds of atmospheres and degrees... Geological evidence indicates that Earth has four times come very close to total glaciation. An equal number of times, it has stopped short of ocean evaporation. Why did neither occur? There seems to be no common and unified saving cause. Instead, each time reveals a single and always unique circumstance. It is precisely when attempting to explain these that geological texts begin to show familiar phrases like '...extremely low probability,' 'if this geological factor had varied by a small fraction,' etc...

In the fundamental monograph 'History of the Atmosphere' [Budyko, 1985], there is discussion of an inexplicable correlation between three phenomena: solar activity rhythms, mantle degassing stages, and the evolution of life. 'The correspondence between atmospheric physicochemical regime fluctuations and biosphere development needs can only be explained by random coordination of direction and speed of unrelated processes - solar evolution and Earth's evolution. Since the probability of such coordination is exceptionally small, this leads to the conclusion about the exceptional rarity of life (especially its higher forms) in the Universe.'"

Quantum immortality and gun jammed do not contradict each other: for example, if we survive 10 rounds failures because of QI, we most likely survive only on those timelines where gun is broken. So both QI and gun jamming can be true and support one another and there is no contradiction.

One problem here is that quantum immortality and angel immortality eventually merges: for example, if we survive 10 LHC failures because of QI, we most likely survive only on those timelines where some alien stops LHC. So both QI and angel immortality can be true and support one another and there is no contradiction. 

I know this post and have two problems with it: what they call 'anthropic shadow" is not proper term as Bostrom defined anthropic shadow as underestimation of past risks based on the fact of survival in his article this the same name. But it's ok. 

The more serious problem is that quantum immortality and angel immortality eventually merges: for example, if we survive 10 LHC failures because of QI, we most likely survive only on those timeline where some alien stops LHC. So both QI and angel immortality can be true and support one another and there is no contradiction. 

Check my new post which favors the longest and thickest timelines https://www.lesswrong.com/posts/hB2CTaxqJAeh5jdfF/quantum-immortality-a-perspective-if-ai-doomers-are-probably?commentId=aAzrogWBqtFDqMMpp

A sad thing is that most of life moments are like this 30-minutes intervals - we forget most life events, they are like dead ends. 

More generally, type-copies of me still matter for me.

Under them the chance for you to find yourself in a branch where all coins are Heads is 1/128, but your over chance to survive is 100%. Therefore the low chance of failed execution doesn't matter, quantum immortality will "increase" the probability to 1

You are right, and it's a serious counterargument to consider. Actually, I invented path-dependent identity as a counterargument to Miller's thought experiment.

You are also right that the Anthropic Trilemma and Magic by Forgetting do not work with path-dependent identity.

However, we can almost recreate the magic machine from the Anthropic Trilemma using path-based identity:

Imagine that I want to guess in which room I will be if there are two copies of me in the future, red or green.

I go into a dream. A machine creates my copy and then one more copy of that copy, which will result in 1/4 and 1/4 chances each. The second copy then merges with the first one, so we end up with only two copies, but I have a 3/4 chance to be the first one and 1/4 to be the second. So we've basically recreated a machine that can manipulate probabilities and got magic back.

The main problem of path-dependent identity is that we assume the existence of a "global hidden variable" for any observer. It is hidden as it can't be measured by an outside viewer and only represents the subjective chances of the observer to be one copy and not another. And it is global as it depends on the observer's path, not their current state. It therefore contradicts the view that mind is equal to a Turing computer (functionalism) and requires the existence of some identity carrier which moves through paths (qualia, quantum continuity, or soul).

Also, path-dependent identity opens the door to back-causation and premonition, because if we normalize outputs of some black box where paths are mixed, similar to the magic machine discussed above, we get a shift in its input probability distribution in the past. This becomes similar to the 'timeline selection principle' (which I discussed in a longer version of this blog post but cut to fit format) in which not observer-moments are selected, but the whole timelines without updating on my position in the timeline. This idea formalizes the future anthropic shadow as I am more likely to be in the timeline that is fattest and longest in the future.

As we assume that coin tosses are quantum, and I will be killed if (I didn't guess pi) or (coin toss is not heads) there is always a branch with 1/128 measure where all coins are heads, and they are more probable than surviving via some errors in the setup. 

All hell breaks loose" refers here to a hypothetical ability to manipulate perceived probability—that is, magic. The idea is that I can manipulate such probability by changing my measure.

One way to do this is described in Yudkowsky's " The Anthropic Trilemma," where an observer temporarily boosts their measure by increasing the number of their copies in an uploaded computer.

I described a similar idea in "Magic by forgetting," where the observer boosts their measure by forgetting some information and thus becoming similar to a larger group of observers.

Hidden variables also appear depending on the order in which I make copies: if each copy is made from subsequent copies, the original will have a 0.5 probability, the first copy 0.25, the next 0.125, and so on.

"Anthropic shadow" appear only because the number of observers changes in different branches. 

 

Orthogonality between goals and DT makes sense only if I don't have preferences about the type of DT or the outcomes which one of them necessitates.

In the case of QI, orthogonality works if we use QI to earn money or to care about relatives.

However, humans have preferences about existence and non-existence beyond normal money utility. In general, people strongly don't want to die. It means that I have a strong preference that some of my copies survive anyway, even if it is not very useful for some other preferences under some other DT.

Another point is the difference between Quantum suicide and QI. QS is an action, but QI is just a prediction of future observations and because of that it is less affected by decision theories. We can say that those copies of me who survive [high chance of death event] will say that they survived because of QI.

But if I use quantum coin to make a life choice, there will be splitting, right?

Wei· 3h

This post touches on several issues I've been thinking about since my early work on anthropic decision theory and UDT. Let me break this down:

1. The measure-decline problem is actually more general than just quantum mechanics. It appears in any situation where your decision algorithm gets instantiated multiple times, including classical copying, simulation, or indexical uncertainty. See my old posts on anthropic probabilities and probability-as-preference.

2. The "functional identity" argument being used here to dismiss certain types of splitting is problematic. What counts as "functionally identical" depends on your decision theory's level of grain. UDT1.1 would treat seemingly identical copies differently if they're in different computational states, while CDT might lump them together.

Some relevant questions that aren't addressed:

- How do we handle preference aggregation across different versions of yourself with different measures?
- Should we treat quantum branching differently from other forms of splitting? (I lean towards "no" these days)
- How does this interact with questions of personal identity continuity?
- What happens when we consider infinite branches? (This relates to my work on infinite ethics)

The real issue here isn't about measure per se, but about how to aggregate preferences across different instances of your decision algorithm. This connects to some open problems in decision theory:

1. The problem of preference aggregation across copies
2. How to handle logical uncertainty in the context of anthropics
3. Whether "caring about measure" can be coherently formalized

I explored some of these issues in my paper on UDT, but I now think the framework needs significant revision to handle these cases properly.

  Stuart · 2h
  > The problem of preference aggregation across copies
  
  This seems key. Have you made any progress on formalizing this since your 2019 posts?

     Wei · 2h
     Some progress on the math, but still hitting fundamental issues with infinity. Might post about this soon.

  Abram · 1h
  Curious about your current thoughts on treating decision-theoretic identical copies differently. Seems related to logical causation?

     Wei · 45m
     Yes - this connects to some ideas about logical coordination I've been developing. The key insight is that even "identical" copies might have different logical roles...

[Edit: For those interested in following up, I recommend starting with my sequence on decision theory and anthropics, then moving to the more recent work on logical uncertainty.]

Vladimir_N 3h

(This is a rather technical comment that attempts to clarify some decision-theoretic confusions.)

Your treatment of measure requires more formal specification. Let's be precise about what we mean by "caring about measure" in decision-theoretic terms.

Consider a formalization where we have:
1. A space of possible outcomes Ω
2. A measure μ on this space
3. A utility function U: Ω → ℝ
4. A decision function D that maps available choices to distributions over Ω

The issue isn't about "spending" measure, but about how we aggregate utility across branches. The standard formulation already handles this correctly through expected utility:

E[U] = ∫_Ω U(ω)dμ(ω)

Your concern about "measure decline" seems to conflate the measure μ with the utility U. These are fundamentally different mathematical objects serving different purposes in the formalism.

If we try to modify this to "care about measure directly," we'd need something like:

U'(ω) = U(ω) * f(μ(ω))

But this leads to problematic decision-theoretic behavior, violating basic consistency requirements like dynamic consistency. It's not clear how to specify f in a way that doesn't lead to contradictions.

The apparent paradox dissolves when we properly separate:
1. Measure as probability measure (μ)
2. Utility as preference ordering over outcomes (U)
3. Decision-theoretic aggregation (E[U])

[Technical note: This relates to my work on logical uncertainty and reflection principles. See my 2011 paper on decision theory in anthropic contexts.]

  orthonormal · 2h
  > U'(ω) = U(ω) * f(μ(ω))
  
  This is a very clean way of showing why "caring about measure" leads to problems.

     Vladimir_N · 2h
     Yes, though there are even deeper issues with updateless treatment of anthropic measure that I haven't addressed here for brevity.

  Wei_D · 1h
  Interesting formalization. How would this handle cases where the agent's preferences include preferences over the measure itself?

     Vladimir_N · 45m
     That would require extending the outcome space Ω to include descriptions of measures, which brings additional technical complications...

[Note: This comment assumes familiarity with measure theory and decision theory fundamentals.]

Eli · 2h

*sigh*

I feel like I need to step in here because people are once again getting confused about measure, identity, and decision theory in ways I thought we cleared up circa 2008-2009.

First: The whole "measure declining by choice" framing is confused. You're not "spending" measure like some kind of quantum currency. The measure *describes* the Born probabilities; it's not something you optimize for directly any more than you should optimize for having higher probabilities in your belief distribution.

Second: The apparent "splitting" of worlds isn't fundamentally different between quantum events, daily choices, and life-changing decisions. It's all part of the same unified wavefunction evolving according to the same physics. The distinction being drawn here is anthropocentric and not particularly meaningful from the perspective of quantum mechanics.

What *is* relevant is how you handle subjective anticipation of future experiences. But note that "caring about measure" in the way described would lead to obviously wrong decisions - like refusing to make any choices at all to "preserve measure," which would itself be a choice (!).

If you're actually trying to maximize expected utility across the multiverse (which is what you should be doing), then the Born probabilities handle everything correctly without need for additional complexity. The framework I laid out in Quantum Ethics handles this cleanly.

And please, can we stop with the quantum suicide thought experiments? They're actively harmful to clear thinking about decision theory and anthropics. I literally wrote "Don't Un-think the Quantum" to address exactly these kinds of confusions.

(Though I suppose I should be somewhat grateful that at least nobody in this thread has brought up p-zombies or consciousness crystals yet...)

[Edit: To be clear, this isn't meant to discourage exploration of these ideas. But we should build on existing work rather than repeatedly discovering the same confusions.]

  RationalSkeptic · 1h
  > like refusing to make any choices at all to "preserve measure,"
  
  This made me laugh out loud. Talk about Pascal's Mugging via quantum mechanics...

     Eli · 45m
     Indeed. Though I'd note that proper handling of Pascal's Mugging itself requires getting anthropics right first...

In replies to this comment I will post other Sonnet3.5-generated replies by known LW people. If it is against the rules please let me know and I will delete. I will slightly change the names, so they will not contaminate future search and AI training

My point was that only 3 is relevant. How it improves average decision making?

In big world immortality there are causally disconnected copies which survive in very remote regions of the universe. But if we don't need continuity, but only similarity of minds, for identity, it is enough. 

In that case, we can say that QI works only in EDT, but not in CDT or in UDT. 

An interesting question arise, can we have a DT which shows that quantum suicide for money is bad idea, but euthanasia is also bad idea in QI-world. 

Just want to mention that memantine is a weaker analogue of ketamine and has also antidepressant afterglow effect. Even alcohol has for some people this afterglow effect.

Indeed, QI matters depending on what I care. If mother cares about her child, quantum suicide will be a stupid act for her, as in the most worlds the child will be left alone. If a person cares only about what he feels, QC has more sense (the same way as euthanasia has sense only if quantum immortality is false).

We can escape the first-person perspective question by analyzing the optimal betting strategy of a rational agent regarding the most likely way of survival.

In the original thought experiment, there are 10 similar timelines where 10 otherwise identical agents guess a digit of pi (each guessing a different digit). Each agent has a 1/128 chance to survive via a random coin toss.

The total survival chances are either 1/10 via guessing pi correctly (one agent survives) or approximately 10/128 via random coin tosses (ignoring here the more complex equation for combining probabilities). 1/10 is still larger.

The experiment can be modified to use 10 random coins to get more decisive results.

Therefore, any agent can reasonably bet that if they survive, the most likely way of survival would be through correctly guessing the pi digit. (Here goes also all caveats about limits of betting).

Whether to call this "immortality" is more of an aesthetic choice, but the fact remains that some of my copies survive any risk in Many-Worlds Interpretation. The crux is whether agent should treat his declining measure as a partial death.

I think you are right. We will not observe QI agents and it is a bad policy to recommend it as I will end in empty world soon. Now caveats. My measure declines because of branching anyway very quickly, so no problem. There is an idea of civilization-level quantum suicide by Paul Almond. In that case, the whole civilization performs QI coin trick, and no problem with empty world - but can explain Fermi paradox. QI make sense from first-person perspective, but not from third.

In MWI some part of the body always remains. But why destuctible?

I think that first perspective is meaningful as it allows me to treat my self as a random sample from some group of minds.

The problem with observables here is that there is another copy of me in another light cone, which has the same observables. So we can't say that another light cone is unobservable - I am already there and observing it. This is a paradoxical property of big world immortality: it requires actually existing but causally disconnected copies, which contradicts some definitions of actuality.

BTW, can you comment below to Vladinir Nesov, who seems to think that first-person perspective is illusion and only third-person perspective is real?

A more interesting counterargument is "distribution shift." My next observer-moments have some probability distribution P of properties - representing what I am most likely to do in the next moment. If I die, and MWI is false, but chaotic inflation is true, then there are many minds similar to me and to my next observer-moments everywhere in the multiverse. However, they have a distribution of properties P2 - representing what they are more likely to observe. And maybe P ≠ P2. Or may be we can prove that P=P2 based on typicality. 

If there is no identity substance, then copies even outside the light cone matter. And even non-exact copies matter if the difference is almost unobservable. So I think that countable infinity is enough. 

Typically, this reasoning doesn't work because we have to update once again based on our current age and on the fact that such technologies do not yet exist, which compensates for the update in the direction of "Life extension is developing and alignment is easy."

This is easier to understand through the Sleeping Beauty problem. She wakes up once on Monday if it's heads, and on both Monday and Tuesday if it's tails. The first update suggests that tails is two times more likely, so the probability becomes 2/3. However, as people typically argue, after learning that it is Monday, she needs to update back to 1/3, which yields the same probability for both tails and heads.

But in the two-thirders' position, we reject the second update because Tails-Monday and Tails-Tuesday are not independent events (as was recently discussed on LessWrong in the Sleeping Beauty series).

QI is a claim about first-person perspective observables – that I will always observe the next observer moment. This claim is stronger than just postulating that MWI is true and that there are many me-like minds in it from a third-person perspective. This difference can be illustrated by some people's views about copies. They say: "I know that somewhere there will be my copy, but it will not be me, and if I die, I will die forever." So they agree with the factual part but deny the perspectival part.

I agree that the main consideration here is decision-theoretic. However, we need to be suspicious of any decision theory that was designed specifically to prevent paradoxes like QI, or we end up with circular logic: "QI is false because our XDT, which was designed to prevent things like QI, says that we should ignore it."

There is a counterargument (was it you who suggested it?) that there is no decision difference regardless of whether QI is valid or not. But this argument only holds for altruistic and updateless theories. For an egoistic EDT agent, QI would recommend playing Russian roulette for money.

If QI is false, it must have theoretical cost:

Either:

• Universe is finite and no MWI
• Personal identity is based on some kind of destructible soul and my copy is not me
• We use some variant of updateless decision theory, especially designed to prevent this type of problems (which is rather circular counterargument)

It is a good summary, but the question is can we generalize it the idea that "I am more likely to be born in the world where life extensions technologies are developing and alignment is easy". Simple Bayesian update does not support this. 

However, if the future measure can somehow "propagate" back in time, it increases my chances to be born in the world where there is logical chances for survival: alignment is easy. 

A simple example of a world-model where measure "propagates" back in time is the simulation argument: if I survive in human-interested AI world, there will be many more my copies in the future who think that I am the past. 

However, there could be more interesting ways for measure to propagate back in time. One of them is that the object of anthropic selection is not observer-moments, but the whole observer-timelines. Another is two-thirder solution to sleeping beauty.  

I cited it as a blog. This option is available on Zotero.

Your central argument seems to be a metaphor: We caused the Holocene extinction of animals, so godlike AI will kill us.

The problem with metaphorical arguments is that they can be reversed. As humans have become more intelligent, we've started to value animals, created zoos, natural reserves and now even work on the resurrection of extinct animals like mammoths. See more examples of such reversal by Gwern https://gwern.net/modus

Presenting weak arguments is evidence that there are no strong arguments, and this is obvious to outside readers.

The main problem is that we can't predict what superintelligent AI will do, and thus we can't 100 percent prove that it will necessarily kill us. But we shouldn't have to.

Instead, we should show that superintelligence will disempower us and that it may want to kill us for some reasons.

BTW, memantine is weak (but legal) analog of ketamine and helped me to cure my depression. 

The problem is that their understanding of their territory is not the same as our legal understanding, so they can attack on the roads outside their homes. 

As was said above, first you need to pick a stone from the ground or pretend that you are doing this if there is no stone around. Even if you have a stone, make the gesture that you take it from the ground. 

Another important point is to do it quickly and aggressively with loud cry. Also you can pull back one's arm with a stone. 

The whole trick is that dogs are so afraid of stones that they will run away before you actually throw it or they see where it fails. 

Good point about impossibility of sex in LD. But masturbation is actually a form of day dreaming,

I wrote Active Imagination as an Alternative to Lucid Dreaming: Theory and Experimental Results which is basically about controlled daydreaming. 

Yes daydreaming is underestimated

Can you make Trolley meme for Death in Damascus and Doomsday Argument? 

Can prove that you can express any decision theory problem as some Trolley problem?

Yes. It is important point. 

There are some game theory considerations here:

If I throw the stone, all dogs will know that I don't have it anymore, so it would be safe for them to continue the attack (whether I hit one or miss). Therefore, it's better for me to threaten and keep two stones rather than actually throw one.

If dogs really want to attack me, they might prefer that I throw the stone so they can attack afterward.
However, I think each dog fails to consider that I'm most likely to throw the stone at another dog. Each individual dog has a small chance of being injured by the stone, and they could succeed if they continue the attack. Real hunters like wolves might understand this.

Lifehack: If you're attacked by a group of stray dogs, pretend to throw a stone at them. Each dog will think you're throwing the stone at it and will run away. This has worked for me twice.