You're in Newcomb's Box

I upvoted this because it was highly amusing -- but ultimately it's silly, a perfect example of how some people can be so sharp that they cut themselves.

I wonder, if instead one-box and two-box for a prize of 100$ or 200$, we had "Selection A: horrible self-mutilation" and "Selection B: one million dollars", with Prometheus creating only the people that he believed would pick Selection A, and reject Selection B.... would the people that one-box here STILL onebox?

Well, I'd just choose to win instead and thus pick the one million dollars instead of the horrible self-mutilation. I think that's the sane thing to do -- if Prometheus has a 99.99% predictive capacity on this there'll be 10000 people who'll select self-mutilation for every one like me who'll pick the money. But I already know I'm like me, and I'm the one I'm concerned about.

The relevant probability isn't P(chooses Self-mutilation|Prometheus created him) ~= 0, but rather the P(chooses one million dollars|Is Aris Katsaris) ~= 1.

Azathoth wants you to maximize your number of descendants; if you fail to have descendants, Azathoth will try not to have created you.

But this seems merely false. Azathoth just creates descendants whose ancestors reproduced. Azathoth isn't exerting any sort of foresight as to whether you reproduce. I can't figure out who or what you're trying to trade with. Not having children simply does not make you retroactively less likely to have existed.

I suppose you could be in a Newcomblike situation with your parents making a similar decision to have birthed you. I don't see how you could be in one with respect to Azathoth/evolution. It's not modeling you, it doesn't contain a computation similar to you, there is no logical update on what it does after you know your own decision.

My initial reaction is "This is all correct... except that Azerthoth isn't smart enough to have invented counter factual trade!". Just imagine trying to counterfactually trade with your past self from before you knew about counter-factual trade. A similar case is coming up with a great product you'd like to buy, only to discover when you get to the market you were the first to come up with the idea and nobody is selling the product despite that it'd be good for both of you if they had.

For further clarity here's a scenario where you reasoning probably WOULD work:

You find yourself on a planet, and you know this planet is in a phase of development in which conditions remain almost perfectly unchanged for 3^^^3 years/generations (the laws of physics would need to be different in this universe for this to work I think). The environment is also completely reset every few generations and only the extremely durable spores for the next generation is able to survive so there's no way to relay messages to the future. No tech solutions like directly editing and putting messages in the genes because you can't develop tech in a single generation.

If i lived on that planet I'd pay very close attention to the reasoning of this post. In the world that's unly existed for a few billion years and is about to hit the singularity, and where I've grown up in conditions completely different from my ancestors... not so much.

Good post thou, upvoted.

Q: Why does this knife have a handle?
A: This allows you to grasp it without cutting yourself.

These kinds of answers are highly compelling, but strictly speaking they are allowing events in the future to influence events in the past.

No, they aren't. They are answering the question according to the standard meaning conveyed with "Why?". When we use the word 'why' we mean a variety of things along the lines of 'What purpose does this serve?' as well as sometimes 'Explain the series of events that lead up to the final state in a convenient way'.

In standard usage if someone answers 'so you don't cut yourself' they usually are not talking about anything to do with temporal relations one way or the other.

Q: Why do I exist?
A: Because you're going to have lots of children.

Just wrong. The only thing close to this with even a little bit of poetic truth would be that I exist because Azathoth, being familiar with my design, rationally expects me to have lots of children.

a mutually beneficial trade with Azathoth (Azathoth creates you; you increase your reproductive fitness in exchange)

At first I reacted negatively to this idea, but eventually I realized that the argument has the same acausal trade structure as the rest of the EDT-inspired nonsense around here. My voluntary reproductive efforts seem to be evidence about my genetic makeup, and that genetic makeup is in a causal relationship with my existence. So that is not what is wrong with this idea.

The trouble is that

1. Azathoth has already given me what I want - my existence.
2. My ancestral history has already given Azathoth the evidence that it wants - any reproductive behavior that I indulge in is superfluous to my own existence (though Azathoth may well take it into account in deciding whether to grant existence to my children).

The 'Azathoth problem' is isomorphic to the smoking lesion problem, which is not isomorphic to Newcomb's problem.

Hence, any decision theory capable of both (i) one-boxing in Newcomb's problem and (ii) choosing to smoke in the 'smoking lesion' problem will have no difficulty here.

EDIT: I'd better sketch out this "isomorphism": "smoking" = "acting virtuously, in defiance of our evolutionary drives", "not smoking" = "giving in to our instincts and trying to optimize number of children". "having the lesion" = "carrying genes that predispose you to virtuous behaviour, and therefore having a smaller chance of having been born in the first place", "not having the lesion" = "carrying genes that predispose you to evolutionarily 'selfish' behaviour and therefore having a larger chance of having been born".

You weren't created by Prometheus; you were created by Azathoth, The God That is Evolution by Natural Selection. You are the product of an ongoing optimization process that is trying to maximize reproductive fitness. Azathoth wants you to maximize your number of descendants; if you fail to have descendants, Azathoth will try not to have created you. If your intelligence reduces your reproduction rate, Azathoth will try not to grant you intelligence. If the Darwinian-optimal choice conflicts with the moral one, Azathoth wants you to choose evil.

It would seem, then, that any decision theory that demands that you one-box (or that allows you to survive the similar Parfit's Hitchhiker problem), also demands that you try to maximize your reproductive fitness. In many cases this injunction would be benign: after all, Azathoth created our morality. But in far too many, it is repugnant; there can be no doubt that in many commonplace situations, Azathoth wants you to cheat, or rape, or murder. It seems that in such cases you should balance a decreased chance of having existed against the rest of your utility function. Do not worship Azathoth, unless you consider never having existed to be infinitely bad. But do make sacrifices.

Creating something that you predict will work is a different thing to killing things that don't work. In the case of a lot of evolutionary reasoning we can more or less get away with equating the two (as well as personifying) because the evolution happened over a large time scale with a relatively stable gradient. The individual generations can kind of be blurred in together. But when considering what we want to do we can't take this kind of short cut.

Doing the things that you describe as "what Azeroth wants" would, if all else was equal, lead us to expect that it is more likely that people similar to us will exist in the future. But when looking in the other direction we don't conclude that submitting to Azeroth makes you more likely exist but rather that the people who do exist are less likely to betray Azeroth.

All of this is basically an elaboration of "No, part two is not a Newcomblike decision task".

Azathoth wants you to maximize your number of descendants; if you fail to have descendants, Azathoth will try not to have created you.

It sure is welcome to try now that I've precommitted to never have children.

(On a silly note, this gave me a mental image of a time-traveling God of Evolution who meddles with the past to achieve desired results in the present. shudder)

But if you one-box in Newcomb's Problem, you should take these answers more literally. The kinds of backwards causal arrows you draw are the same.

But keep in mind that this kind of control, or "backwards causality", is all about your map, not the territory, more precisely it's about your state of logical uncertainty and not about what the definitions you have logically imply. If you already know what the state (probability) of the thing you purport to control is, then you can't control it.

In this manner, you might have weak control over your own evolutionary adaptations (i.e. over past evolution) by having more or less children, if the regularity that links your behavior to past evolution is all you know, and that you do know of such a link (if your decisions are made using advanced considerations which were not considered by your ancestors, then you can't control evolution this way). But as you learn more, you control less, or alternatively, you discover that you actually didn't have any control after all.

So this kind of control is often strictly illusory, and the only reason to take it seriously, to actually try to exert it, is that at that moment you honestly don't know whether it is. If you act on simple enough considerations, that were indeed instantiated in the past, then it might well not be illusory, but considering how complicated human mind is, that would be rare, and so the extent of your control would be low.

For example, to what extent do you control other people during voting? Only to the extent your own resolution to vote so and so controls your anticipation of other people voting similarly, after you take into account all you know about other people independently of your resolution to vote in a certain way. This might in practice be not very much, you know a lot about other people already, without assuming your own decision, and it's hard to (logically) connect others' actions to your own decision.

I would zero-box, so that I could exist twice.

It's dangerous to update after observing your own existence because no counterfactual you can update on their non-existence so your updates can't possibly make sense in aggregate.

Onebox in Part 1 if you value your existence because the probability of your one-boxing directly determines the probability of you existing and Prometheus accepts or rejects you as a package. You need not bother (much) with Azathoth because he's an idiot, has no predictive abilities at all beyond the simplest form of induction, only took the effects of (some of) the parts that make you up in completely different combinations into account, and what your ancestors did is much more weakly entangled with what you are going to do.

How does Prometheus being wrong make me cease to exist?

By creating a simulation to interrogate, Omega/Prometheus/Azathoth have brought a being into existence, which means the being may have preferences to continue to exist (in some other form). So I'd tend to pick B for Prometheus, to continue existing. I wouldn't do so for Azathoth, because evolution doesn't have to create a living version of me to see what I would do; there is no "I" to regret dying or not existing there.

Thanks, this is a highly thought-provoking (and headache-inducing :p) post.

The "obvious" objection to the (evolutionary) argument here is that, were I to in fact make some choices that increase my inclusive genetic fitness in response to this argument, that would bear almost no connection to the genetic fitness of my ancestors, who had never been exposed to the argument, and were presumably causal decision theorists too. (If what I have just said is true, by the way, that would make the argument also a basilisk, in that by disseminating it you would [eventually] force our descendents to consider it in order to increase their measure, decreasing morality. This is unequivocally a bad thing, unless having lots of children [=increasing the number of sentient beings in existence, ala. total utilitarianism] is good directly.)

There are flavors of the smoking lesion in here too, I think. Generally, if the vector involved in the decision is genetic predisposition, having any mathematical decision theory whatsoever, and sticking to it (overriding my instincts) ought to screen off the influence of my genes. If I made this choice because UDT demands it of me, I didn't do it because of my genetic predisposition, unless I have a predisposition to adopt UDT (which could I suppose be called "intelligence").

Other issues, which I am confused about:

• What utility do I assign to never having existed at all? This is of course addressed in the post, somewhat. Possibly relevant: it doesn't feel like I'm deliberately thwarting the preferences/rights of my potential children when I refuse to have as many children as possible, as quickly as possible.
• Related to the above, were I an impartial sentient AI that computed only what is right, I know I wouldn't care about never having existed at all. If I weren't alive and happy, presumably someone else would be, and in the 1-place rightness function these are precisely the same thing.
• UDT doesn't update on anything, even existence, apparently. Am I allowed to condition on the "obvious fact" that I exist? I don't even know.

Your idea sounds plausible and interesting, but I don't completely understand the implications. What am I supposed to do if the environment changes from generation to generation, e.g. due to advances in science? Should I adopt the behaviors that helped my ancestors have many kids, or the behaviors that will help me have many kids?

Others in this thread have pointed this out, but I will try to articulate my point a little more clearly.

Decision theories that require us to two-box do so because we have incomplete information about the environment. We might be in a universe where Omega thinks that we'll one-box; if we think that Omega is nearly infallible, we increase this probability by choosing to one-box. Note that probability is about our own information, not about the universe. We're not modifying the universe, we're refining our estimates.

If the box is transparent, and we can see the money, we simply don't care what Omega says. As long as we trust that the bottom won't fall out (or any number of other possibilities), we can make our decision because our information (about which universe we are in) is not incomplete.

Likewise, our information about whether we exist is not incomplete; we can't change it by choosing to go against the genes that got us here.

For situations where our knowledge is incomplete, we actually can derive information (about what kind of a world we inhabit) from our desires, but it is evidence, not certainty, and certainly not acasual negotiation. We can easily have evidence that outweighs this relatively meager data.

So...if Prometheus created you to one-box, and you should one-box anyway...why not one-box? "Too much useless information" alarm bells are ringing in my head.

Edit: Italics to quotes.

Edit2: I failed to read. See this comment

Ok, so as I understand timeless decision theory, one wants to honor precommitments that one would have made if the outcome actually depended on the answer regardless of whether or not the outcome actually depends on the answer or not. The reason for this seems to be that behaving as a timeless decision agent makes your behavior predictable to other timeless decision theoretical agents (including your future selves), and therefore big wins can be had all around for all, especially when trying to predict your own future behavior.

So, if you buy the idea that there are multiple universes, and multiple instantiations of this problem, and you somehow care about the results in these of these other universes, and your actions indicate probabalistically how other instantiations of your predicted self will act, then by all means, One Box on problem #1.

However, if you do NOT care about other universes, and believe this is in fact a single instantiation, and you are not totally freaked out by the idea of disobeying the desires of your just revealed upon you creator (or actually get some pleasure out of this idea), then please Two Box. You as you are in this universe will NOT unexist if you do so. You know that going into it. So, calculate the utility you gain from getting a million dollars this one time vs the utility you lose from being an imperfect timeless decision theoretical agent. Sure, there's some loss, but at a high enough pay out, it becomes a worthy trade.

I think Newcomb's problem would be more interesting if the 1st box contained 1/2 million and the 2nd box contained 1 million, and omega was only right, say 75% of the time... See how fast answers start changing. What if omega thought you were a dirty two-boxer and only put money in box b? Then you would be screwed if you one-boxed! Try telling your wife that you made the correct 'timeless decision theoretical' answer when you come home with nothing.

I don't get it. I do exist. If I never reproduce, then Azathoth predicted incorrectly (which will hardly be the first time).

(I also agree with the response that the universe isn't better off for having me in it, but that doesn't matter, since it has me anyway.)

Total utilitarians want to one-box in Prometheus' game, and average utilitarians want to two-box.

Let me try to make my objection clearer. You seem to be concerned with things that make your existence less likely. But that is never going to be a problem. You already know the probability of your own existence is 1; you can't update it based on new data.

I don't get it. Is this supposed to be some weird form of evidential or maybe timeless decision theory? It hardly matters; whatever decision theory you're using, you already know you exist; conditioning on the possibility that you don't is nonsensical. Hell, even if you're an AI using UDT you gain nothing from not assuming you exist; you were built to not update in the normal sense because whoever built you cared about all possible worlds you might end up in, but regardless, if you're standing there making the decision, you exist (i.e. this can be assumed at the start and taken into account).

Edit: Just for the purpose of explicitness, I should probably state that the conclusion here is that you should two-box in this case.

Although I find one-boxing difficult to do in that scenario, as a human, it is apparent that a reflectively consistent decision theory would one-box, as that is what it would have precommitted to do if it had the option (prior to its not yet determined existence) to precommit. No backwards arrows of causality are needed, just a particular type of consistency (updatelessness or timelessness).

I've been trying to work on this problem based on my admittely poor understanding of Updateless Decision Theory and I think I've come to the conclusion that, while you should one-box in Newcomb's problem and in Transparent Newcomb's problem, you should two-box when dealing with Prometheus, ignore Azathoth, and ignore the desires of evil parents.

Why? My reasoning is based on these lines from cousin_it's explanation of UDT:

When you're faced with a decision, you find all copies of you in the entire "multiverse" that are faced with the same decision ("information set"), and choose the decision that logically implies the maximum sum of resulting utilities weghted by universe-weight......

.......For example, Counterfactual Mugging: by assumption, your decision logically affects both heads-universe and tails-universe, which (also by assumption) have equal weight, so by agreeing to pay you win more cookies overall.

I start by taking into account that there are some universes where I was created by Prometheus/Azathoth/evil parents and some where I was not. I then try to make the decision that will increase the utility of all my copies in all possible universes where a version of me is faced with this decision. If I two-box then all the existing copies of me faced with the same decision will also two-box, and get $200. The nonexistant mes in the universes where I was not created will keep right on not existing. If I one-box all the copies of me will get $100. Again, the nonexistant mes in the universes where I was not created will keep right on not existing, their nonexistant utility unchanged. So all my copies will get more utility if I two-box, ignore Azathoth, and tell my evil parents to do something anatomically improbable. The nonexistant mes cannot be affected.

The key is that you're supposed to consider the utility of "all copies of you in the entire "multiverse" that are faced with the same decision," and copies that were never created are obviously not faced with the same decision. This differs from the counterfactual mugging because there you exist in the heads and tails universe, so you have to take the utility of both copies into account. I believe that it differs from Newcomb's problem and Transparent Newcomb's problem for the same reason.

So it looks like, if I understand UDT correctly, in newcomblike-problems where you never having existed is part of the problem, one-boxing is not neccessarily rational. (An aside: I should also mention that I am assuming whatever method of prediction Prometheus used did not result in the creation of a morally signtificant copy of you in his head. That would be a whole other ballgame, and I think the spirit of the original post was that Prometheus' prediction method did not do this)

Did I get this right, or is my understanding of UDT wrong? I'm not very certain of this at all, and would like it if someone with a stronger understanding of UDT could confirm or disconfirm it.

UPDATE: I think that a possible flaw in my reasoning is that I have misunderstood UDT to mean "make all your copies make whatever decision maximizes their utility in all possible situations," when what it really means is more like "make your decision as if the Omega/Prometheus in those other universes is watching your universe and basing your decision on your behavior there, rather than the behavior of the you in their native universe; and act to maximize utility across all possible universes." I think my previous formulation implies two-boxing in Newcomb's, which seems wrong. With my second formulation it might indeed be better to one-box in the Prometheus problem because in some other universe Prometheus is "watching (i.e. simulating) the you of your universe and is going to decide to create you based on what you do.

I'm not sure my second formulation is quite right either. It still seems to me that these nonexistance problems are qualitatively different from other Newcomblike problems, it seems that the fact that in some universes I don't exist changes the nature of the problem in some way so that one-boxing is no longer rational, maybe because in those universes I'm not part of the same "information set."

That being said, even if UDT recommends one-boxing, there are still several strong objections to the original post's conclusions, and AnasKateris' disturbing "evil parent" variant:

• Azathoth is not "watching your universe," it's not simulating anything, so it is not analogous to Prometheus
• I'm not sure not updating the fact that you exist is logically coherent.
• In any "evil parent" situation if the demands they make are sufficiently horrible it is better to not exist than to obey them.
• These discussions focus primarily on what is individually rational, not what is collectively rational. It is collectively rational to two-box in the "evil parent" variant to deter evil parents from actually trying this, even if it isn't individually rational. Similarly, the collectively rational thing to do in the Prometheus problem is probably to ask Omega if it can help you track down Prometheus, chain him to a rock, and make a giant eagle eat his constantly-regenerating liver to make sure he stops pulling crap like this.
• If UDT makes you "lose" in the Prometheus problem and the "evil parent" variant maybe it still needs some work.

For this specific formulation of the question, I think it may be relevant to know whether Prometheus updates on your decisions in order to improve his projections on whether future individuals will one box or two box.

What was the point of reposting this after it was in the discussion section, without seeming to edit it in response to comments since then?

I think the primary reason why this Prometheus problem is flawed is that in Newcomb's problem, the presence or absence of the million dollars is unknown, while in this Prometheus problem, you already know what Prometheus did as a result of his prediction. Think of a variation on Newcomb's problem where Omega allows you to look inside box B before choosing, and you see that it is full. Only an idiot would take only one box in that scenario, and that's why this analysis is flawed.

Good stuff, thanks.

Acausal version: "If you have goals that would be served by you existing, then try to have many kids because it increases the number of worlds in which you exist." Note how this completely ignores the "fact" that you "already exist" - of course you do, we're living in a multiverse! What's left to you is to increase the measure.

Causal version 1: "If you're a good person, and you believe the world needs more good people, then try to have many kids." Note that this argument doesn't rely on genetics only: how your kids will turn out depends on nurture too. Which brings us to...

Causal version 2: "If you have a good idea, and you believe the world needs more people who agree with that idea, then try to spread it." In other words, exactly what Eliezer is doing by creating LW :-)

"You were created by a god: a being called Prometheus. Prometheus was neither omniscient nor particularly benevolent. He was given a large set of blueprints for possible human embryos, and for each blueprint that pleased him he created that embryo and implanted it in a human woman. Here was how he judged the blueprints: any that he guessed would grow into a person who would choose only Box B in this situation, he created. If he judged that the embryo would grow into a person who chose both boxes, he filed that blueprint away unused. Prometheus's predictive ability was not perfect, but it was very strong; he was the god, after all, of Foresight."

Do you take both boxes, or only Box B?

I take one box. Normal Newcomblike reasoning.

This seems to highlight my main complaint with Newcomb's problem. It assumes reverse causation is possible. Perhaps I'm being narrow minded, but, "Assume reverse causation is possible. How do you deal with this hypothetical?" does not mean you should actually design a decision theory to take into account reverse causation, without adequate evidence it exists.

there can be no doubt that in many commonplace situations, Azathoth wants you to cheat, or rape, or murder.

It seems as though rape and murder often lead to prison sentences, which involves being confined in an environment with no members of the opposite sex. This is especially true in the modern surveillence-saturated world.

Baby makers who are positively inclined towards rape and murder are in a tiny minority, and are probably desperate - and desperate people often do bad things, largely irrespective of what their goals are - witness the priests and the choirboys.