Why do theists, undergrads, and Less Wrongers favor one-boxing on Newcomb?

Will's link has an Asimov quote that supports the "self-image vs right answer" idea, at least for Asimov:

I would, without hesitation, take both boxes . . . I am myself a determinist, but it is perfectly clear to me that any human being worthy of being considered a human being (including most certainly myself) would prefer free will, if such a thing could exist. . . Now, then, suppose you take both boxes and it turns out (as it almost certainly will) that God has foreseen this and placed nothing in the second box. You will then, at least, have expressed your willingness to gamble on his nonomniscience and on your own free will and will have willingly given up a million dollars for the sake of that willingness-itself a snap of the finger in the face of the Almighty and a vote, however futile, for free will. . . And, of course, if God has muffed and left a million dollars in the box, then not only will you have gained that million, but far more imponant you will have demonstrated God's nonomniscience.9

Two boxing is relying on your own power to think and act. One boxing is basically knuckling under to the supposed power of the predictor. I will trust that I am helpless before him. He is an agent and I am his object.

Theists one box and atheists two box. Not surprising.

The twist here at LW is that lots of people seem terribly enamored of these recursive ill posed problems, and think they have solutions to them. That's where the one boxing comes from here, IMO.

It also helps that most Bayesian decision algorithms actually take on the arg max_a U(a)*P(a) reasoning of Evidential Decision Theory, which means that whenever you invoke your self-image as a capital-B Bayesian you are semi-consciously invoking Evidential Decision Theory, which does actually get the right answer, even if it messes up on other problems.

(Commenting because I got here while looking for citations for my WIP post about another way to handle Newcomb-like problems.)

It should take quite a lot of evidence for me to give up on causality as is.

Your past, Omega-observed self can cause both Omega's prediction and your future choice without violating causality.

What you're objecting to is your being predictable.

We have to determine what counts as "unfair". Newcomb's problem looks unfair because your decision seems to change the past. I have seen another Newcomb-like problem that was (I believe) genuinely unfair, because depending on their decision theory, the agents were not in the same epistemic state.

Here what I think is a "fair" problem. It's when

1. the initial epistemic state of the agent is independent of its source code;
2. given the decisions of the agent, the end result is independent of its source code;
3. if there are intermediary steps, then given the decisions of the agent up to any given point, its epistemic state and any intermediate result accessible to the agent at that point are independent of its source code.

If we think of the agent as a program, I think we can equate "decision" with the agent's output. It's harder however to equate "epistemic state" with its input: recall Omega saying "Here is the 2 usual boxes. I have submitted this very problem in a simulation to TDT. If it one boxed, box B has the million. If it two boxed, box B is empty". So, if you're TDT, this problem is equivalent to the old Newcomb problem, where oneBox <=> $$$. But any other agent could 2 box, and get the million and the bonus. (Also, "TDT" could be replaced by a source code listing that the agent would recognize as its own.)

Anyway, I believe there's a good chance a decision theory exists such that it gets the best results out of any "fair" problem.Though now that I think of it, condition 2 may be a sufficient criterion for "fairness", for the problem above violates it: if TDT two-boxes, it does not get the million. Well except it does not two box, so my counter-factual doesn't really mean anything…

Simulation's choice | Our Choice | Payoff
$1         | $1         = $1
$1         | $2 or $100 = $100
$2 or $100 | $1         = $1
$2 or $100 | $2 or $100 = $2

The problem of cooperation and simulation is one that happens in reality, right now, even though simulation accuracy is far far lower. The problem of Omega is a reductio of this but I think it's plausible that entities can approach omega-level abilities of prediction even if they'll never actually get that accurate.

I'd just like to say that this comparison of CDT, TDT, and UDT was a very good explanation of the differences. Thanks for that.

This explanation makes UDT seem strictly more powerful than TDT (if UDT can handle Parfit's Hitchhiker and TDT can't).

If that's the case, then is there a point in still focusing on developing TDT? Is it meant as just a stepping stone to an even better decision theory (possibly UDT itself) down the line? Or do you believe UDT's advantages to be counterbalanced by disadvantages?

You may not expect to encounter Newcomb's problems, but you might expect to encounter prisoner's dilemmas, and CDT recommends defecting on these.

the connection to the prisoners' dilemma? (which is there from the very beginning, right?)

I think David Lewis was the first to observe this connection, in 1979, 10 years after Nozick's publication of the problem. But the prisioner's dilemma is only Newcomb-like if the two prisoners are psychological twins, i.e. if they use the same decision proceedure and know this about each other. One might object that this is just as unfair as Newcomb's problem.

But the objection that Newcomb's is unfair isn't to be confused with the objection that it's unrealistic. I think everybody working on the problem accepts that Newcomb-like situations are practically possible. Unfairness is a different issue.

Well, that depends. It could turn out to be the case that, in reality, for some fixed definition of fair, the universe is unfair. If that were the case, I think at least some of the philosophers who study decision theory would maintain a distinction between ideal rational behavior, whatever that means, and the behavior that, in the universe, consistently results in the highest payoffs. But Eliezer / MIRI is solely interested in the latter. So it depends on what your priorities are.

Oops. Yes.

August. (I'm writing one.)

One-boxing is normal and does not call out for an explanation. :)

Timeless decision theory. UDT = Updateless decision theory.

Yes, because it's impossible for AI to get better than a rudimentary program running on the NYT server.

I committed to sharing results beforehand: First twenty versus Veteran mode: +8 -6 =6. Second twenty: +8 -8 =4. I spent about five seconds thinking between moves. I love RPS, I could easily get addicted to this... ETA: I decided to play ten rounds where I thought really hard about it. I got +4 -0 =6. ETA2: Okay, I'll play twenty rounds thinking less than a second per move...: +7 -8 =5.

11 ties vs 9 non-ties? How odd.

Good breakdown. #7 is not an explanation unless coupled with a hypothesis on why LW readers are more adept than mainstream philosophers and decision theorists at spotting the right answer on this problem. Unless one claims that LWers just have a generally higher IQ (implausible) an explanation for this would probably go back to #1 or something like it.

Personally, I think the answer is a combination of #1, #2, #3. I'm not sure about the relative roles played by each of them (which have a decreasing level of "rationality") but here is an analogy:

Suppose you know that there is a controversy between two views A and B in philosophy (or economics, or psychology, or another area which is not a hard science), that University X has in its department a leading proponent of theory A, and that bunch of theorists have clustered around her. It is surely not surprising that there are more A proponents among this group than among the general discipline. As possible explanations, the same factors apply in this general case: we could hypothesize that philosophers in X are exposed to unusually strong arguments for A, or that B-proponents disproportionately go to other universities, or that philosophers in X are slavishly following their leader. I contend that the question about LW is no different in essence from this general one, and that whatever view about the interplay of sociology, memetic theory and rationality you have as your explanation of "many A-ers at X" also should apply for "many 1-boxers at LW".

I suspect 4 is pretty strong. I can't distinguish 1 and 2, but 3 doesn't seem right. People disagree all the time with little regard to whom they're disagreeing with.

Outside view it's the most plausible hypothesis (start by asking "who brought this question to the attention of LWers in the first place"...). It explains why LWers hold various other views as well (e.g. regarding many-worlds).

For me I think part of the reason I'm so very very quick to commit to one boxing is the low improvement in outcomes from 2 boxing as the problem was presented on the wiki.

The wiki lists 1000 vs 1000,000.

If I was sitting across from Derren Brown or similar skilled street magician I'd say there's much more than a 1 in a thousand chance that he'd predict that I'd one box.

If the problem was stated with a lesser difference, say 1000 vs 5000 I might 2 box in part because a certain payoff is worth more to me than an uncertain one even if the expected return on the gamble is marginally higher.

I don't see how precommitment is relevant, whether you are "real" or a simulation. Omega knows what you will do even if you don't, so why bother precommitting?

For the record, when I first really considered the problem, my reasoning was still very similar. It ran approximately as follows:

"The more strongly I am able to convince myself to one-box, the higher the probability that any simulations of me would also have one-boxed. Since I am currently able to strongly convince myself to one-box without prior exposure to the problem, it is extremely likely that my simulations would also one-box, therefore it is in our best interests to one-box."

Note that I did not run estimated probabilties and tradeoffs based on the sizes of the reward, error probability of Omega, and confidence in my ability to one-box reliably. I am certain that there are combinations of those parameters which would make two-boxing better than one, but I did not do the math.

He makes a statement about the future which, when evaluated, is true. What's the difference between accurate predictions and omniscience?"

So when I look at the source code of a program and state "this program will throw a NullPointerException when executed" or "this program will go into endless loop" or "this program will print out 'Hello World'" I'm being omniscient?

Look, I'm not discussing Omega or Newcomb here. Did you just call ME omniscient because in real life I can predict the outcome of simple programs?

WTF? Who creates a system in which they can offer either some help now, or significantly more later, unless they are malicious or running an experiment?

Situations where you can get something now or something better later but not both come up all the time as consequences of growth, investment, logistics, or even just basic availability issues. I expect it would usually make more sense to do this analysis yourself and only offer the option that does more long-term good, but if clients' needs differ and you don't have a good way of estimating, it may make sense to allow them to choose.

Not that it's much of an offer if you can reliably predict the way the vast majority of them will go.

If the Predictor is accurate or omniscient, then the game is rigged and it becomes a different problem. If the Predictor is making guesses then box predicting and box selecting are both interesting to figure out.

WTF? Who creates a system in which they can offer either some help now, or significantly more later, unless they are malicious or running an experiment?

Or you live in a system nobody in particular created (capitalism) and work at a social service with limited resources with a clientele who have no background experience with adults who can be trusted. An employer telling hem "work now and I'll pay you later" is not convincing, while a peanut butter sandwich right now is.

What does always choosing some now over more later have to do with Newcomb's problem?

Simply stating that box B either contains a million dollars or nothing will make people see the million dollars as more distant than the guaranteed thousand in box A, I imagine. That the probabilities reduce that distance to negligible matters only if the person updates appropriately on that information.

This is also a perfectly reasonable explanation for Omega's success rate.

magician isn't engaging in prediction but is cheating.

If by cheating you mean "removes the reward after you made your choice to two-box", this can be mitigated by having a third party, like philosopher's friend, write down what's in the boxes before the philosopher in question makes his decision. I imagine that in such a situation the number of two-boxers would go down dramatically.

I will really want to just take them both.

Maybe the first time. What will you do the second time?

If I have even a little bit of reason to believe the problem is newcomboid (like, I saw it make two or three successful predictions, and no unsuccessful ones, or I know the omega would face bad consequences for predicting wrongly (even just in terms of reputation), or I know the omega studied me well), I'd one box in real-life too.

Right, good point about revealed reflective inconsistency. I'd guess that repeated experiments would probably turn any two-boxer into a one-boxer pretty quickly, if the person actually cares about the payoff, not about making a point, like Asimov supposedly would, as quoted by William Craig in this essay pointed out by Will Newsome. And those who'd rather make a point than make money can be weeded out by punishing predicted two-boxing sufficiently harshly.

Two-boxers grant that one-boxing gets you more money.

I'm not sure they do.

Social signalling doesn't make one easier to accurately predict. Costly signalling and precommitment costs might, but everyone rational would implement a free signal that made the judge more likely to put the money in the box- regardless of their actual intent.

I think the "think explicitly about decision theory" part was supposed to indicate the direction, since CDT as been the leader.

Um, huh?

I don't enjoy word games, but what does "instead of" mean here?

Yes. Which is to say, clearly you fall into the second class of people (those who have studied decision theory a lot) and hence my explanation was not meant to apply to you.

Which isn't to say I agree with everything you say.

From the standpoint of reflective consistency, there should not be a divergence between rational decisions and rational algorithms; the rational algorithm should search for and output the rational decision,

Decisions can have different causal impacts to decision theories and so there seems to be no reason to accept this claim. Insofar as the rational decision is the decision which wins which depends on the causal effects of the decision and the rational algorithm is the algorithm which wins which depends on the causal effects of the algorithm then there seems to be no reason to think these should coincide. Plus, I like being able to draw distinctions that can't be drawn using your terminology.

and the rational decision should be to adopt the rational algorithm.

Agreed (if you are faced with a decision of which algorithm to follow). Of course, this is not the decision that you're faced with in NP (and adding more options is just to deny the hypothetical)

Suppose you regard Newcomb's Problem as rewarding an agent with a certain decision-type, namely the sort of agent who one-boxes. TDT can be viewed as an algorithm which searches a space of decision-types and always decides to have the decision-type such that this decision-type has the maximal payoff. (UDT and other extensions of TDT can be viewed as maximizing over spaces broader than decision-types, such as sensory-info-dependent strategies or (in blackmail) maximization vantage points).

Yes, and I think this is an impressive achievement and I find TDT/UDT to be elegant, useful theories. The fact that I make the distinction between rational theories and rational decisions does not mean I cannot value the answers to both questions.

once you realize that a rational algorithm can just as easily maximize over its own decision-type as the physical consequences of its acts, there is just no reason to regard two-boxing as a winning decision or winning action in any sense, nor regard yourself as needing to occupy a meta-level vantage point in which you maximize over theories.

Well...perhaps. Obviously just because you can maximise over algorithms, it doesn't follow that you can't still talk about maximising over causal consequences. So either we have a (boring) semantic debate about what we mean by "decisions" or a debate about practicality: that is, the argument would be that talk about maximising over algorithms is clearly more useful than talk about maximising over causal consequences so why care about the second of these. For the most part, I buy this argument about practicality (but it doesn't mean that two-boxing philosophers are wrong, just that they're playing a game that both you and I feel little concern for).

This seems akin to precommitment, and precommitment means dynamic inconsistency means reflective inconsistency. Trying to maximize over theories means you have not found the single theory which directly maximizes without any recursion or metaness, and that means your theory is not maximizing the right thing.

I know what all these phrases mean but don't know why it follows that your theory is not maximising the "right" thing. Perhaps it is not maximising a thing that you find to be useful or interesting (particularly for self-modifying AIs). If this is what you mean then fine. If you mean, however, that two-boxers are wrong on their own terms then I would need a more compelling argument (I've read your TDT paper btw, so reference to that won't resolve things here).

Claiming that TDTers are maximizing over decision theories, then, is very much a CDT standpoint which is not at all how someone who sees logical decision theories as natural would describe it. From our perspective we are just picking the winning algorithm output (be the sort of agent who picks one box) in one shot, and without any retreat to a meta-level. The output of the winning algorithm is the winning decision, that's what makes the winning algorithm winning.

Sure, the distinction is from the CDT perspective. You use words differently to the proponents of CDT (at which point, the whole difference between LWer views and philosopher's views should be unsurprising). I'm not really interested in getting into a semantic debate though. I think that LWers are too quick to think that philosophers are playing the game wrong whereas I think the view should actually be that they're playing the wrong game.

First, removal of those scenarios is inconsistent with CDT as it is normally interpreted: CDT evaluates the utility of an act by the expected outcome of an exogenous choice being set without dependence on past causes, i.e. what would happen if a force from some unanticipated outside context came in and forced you to one-box or two-box, regardless of what you would otherwise have done.

It doesn't matter if the counterfactual computed in this way is unphysical, at least without changing the theory.

Second, to avoid wrangling over this, many presentations add small or epsilon error rates (e.g. the Predictor flips a weighted coin to determine whether to predict accurately or inaccurately, and is accurate 99% of the time, or 99.999999% of the time). What's your take with that adjustment?

CDT doesn't assign credences to outcomes in the way you are suggesting.

One way to think about it is as follows: Basically CDT says that you should use your prior probability in a state (not an outcome) and update this probability only in those cases where the decision being considered causally influences the state. So whatever prior credence you had in the "box contains $M" state, given that the decision doesn't causally influence the box contents, you should have that same credence regardless of decision and same for the other state.

There are so many different ways of outlining CDT that I don't intend to discuss why the above account doesn't describe each of these versions of CDT but some equivalent answer to that above will apply to all such accounts.

We need to distinguish two meanings of "being a proponent of CDT". If by "be a proponent of CDT" we mean, "think CDT describes the rational decision" then the answer is simply that the CDTer thinks that rational decisions relate to the causal impact of decisions and rational algorithms relate to the causal impact of algorithms and so there's no reason to think that the rational decision must be endorsed by the rational algorithm (as we are considering different causal impacts in the two cases).

If by "be a proponent of CDT" we mean "think we should decide according to CDT in all scenarios including NP" then we definitely have a problem but no smart person should be a proponent of CDT in this way (all CDTers should have decided to become one-boxers if they have the capacity to do so because CDT itself entails that this is the best decision)

No, it's mainly people who weren't familiar enough with the problem to have a view.

My initial reaction is to find that aggravating and to try to come up with another experiment that would allow me to poke at the universe by exploiting the Predictor, but it seems likely that this too would be sidestepped using the same tactic. So we could generalize to say that any experiment you come up with that involves the Predictor and gives evidence regarding the temporal direction of causation will be sidestepped so as to give you no new information.

But intuitively, it seems like this condition itself gives new information in the paradox, yet I haven't yet wrapped my head around what evidence can be drawn from it.

On another note, even if causality flows always forward, it is possible that humans might be insufficiently affected by nondeterministic phenomena to produce significantly nondeterministic behavior, at least at the time scale we're talking about. If that is the case, then it could potentially be the case that human reasoning has approximate t-symmetry over short time scales, and that this can be exploited to "violate causality" with respect to humans without actually violating causality with respect to the universe at large.

Which means that I have a more general hypothesis, "human reasoning causality can be violated" for which the violation of causality in general would be strong evidence, but the non-violation of causality would only be weak counter-evidence. And in learning of the Predictor's success, I have observed evidence strongly recommending this hypothesis.

So upon further consideration, I think that one-boxing is probably the way to go regardless, and it must simply be accepted that if you actually observe the Predictor, you can no longer rely on CDT if you know that such an entity might be involved.

The only part of the paradox that still bugs me then is the hand-waving that goes into "assume you believe the Predictor's claims". It is actually hard for me to imagine what evidence I could observe for that which would both clearly distinguish "the Predictor is honest" hypothesis from the "I'm being cleverly deceived" and "I've gone crazy" hypotheses, and also does not directly tip the Predictor's hand as to whether human reasoning causality can be violated.

It seems to me that there is backward causation under the decoherence interpretation, as the world we inhabit is affected by the experimental set-up (there's either a diffraction pattern on the back screen characteristic of a wave, or a pattern characteristic of a single slit). I really think people tend to overestimate the latitude that exists among the various quantum interpretations. They are just interpretations, after all.

I don't think that Omega knowing a person better than they know themselves is sufficient to explain the 100% accuracy of Omega's prediction.

I've had people choose only Box A because "screw you, that's why."

From the context, I would presume "about" in the sense of "this is why it's fascinating to the people who make a big deal about it". (I realise the stated reason for LW interest is the scenario of an AI whose source code is known to Omega having to make a decision, but the people being fascinated are humans.)

It's still more about magic and time-reversed causation than it is about deciding which box to take.

Particularly since it rewards the reflexively inconsistent agent that, at the time the money was placed, was going to one-box when it had the chance, but at the time the decision was made two-boxed. (At time A, when Omega makes the prediction, it is the case that the highest-performing decision model will at time B select one box; at time B, the highest-performing model selects both boxes.)

I'm just noticing that all the arguments in favor of 2boxing that I've read seem to come down to refusal to believe that Omega can be a perfect predictor.

That's hits what I meant pretty much on the head. If Omega is a perfect predictor, then it is meaningless to say that the human is making a choice.

Oh, and Parfit's Hitchhiker highlights that the concept of honor is a layman's version of reflective consistency: you tell the driver that you are a (wo)man of your word, because you truly are, for decision-theoretical reasons, as well as because you were brought up this way.

So, my hypothesis predicts that theists will not do better on "Newcomblike problems" not involving deities.

I've always had trouble with this part. I went through the reasoning that Newcomb is two PDs side by side, but this side-by-sideness seems to kill the essential part of PD, its unpredictability. Newcomb is perfectly deterministic, whereas in PD you depend on what the other party will do and often hope that they are reflectively consistent. The one-shot counterfactual mugging is again different from one-shot PD, even if one is reflectively consistent.