"Rational Agents Win"

"Good point", says Irene. She opens the second box, and goes home with $1,001,000. Why shouldn't she? Omega's dead.

SIMULATION COMPLETE.

RESULT: BOTH BOXES TAKEN.

OMEGA-COMMITMENT CONSISTENT ACTION: OPAQUE BOX EMPTY

Irene promptly walks up to the opaque box and opens it, revealing nothing.  She stares in shock.

I liked the story, but I think it misses some of how newcomb's problem works (at least as I understand it). Also, welcome to LessWrong! I think this is a pretty good first post.

So, this is not a problem of decision theory, but a problem with Omega's predictive capabilities in this story (maybe she was always right before this, but she wasn't here).

In the case where you find yourself holding the $1,000,000 and the $1000 are still available, sure, you can pick them up. That only happens if either Omega failed to predict what you will do, or if you somehow set things up such that you couldn't, or had to pay a big price, to break your precommitment.

If Omega really had perfect predictive capabilities in this story this is what would happen

-

Irene promptly walks up to the opaque box and opens it, revealing... Nothing.

"What?!" Irene exclaimed, "But I didn't mean to open the transparent box. I precommited to not doing it!"

"Maybe Omega knew better than you." Rachel said, "Anyway, now that you're only left with the $1000 in the transparent box, are you going to take them?"

"What? no! Then I would break my commitment and prove Omega right!"

"You can either do that, or walk out with a $1000. You're not getting anything from not taking it, I'm sure if I threatened you to take it you would. That's the rational decision."

Irene sighed, "Good point, I guess." And she opened the box and walked out with a $1000.

-

Also it assumes a non iterated game, and by not iterated I don't just mean she doesn't play against Omega again, but doesn't play against anyone else again, otherwise this becomes part of her reputation and thus is no longer "free".

Therefore, the world where the events take place is not real, it's a figment of past-Omega's imagination, predicting what you'd do. Your actions in the counterfactual just convinced past-Omega not to put the $1M in the box in the real world. Usually you don't have a way of knowing whether the world is real, or what relative weight of reality it holds. Though in this case the reality of the world was under your control.

I don't see the point of making the first box opaque, if you can open it and then decide whether to take the second box.

Here's a dumb question...  In the version of this paradox where some agent can perfectly predict the future, why is it meaningful or useful to talk about "decisions" one might make?

Assuming perfect prediction by Omega:

She opens the second box, and goes home with $1,001,000. Why shouldn't she? Omega's dead.

No, she goes home with $1,000. She two-boxes, so Omega predicted she'd two-box.

"But Omega is dead, and she sees the $1,000,000."

Sure, but you can't have it both ways. You can't have a perfect predictor and have Irene disprove the prediction.

If Omega's prediction isn't perfect, but e.g. 99% accurate, two-boxing on a one-box prediction might be possible, but still isn't wise, since then Omega will almost certainly have put the $1,000,000 in the box.

"Good point", says Irene. She opens the second box, and goes home with $1,001,000. Why shouldn't she? Omega's dead.

Epilogue: The next day, Irene is arrested for attempting to deposit obviously forged notes. One of the notes buried in one of the stacks was just a cartoon of Omega thinking .oO(Sigh).

Fortunately the authorities believe Irene's story of how she got the fake money. This sort of thing happens all the time, after all. And she did get to keep the real $1000.

In a Newcombless problem, where you can either have $1,000 or refuse it and have $1,000,000, you could argue that the rational choice is to take the $1,000,000, and then go back for the $1,000 when people's backs were turned, but it would seem to go against the nature of the problem.

In much the same way, if Omega is a perfect predictor, there is no possible world where you receive $1,000,000 and still end up going back for the second. Either Rachel wouldn't have objected, or the argument would've taken more than 5 minutes, and the boxes disappear, or something.

I'm not sure how Omega factors in the boxes' contents in this "delayed decision" version. Like, let's say Irene is will, absent external forces, one box, and Rachel, if Irene receives $1,000,000, will threaten Irene sufficiently to take the second box, and will do nothing if Irene receives nothing. (Also they're automatons, and these are descriptions of their source code, and so no other unstated factors are able to be taken into account)

Omega simulates reality A, with the box full, sees that Irene will 2 box after threat by Rachel.

Omega simulates reality B, with the box empty, and sees that Irene will 1 box.

Omega, the perfect predictor, cannot make a consistent prediction, and, like the unstoppable force meeting the immovable object, vanishes in a puff of logic.

I think, if you want to aim at this sort of thing, the better formulation is to just claim that Omega is 90% accurate. Then there's no (immediate) logical contradiction in receiving the $1,000,000 and going back for the second box. And the payoffs should still be correct.

1 box: .9*1,000,000 + .1*0 = 900,000

2 box: .9*1,000 + .1*1,001,000 = 101,000

I expect that this formulation runs folly of what was discussed in this post [LW · GW] around the Smoking Lesions problem, where repeated trials may let you change things you shouldn't be able to (in their example, if you chose to smoke every time, then if the correlation between smoking and lesions was held, then you can change the base rate of the lesions).

That is, I expect that if you ran repeated simulations, to try things out, then strategies like "I will one box, and iff it is full, then I will go back for the second box" will make it so Omega is incapable of predicting at the proposed 90% rate.

I think all of these things might be related to the problem of embedded agency, and people being confused (even if they don't put it in these terms) that they have an atomic free will that can think about things without affecting or being affected by the world. I'm having trouble resolving this confusion myself, because I can't figure out what Omega's prediction looks like instead of vanishing in a puff of logic. It may just be that statements like "I will turn the lever on if, and only if, I expect the lever to be off at the end" are a nonsense decision criteria. But the problem as stated doesn't seem like it should be impossible, so... I am confused.

Some clarifications on my intentions writing this story.

Omega being dead and Irene having taken the money from one box before having the conversation with Rachel are both not relevant to the core problem. I included them as a literary flourish to push people's intuitions towards thinking that Irene should open the second box, similar to what Eliezer was doing here [LW · GW].

Omega was wrong in this scenario, which departs from the traditional Newcomb's problem. I could have written an ending where Rachel made the same arguments and Irene still decided against doing it, but that seemed less fun. It's not relevant whether Omega was right or wrong, because after Irene has made her decision, she always has the "choice" to take the extra money and prove Omega wrong. My point here is that leaving the $1000 behind falls prey to the same "rational agents should win" problem that's usually used to justify one-boxing. After taking the $1,000,000 you can either have some elaborate justification for why it would be irrational to open the second box, or you could just... do it.

Here's another version of the story that might demonstrate this more succinctly:

Irene wakes up in her apartment one morning and finds Omega standing before her with $1,000,000 on her bedside table and a box on the floor next to it. Omega says "I predicted your behavior in Newcomb's problem and guessed that you'd take only one box, so I've expedited the process and given you the money straight away, no decision needed. There's $1000 in that box on the floor, you can throw it away in the dumpster out back. I have 346 other thought experiments to get to today, so I really need to get going."