Newcomb II: Newer and Comb-ier

post by Nathaniel Monson (nathaniel-monson) · 2023-07-13T18:49:00.989Z · LW · GW · 11 comments

Contents

11 comments

So, you're walking along and a giant Greek letter appears. It snaps its serifs, and two boxes materialize before you with a puff of reality-defying smoke. The one on your left is clear as a blue sky and crammed full of cash. The one on your right is featureless and perfectly opaque.

"I've heard of you! You're Omega. In the '60s, you were flying around giving everyone money! I've thought about this and decided I would probably one-box. Although now that I'm here, a guaranteed pile of money sure looks nice..." You're delighted by your good fortune, but indecision paralyzes you. You lean rightward, then glance back at the tempting transparent box.

"Whoa there, cowboy. If what you've just told me is true, your big payday in the right box is as empty as a politician's promise."

Your step halts. "Hold on. I thought this worked like...you stuff the right box with bills if, and only if, you predicted I'd pick that one alone."

"I used to, sure. For a solid year, I was handing out cash like Halloween candy, about 3000 times a day. Got a good chunk of data, and then, yawn, it got boring."

You frown. "Wait, what do you mean it got boring? Aren't you a super-intelligence? Wasn't it boringly predictable from the beginning? I heard you were right every time."

"Well, predicting whether people will one-box or two-box is a no-brainer. Their emotional aftermath? Just as easy to guess. Lots of jolly one-boxers, plenty of grumpy two-boxers, and a sprinkle of joyous two-boxers. All too predictable. But boy, the mental gymnastics and philosophical essays people churned out? That was the real popcorn material. Some believed I was simply rewarding a form of irrationality, while others argued that whatever I rewarded was, by definition, rational.."

Your frown deepens. "Well, which is it? Is it more rational to one-box or to two-box?"

""LOL, no idea, dude. I don't really have a concept of 'rational' or 'irrational'. I just play my games, watch the fallout, and enjoy the show. It's like you watching ripples in a pond. And right now, my current kick is to present you with a choice. The right box? As empty as a deserted ghost town. But this time, I've tossed in a new twist: do anything different from what you'd do in the classic Newcomb problem, and poof, you're gone, dead, an ex-parrot. After a year of this, I'll have my happy two-boxers, and my not-so-happy one-boxers. But what will really make my millennia will be the unhappy two-boxers and the ecstatic one-boxers. Which one will you be?"

"I understand the happy two-boxers. They have lots of money, and those who were proponents of causal decision theory will feel vindicated. And the unhappy one-boxers, I get. They had committed to a certain course of action which they thought would be positive and proved to be a bad idea. But what's up with the other groups?"

"Well, here's the thing. I'm not going to end up killing anyone--it turns out that superintelligences which are also Greek letters are psuedo-aligned by default. (You can ponder whether that's me fibbing about my death threat, or if everyone will just believe me and end up making the consistent choice). The unhappy two-boxers are an interesting bunch. They've convinced themselves that one-boxing was the pinnacle of rational thought and prided themselves on their rationality, only to discover their self-deception. They claimed one-boxing for social cred, but when the chips were down, they were always going to take both boxes. Now they're filthy rich and facing a severe identity crisis! But the cream of the crop? The joyous one-boxers. They staked their lot on one-boxing, hoping for a big win, wealth, or something. And now, even though that very decision has backfired spectacularly (you haven't counted, but that clear box is overflowing with riches) they still manage to persuade themselves they made the right call! Think about it--they essentially said, 'in a situation where I could cause myself to have more utility, I precommit to not doing so', and then, entirely predictably, ended up with less utility than they could have had. But they're thrilled, because they think this is the true path to maximizing utility!" Omega roles through the air, laughing uproariously. "I don't think you have a word for the hyperemotion I get out of this. Oh, wait--amusement. That's probably the right word. Well? What will it be, champ?"

---

TL; DR: Why on earth would you decide now to one-box in Newcomb-likes? There is a vanishingly small chance of that happening to you, and it seems equally plausible a priori that you could get rewarded for the opposite commitment.  (In fact, more plausible--the position of 'I will make the decisions which I expect to cause my utility to increase the most' seems likely to...cause your utility to increase more than other positions). Yes, if you hear about and believe there is an actual Omega doing this actual thing, go ahead and decide then (and not before), "If this particular thing which is happening to people does happen to me, I will one-box", but your default should probably be two-boxing.

11 comments

Comments sorted by top scores.