The Doubling Box

This problem makes more sense if you strip out time and the doubling, and look at this one:

Choose an integer N. Receive N utilons.

This problem has no optimal solution (because there is no largest integer). You can compare any two strategies to each other, but you cannot find a supremum; the closest thing available is an infinite series of successively better strategies, which eventually passes any single strategy.

In the original problem, the options are "don't open the box" or "wait N days, then open the box". The former can be crossed off; the latter has the same infinite series of successively better strategies. (The apparent time-symmetry is a false one, because there are only two time-invariant strategies, and they both lose.)

The way to solve this in decision theory is to either introduce finiteness somewhere that caps the number of possible strategies, or to output an ordering over choices instead of a single choice. The latter seems right; if you define and prove an infinite sequence of successively better options, you still have to pick one; and lattices seem like a good way to represent the results of partial reasoning.

I've never met an infinite decision tree in my life so far, and I doubt I ever will. It is a property of problems with an infinite solution space that they can't be solved optimally, and it doesn't reveal any decision theoretic inconsistencies that could come up in real life.

Consider this game with a tree structure: You pick an arbitrary natural number, and then, your opponent does as well. The player who chose the highest number wins. Clearly, you cannot win this game, as no matter which number you pick, the opponent can simply add one to that number. This also works with picking a positive rational number that's closest to 1 - your opponent here adds one to the denominator and the numerator, and wins.

The idea to use a busy beaver function is good, and if you can utilize the entire universe to encode the states of the busy beaver with the largest number of states possible (and a long enough tape), then that constitutes the optimal solution, but that only takes us further out into the realm of fiction.

After considering this problem, what I found was surprisingly fast, the specifics of the boxes physical abilities and implementation becomes relevant. I mean, let's say Clippy is given this box, and has already decided to wait a mere 1 year from day 1, which is 365.25 days of doubling, and 1 paperclip is 1 utilon. At some point, during this time, before the end of it, There are more paperclips then there used to be every atom in the visible universe. Since he's predicted to gain 2^365.25 paperclips, (which is apparently close to 8.9*10^109) and the observable universe is only estimated to contain 10^80 atoms. So to make up for that, let's say the box converts every visible subatomic particle into paperclips instead.

That's just 1 year, and the box has already announced it will convert approximately every visible subatomic particle into pure paperclip bliss!

And then another single doubling... (1 year and 1 day) Does what? Even if Clippy has his utility function unbounded, it should presumably still link back to some kind of physical state, and at this point the box starts having to implement increasingly physically impossible ideas to have to double paperclip utility, like:

Breaking the speed of light.

Expanding the paperclip conversion into the past.

Expanding the paperclip conversion into additional branches of many worlds.

Magically protecting the paperclips from the ravages of time, physics, or condensing into blackholes, despite the fact it is supposed to lose all power after being opened.

And that's just 1 year! We aren't even close to a timeless eternity of waiting yet, and the box already has to smash the currently known laws of physics (more so than it did by converting every visible subatomic particle into paperclips) to do more doublings, and will then lose power afterwards.

Do the laws of physics resume being normal after the box loses power? If so, massive chunks of utility will fade away almost instantly (which would seem to indicate the Box was not very effective), but if not I'm not sure how the loop below would get resolved:

The Box is essentially going to rewrite the rules of the universe permanently,

Which would affect your utility calculations, which are based on physics,

Which would affect how the Box rewrote the rules of the universe,

Which would affect your utility calculations, which are based on physics,

Except instead of stopping and letting you and the box resolve this loop, it must keeps doubling, so it keeps changing physics more.

By year 2, it seems like you might be left with either:

A solution, in which case whatever the box will rewrite the laws of physics to, you understand and agree with it and can work on the problem based on whatever that solution is. (But I have no idea how you could figure out what this solution would be in advance, since it depends on the specific box?)

Or, an incredibly intractable metaphysics problem which is growing more complicated faster than you can ever calculate, in which case you don't even understand what the box is doing anymore.

The reason I said that this was incredibly fast is that my original guess was that it would take at least 100 years of daily doubling for the proposed world to become that complicated, but when I tried doing a bit of math it didn't take anywhere near that long.

Edit: Fixed a few typos and cleared up grammar.

Your other option is to sell the box to the highest bidder. That will probably be someone who's prepared to wait longer than you, and will therefore be able to give you a higher price than the utilons you'd have got out of the box yourself. You get the utilons today.

If you can use mixed strategies (i.e. are not required to be deterministically predictable), you can use the following strategy for the doubling-utility case: every day, toss a coin; if it comes up heads, open the box, otherwise wait another day. Expected utility of each day is constant 1/2, since the probability of getting heads on a particular day halves with each subsequent day, and utility doubles, so the series diverges and you get infinite total expected utility.

So I don't really know how utilons work, but here is an example of a utility function which is doubling box-proof. It is bounded; furthermore, it discounts the future by changing the bound for things that only affect the future. So you can get up to 1000 utilons from something that happens today, up to 500 utilons from something that happens tomorrow, up to 250 utilons from something that happens two days from now, and so on.

Then the solution is obvious: if you open the box in 4 days, you get 16 utilons; if you open the box in 5 days, you'd get 32 but your utility function is bounded so you only get 31.25; if you open the box in 6 days or more, the reward just keeps shrinking. So 5 days is the best choice, assuming you can precommit to it.

A suitable transformation of this function probably does capture the way we think about money: the utility of it is both bounded and discounted.

Am I correct in assessing that your solution is to stop when you can no longer comprehend the value in the box? That is, when an additional doubling has no subjective meaning to you? (Until that point, you're not in a state loop, as the value with each doubling provides an input you haven't encountered before.)

I was about to suggest stopping when you have more utilons than your brain has states (provided you could measure such), but then it occurred to me the solutions might be analogous, even if they arrive at different numbers.

I wait until there are so many utiltons in the box that I can use them to get two identical boxes and have some utiltions left over. Every time a box has more than enough utilitons to make two identical boxes, I repeat that step. Any utilitons not used to make new boxes are the dividend of the investment.

What if instead of growing exponentially without bound, it decays exponentially to the bound of your utility function?

I think you mean 'asymptotically'.

I way to think about this problem to put you in near mode is to imagine what the utility might look like. Ex:

Day 1: Finding a quarter on the ground

Day 2: A child in Africa getting $5

.....

Day X: Curing cancer

Dax X+1: Curing cancer, Alzheimers, and AIDS.

On one hand, by waiting a day, more people would die of cancer. On the other, by not waiting, you'd doom all those future people to die of AIDS and Alzheimers.

How exactly do the constant utilons in the box compensate me for how I feel the day after I open the box (I could have doubled my current utility!)? The second day after (I could have quadrupled my current utility!!)? The Nth day after (FFFFFFFFFFFFUUUUU!!!)? I'm afraid the box will rewrite me with a simple routine that says "I have 2^(day-I-opened-the-box - 1) utility! Yay!"

If you return to a state you have already been at, you know you are going to be waiting forever and lose and get nothing.

You seem to be assuming here that returning to a state you have already been at is equivalent to looping your behavior, so that once a Turing machine re-enters a previously instantiated state it cannot exhibit any novel behavior. But this isn't true. A Turing machine can behave differently in the same state provided the input it reads off its tape is different. The behavior must loop only if the the combination of Turing machine state and tape configuration recurs. But this need never happen as long as the tape is infinite. If there were an infinite amount of stuff in the world, even a finite mind might be able to leverage it to, say, count to an arbitrarily high number.

Now you might object that it is not only minds that are finite, but also the world. There just isn't an infinite amount of stuff out there. But that same constraint also rules out the possibility of the utility box you describe. I don't see how one could squeeze arbitrarily large amounts of utility into some finite quantity of matter.

You have given reasons why requiring bounded utility functions and discounting the future are not adequate responses to the problem if considered individually. But your objection to the bounded utility function response assumes that future utility isn't discounted, and your objection to the discounting response assumes that the utility function is unbounded. So what if we require both that the utility function must be bounded and that future utility must be discounted exponentially? Doesn't that get around the paradox?

I remember reading a while ago about a paradox where you start with $1, and can trade that for a 50% chance of $2.01, which you can trade for a 25% chance of $4.03, which you can trade for a 12.5% chance of $8.07, etc (can't remember where I read it).

The problem statement isn't precisely the same as what you specify here, but were you thinking of the venerable St. Petersburg paradox?

If you know the probability distribution P(t) of you dying on day t, then you can solve exactly for optimal expected lifetime utilons out of the box. If you don't know P(t), you can do some sort of adaptive estimation as you go.

1.  Open the box when and iff you need the utilon.  This simple solution gets you the maximum utilons if you need a utilon and none if you don't need a utilon.

I asked Ask Philosophers about this a few years ago.

You could build a machine that opens the box far in the future, at the moment when the machine's reliability starts degrading faster than the utilons increase. This maximizes your expected utility.

Or if you're not allowed to build a machine, you simply do the same with yourself (depending on our model, possibly multiplying by your expected remaining lifespan).

Bringing together what others have said, I propose a solution in three steps:

• Adopt a mixed strategy where, for each day, you open the box on that day with probability p. The expected utility of this strategy is the sum of (p (1-p)^n 2^n), for n=0... which diverges for any p in the half-open interval (0,0.5]. In other words, you get infinite EU as long as p is in (0,0.5]. This is paradoxical, because it means a strategy with a 0.5 risk of ending up with only 1 utilon is as good as any other.

• Extend the range of our utility function to a number system with different infinities, where a faster-growing series has greater value than a slower-growing series, even if they both grow without bound. Now the EU of the mixed strategy continues to grow as p approaches 0, bringing us back to the original problem: The smaller p is, the better, but there is no smallest positive real number.

• Realize that physical agents can only choose between a finite number of strategies (because we only have a finite number of possible mind states). So, in practice, there is always a smallest p: the smallest p we can implement in reality.

So that's it. Build a random number generator with as many bits of precision as possible. Run it every day until it outputs 0. Then open the box. This strategy improves on the OP because it yields infinite expected payout, and is intuitively appealing because it also has a very high median payout, with a very small probability of a low payout. Also, it doesn't require precommitment, which seems more mathematically elegant because it's a time-symmetric strategy for a time-symmetric game.

There is a good reason to use a bounded utility function, note -- if the conditions of Savage's theorem hold, the utility function you get from it is bounded.

How long do you wait before opening it? If you never open it, you get nothing (you lose! Good day, sir or madam!) and whenever you take it, taking it one day later would have been twice as good.

When do I "lose" precisely? When I never take it? By happy coincidence 'never' happens to be the very next day after I planned to open the box!

There are no other ways to get utilons.

Is a weakness in your argument. Either you can survive without utilons, a contradiction to utility theory, or you wait until your "pre-existing" utilons are used up and you need more to survive.

If I am actually immortal, and there is no other way to get Utilions then each day, the value of me opening the box is something like:

Value=Utilions/Future Days

Since my Future Days are supposedly infinite, we are talking about at best an infinitesimal difference between me opening the box on Day 1 and me opening the box on Day 3^^^^3. There is no actual wrong day to open the box. If that seems implausible, it is because the hypothetical itself is implausible.

I admire the way this post introduces an ingenious problem and an ingenious answer.