Ruining an expected-log-money maximizer

the way we're treating infinties here

Yeah, that seems key. Even if the probability that Linda will eventually get 0 money approaches 1, that small slice of probability in the universe where she has always won is approaching an infinity far larger that Logan's infinity as the number of games approaches infinity. Some infinities are bigger than others. Linear utility functions and discount rates of zero necessarily deal with lots of infinities, especially in simplified scenarios. 

Linda can always argue that in every universe where she lost everything, there's more (6 vs 4) universes where her winnings were double what they would have been had she not taken that bet.
 

Kind of tangential to the post, but: if you're actually in a situation where you're facing bets like this in real life, denominated in actual utilons, it is worth thinking through whether there are any actions you can take outside of the thought experiment to extract more utility from whatever entity is offering these bets, by hook or by crook or negotiation or force, if necessary.

Even if you are reasonably confident that the entity is honest and the game is real, acting Lawfully doesn't necessarily mean playing by the rules set down by some strange entity, which you might never have agreed to abide by in the first place. (Maybe if the entity is a god or a superintelligence, doing anything other than playing by its rules would be futile and counterproductive, but it is still worth checking.)

Self review: I really like this post. Combined with the previous one [LW · GW] (from 2022), it feels to me like "lots of people are confused about Kelly betting and linear/log utility of money, and this deconfuses the issue using arguments I hadn't seen before (and still haven't seen elsewhere)". It feels like small-but-real intellectual progress. It still feels right to me, and I still point people at this when I want to explain how I think about Kelly.

That's my inside view. I don't know how to square that with the relative lack of attention the post got, and it feels weird to be writing it given that fact, but oh well. There are various stories I could tell: maybe people were less confused than I thought; maybe my explanation is unclear; maybe I'm still wrong on the object level; maybe people just don't care very much; maybe it just happened not to get seen.

If I were writing this today, my guess is:

• It's worth combining the two posts into one.
• The rank optimization stuff is fine to cut, given that I tentatively propose it in one post and then in the next say "probably not very useful". Maybe have a separate post for exploring it. No need to go into depth on "extending Kelly outside its original domain".
• The charity stuff might also be fine to cut. At any rate it's not a focus.
• Someone sent me an example function satisfying the "I'm pretty sure yes" criteria, so that can be included.
• Not sure if this belongs in the same place, but I'd still like to explore more the "what if your utility function is such that maximizing expected utility at time $t_1$ doesn't maximize expected utility at time $t_2$?" thing. (I thought I wrote this in the post somewhere, but can't see it: the way I'd explore this is from the perspective of "a utility function is isomorphic to a description of betting preferences that satisfy certain constraints, so when we talk about a utility function like that, what betting preferences are we talking about?" Feels like the kind of thing someone's likely already explored, but I haven't seen it if so.)

I would posit that humans behave in a much more optimal manner in terms of long-run quality of life than are given credit for, excluding gambling addicts.

A lot of people who are willing to bet everything (ie. follow a linear utility function) are lower income. It is more that just that, however. Lower income people just by necessity have less savings relative to income, so losing all their savings isn't a big deal compared to work-derived income. Losing a couple months of pay sucks, but eh.

People who like to think they're being more rational by not betting the farm usually just have more to lose. If you're a professional who accrued a few million over a few decades of work, you can't make it back; you will invest prudently, with diversification across asset classes and markets; you might not even keep everything in one country.

What would the optimal utility function look like for someone who has a steady income? I would expect it to smoothly transition from a linear to a log regime, as the total winnings exceed the income per turn. Fun textbook exercise for stats undergrads, maybe I'll use it sometime.

I'm not sure I've ever seen a treatment of utility functions that deals with this problem? (The problem being "what if your utility function is such that maximizing expected utility at time t1 doesn't maximize expected utility at time t2?") It's no more a problem for Linda than for Logan, it's just less obvious for Logan given this setup.

In economics, this is considered via the time-value of money. It is considered at the market level, however, not at the level of individuals so plausibly it could have individual variances.

This assumes that both utility functions are implemented with perfect reason, unlimited intellect, and no interference from emotion.

Can I instead offer [Logan] bets where he chooses how much of his money to put in, and he still puts in all but a penny?

It appears to an outside observer that "Logan has a utility function that's logarithmic in money. He'll bet 20% of his bankroll every time, and his wealth will grow exponentially." I'd posit that Logan's internal narrative about gambling, which manifests as appearing to be the stated utility function, is much more like "I don't like the risk of going broke so I'm going to bet in a way that seems unlikely to blow all my money on any one bet that might lose it".

Considering the emotional context of Logan's behavior with money, I think it's actually quite unlikely that you could persuade him to make any bet that will leave him forced to bet all his remaining money in the subsequent round to stay in the game if he loses it. I'm not sure what math words apply this lookahead that a human Logan would perform to say "that's a tempting bet but if I lose it I'm screwed in the next round and I don't want to be screwed", but that's a type of thought and behavior that I think your utility function modeling neglects to account for.

If Logan was perfectly intelligent and believed himself to be so, he might behave in a perfectly rational manner even when all but 1 cent of his money was at stake. But I don't think you could introduce me to any human in the world who both is perfectly intelligent, and believes that they are. There are people who erroneously believe they're perfectly intelligent, but they rarely believe in the way that a perfectly rational person would be expected to. There are highly rational people who know their intellectual limitations, but one trait of that rationality is planning ahead and considering that their appraisal of the odds of any gamble could be inaccurate, and keeping a financial safety net in case they make some mistake.

In short, I think I've entirely missed the point of why it's useful to speculate about the behavior of hypothetical people whose behavior differs so significantly from what we see in actual people.

I think it's fine to bite the bullet that an unlimited linear utility function has the property of preferring an infinitesimal chance of a ludicrous payout.  Jane can be sad at a particular outcome without regretting her decisions.  Kelly optimizes Logan's utility function, and NOT Jane's.

I think you need to be a bit more formal with your termination conditions - infinity doesn't exist, and rounding things off means you're making incorrect inferences.    An example is when you say 

According to my usage of the term, one bets Kelly when one wants to "rank-optimize" one's wealth, i.e. to become richer with probability 1 than anyone who doesn't bet Kelly, over a long enough time period.

this is simply incorrect, and contradicts your above analysis of Jane's preferences. In fact, Kelly is only richer than "bet it all, every time" with a probability equal to the "bet it all" strategy's likelihood of ruin.  Kelly is poorer than "bet it all" otherwise.  This probability is never 1, though it can get arbitrarily close.  But the value of the (tiny) probability of continued wins is so much larger than Kelly's outcome in that situation that the mean of ALL outcomes still favors the risk.  Reasonable termination conditions are "until the player dies or goes broke", "until the player meets a threshold or goes broke", "until the casino can't cover the bet (or the player goes broke)".  If you're feeling silly, "until the heat-death of the universe", but it's hard to really think about such utility functions, and we probably don't have enough time or compute capacity to handle the calculations cheaply.

I think a lot of confusion comes from considering a utility function that "seems reasonable", in a fairly narrow range of situations, and then extending them to unreasonable lengths and being surprised that it contradicts our intuitions.  Along with your observations that it's never the case that we have all this much knowledge about our own utility, or about the actual bets on offer.  In the real world, it pays to be very suspicious of probability or amount calculations that are very large or very small - the unknowns and outlier events come to dominate those decisions.

The problem with Linda's betting strategy is that as the number of bets approaches infinity, the worlds where she wins end up with probability zero.

Incidentally, you can be offered an infinite set of bets such that if you accept any finite subset of them you have positive expected value, but if you accept the entire infinite set you're guaranteed to lose money.

why on Earth would I offer any of these bets? They all lose money for me. Am I just a cosmic troll trying to bankrupt some utilitarians or something?

Casino would definitely attempt that, I think.

The game is zero-sum; if you have significantly larger money than the bettor, you may try to bankrupt him and get his money with very high probability, and it is easier to do that against Linda than against Logan.

I suppose one thing you could do here is pretend you can fit infinite rounds of the game into a finite time. Then Linda has a choice to make: she can either maximize expected wealth at $t_n$ for all finite $n$, or she can maximize expected wealth at $t_{\omega }$, the timestep immediately after all finite timesteps. We can wave our hands a lot and say that making her own bets would do the former and making Logan's bets would do the latter, though I don't endorse the way we're treating infinties here.

If one strategy is best for $t_n$, it's still going to be best at  $t_{\omega }$ as t goes to infinity. Optimal strategies don't just change like that as n goes to infinity. Sure you can argue that p(won every time) --> 0, but also that number is being multiplied by an extremely large infinity, so you can't just say that it totals to zero (in fact, 1.2, which is her EV from a single game, raised to infinity is infinity, so I argue as n goes to infinity, her EV goes to infinity, not 0, and not a number less than the EV from Logan's strategy). 

Linda's strategy is always optimal with respect to her own utility function, even as n goes to infinity. She's not acting irrationally or incorrectly here.

The one world where she has won wins her $2**n, and that world exists with probability 0.6**n.

Her EV is always ($2**n)*(0.6**n), which is a larger EV (with any n) than a strategy where she doesn't bet everything every single time. Even as n goes to infinity, and even as probability approaches 1 that she has lost everything, it's still rational for her to have that strategy because the $2**n that she won in that one world is so massive that it balances out her EV. Some infinities are much larger than others, and ratios don't just flip when a large n goes to infinity.

"I'll give you £1 now and you give me £2 in a week". Will she accept?

In the universe where she's allowed to make the 60/40 doubled bet at least once a week, it seems like she's always say yes? I'm not seeing the universe in which she'd say no, unless she's using a non-zero discount rate that wasn't discussed here.

| I'm not sure I've ever seen a treatment of utility functions that deals with this problem?

Isn't this just discount rates?

A log money maximizer that isn't stupid will realize that their pennies are indivisible and not take your ruinous bet. They can think more than one move ahead. Discretised currency changes their strategy.

your utility function is your utility function [LW · GW]

The author is trying to tacitly apply human values to Logan while acknowledging Linda as following her own not human utility function faithfully.

Notice that the log(funds) value function does not include a term for the option value of continuing. If maximising EV of log(funds) can lead to a situation where the agent can't make forward progress (because log(0)=-inf so no risk of complete ruin is acceptable) the agent can still faithfully maximise EV(log(funds)) by taking that risk.

In much the same way as Linda faithfully follows her value function while incurring 1-ε risk of ruin, Logan is correctly valuing the log(0.01)=-2 as an end state.

Then you'll always be able to continue betting.

Humans don't like being backed into a corner and having no options for forward progress. If you want that in a utility function you need to include it explicitly.