Kelly isn't (just) about logarithmic utility

OK, here is my fuller response.

First of all, I want to state that Ergodicity Economics is great and I'm glad that it exists -- like I said in my other comment, it's a true frequentist alternative to Bayesian decision theory, and it does some cool things, and I would like to see it developed further.

That said, I want to defend two claims:

• From a strict Bayesian perspective, it's all nonsense, so those arguments in favor of Kelly are at best heuristic. A Bayesian should be much more interested in the question of whether their utility is log in money. (Note: I am not a strict bayesian [LW · GW], although I think strict Bayes is a pretty good starting point.)
• From a broader perspective, I think Peters' direction is interesting, but his current methodology is not very satisfying, and doesn't support his strong claims that EG Kelly naturally pops out when you take a time-averaging rather than ensemble-averaging perspective.

"It's All Nonsense"

Let's deal with the strict Bayesian perspective first.

You start your argument by discussing the "variance drag" phenomenon. You point out that the expected monetary value of an investment can be constant or even increasing, but it may be that due to variance drag, it typically (with high probability) goes down over time.

You go on to assert that "people need to account for this" and "adjust their strategy" because "variance is bad".

To a strict Bayesian, this is a non-sequitur. If it be the case that a strict Bayesian is a money-maximizer (IE, has utility linear in money), they'll see no problem with variance drag. You still put your money where it gets the highest expected value.

Furthermore, repeated bets don't make any material difference to the money-maximizer. Suppose the optimal one-step investment strategy has a tare of return R. Then the best two-step strategy is to use that strategy twice to get an expected rate of return R^2. This is because the best way they can set themselves up to succeed at stage 2 (in expectation!) is to get as much money as they can at stage 1 (in expectation!). This reasoning applies no matter how many stages we string together. So to the money-maximizer, your argument about repeated bets is totally wrong.

Where a strict Bayesian would start to agree with you is if they don't have utility linear in money, but rather, have diminishing returns.

In that case, the best strategy for single-step investing can differ from the best strategy for multi-step investing. 

Say the utility function is $u\left(x\right)$. Treat an investment strategy as a stochastic function $s\left(x\right)$. We assume the random variable $c\cdot s\left(x\right)$ behaves the same as the random variable $s(c\cdot x)$; IE, returns are proportional to investment. So, actually, we can treat a strategy as a random variable $S$ which gets multiplied by our money; $s\left(x\right)=S\cdot x$. 

In a one-step scenario, the Bayesian wants to maximize $E\left[u\right(s\left(x\right)\left)\right]$ where $x$ is your starting money. In a two-step scenario, the Bayesian wants to maximize $E\left[u\right(s_2\left(s_1\right(x\left)\right)\left)\right]$. And so on. The reason the strategy was so simple when $u\left(x\right)=x$ was that this allowed us to push the expectation inwards; $E\left[u\right(s_2\left(s_1\right(x\left)\right)\left)\right]=E\left[s_2\right(s_1\left(x\right)\left)\right]$ $=E[S_2\cdot S_1\cdot x]$ $=E\left[S_1\right]\cdot E\left[S_2\right]\cdot x$ (the last step holds because we assume the random variables are independent). So in that case, we could just choose the best individual strategy and apply it at each time-step.

In general, however, nonlinear $u$ stops us from pushing the expectations inward. So multi-step matters.

A many-step problem looks like $E\left[u\right(S_1\cdot S_2\cdot S_3\cdot ...\cdot S_n\cdot x\left)\right]$. The product of many independent random variables will closely approximate a log-normal distribution, by the multiplicative central limit theorem. I think this somehow implies that if the Bayesian had to select the same strategy to be used on all days, the Bayesian would be pretty happy for it to be a log-wealth maximizing strategy, but I'm not connecting the dots on the argument right now.

So anyway, under some assumptions, I think a Bayesian will behave increasingly like a log-wealth maximizer as the number of steps goes up.

But this is because the Bayesian had diminishing returns in the first place. So it's still ultimately about the utility function.

Most attempts to justify Kelly by repeated-bet phenomena overtly or implicitly rely on an assumption that we care about typical outcomes. A strict Bayesian will always stop you there. "Hold on now. I care about average outcomes!"

(And Ergodicity Economics says: so much the worse for Bayes!)

Bayesians may also raise other concerns with Peters-style reasoning:

• Ensemble averaging is the natural response to decision-making under uncertainty; you're averaging over different possibilities. When you try to time-average to get rid of your uncertainty, you have to ask "time average what?" -- you don't know what specific situation you're in.
• In general, the question of how to turn your current situation into a repeated sequence for the purpose of time-averaging analysis seems under-determined (even if you are certain about your present situation). Surely Peters doesn't want us to use actual time in the analysis; in actual time, you end up dead and lose all your money, so the time-average analysis is trivial. So Peters has to invent an imagined time dimension, in which the game goes on forever. In contrast, Bayesians can deal with time as it actually is, accounting for the actual number of iterations of the game, utility of money at death, etc.
• Even if you settle on a way to turn the situation into an iterated sequence, the necessary limit does not necessarily exist. This is also true of the possibility-average, of course (the St Petersburg Paradox being a classic example); but it seems easier to get failure in the time-avarage case, because you just need non-convergence; ie, you don't need any unbounded stuff to happen.

It's Not Nonsense, But It's Not Great, Either

Each of the above three objections are common fare for the Frequentist:

• Frequentist approaches start from the objective/external perspective rather than the agent's internal uncertainty. (For example, an "unbiased estimator" is one such that given a true value of the parameter, the estimator achieves that true value on average.) So they're not starting from averaging over subjective possibilities. Instead, Frequentism averages over a sequence of experiments (to get a frequency!). This is like a time-average, since we have to put it in sequence. Hence, time-averages are natural to the way Frequentists think about probability.
• Even given direct access to objective truth, frequentist probabilities are still under-defined because of the reference class problem -- what infinite sequence of experiments do you conceive of your experiment as part of? So the ambiguity about how to turn your situation into a sequence is natural.
• And, again, once you select a sequence, there's no guarantee that a limit exists. Frequentism has to solve this by postulating that limits exist for the kinds of reference classes we want to talk about.

So, actually, this way of thinking about decision theory is quite appealing from a Frequentist perspective.

Remember all that stuff about how a Bayesian money-maximizer would behave? That was crazy. The Bayesian money-maximizer would, in fact, lose all its money rather quickly (with very high probability). Its in-expectation returns come from increasingly improbable universes. Would natural selection design agents like that, if it could help it?

So, the fact that Bayesian decision theory cares only about in-expectation, rather than caring about typical outcomes, does seem like a problem -- for some utility functions, at least, it results in behavior which humans intuitively feel is insane and ineffective. (And philosophy is all about trying to take human intuitions seriously, even if sometimes we do decide human intuition is wrong and better discarded).

So I applaud Ole Peters' heroic attempt to fix this problem.

However, if you dig into Peters, there's one critical place where his method is sadly arbitrary. You say:

Compounding is multiplicative, so it becomes "natural" (in some sense) to transform everything by taking logs.

Peters makes much of this idea of what's "natural". He talks about additive problems vs multiplicative problems, as well as the more general case (when neither additive/multiplicative work).

However, as far as I can tell, this boils down to creatively choosing a function which makes the math work out.

I find this inexcusable, as-is.

For example, Peters makes much of the St Petersburg lottery:

A resolution of the St Petersburg paradox is presented. In contrast to the standard resolution, utility is not required. Instead, the time-average performance of the lottery is computed. The final result can be phrased mathematically identically to Daniel Bernoulli’s resolution, which uses logarithmic utility, but is derived using a conceptually different argument. The advantage of the time resolution is the elimination of arbitrary utility functions.

Daniel Bernoulli "solved" the St Petersburg paradox by suggesting that utility is log in money, which makes the expectations work out. (This is not very satisfying, because we can always transform the monetary payouts by an exponential function, making things problematic again! D Bernoulli could respond "take the logarithm twice" in such a case, but by this point, it's clear he's just making stuff up, so we ignore him.) 

Ole Peters offers the same solution, but, he says, with an improved justification: he promises to eliminate the arbitrary choice of utility functions.

He does this by arbitrarily choosing to examine the time-average behavior in terms of growth rate, which is defined multiplicatively, rather than additively. 

"Growth rate" sounds like an innocuous, objective choice of what to maximize. But really, it's sneaking in exactly the same arbitrary decision which D Bernoulli made. Taking a ratio, rather than a difference, is just a sneaky way to take the logarithm.

So, similarly, I see the Peters justification of Kelly as ultimately just a fancy way of saying that taking the logarithm makes the math nice. You're leaning on that argument to a large extent, although you also cite some other properties which I have no beef with.

Again, I think Ole Peters is pretty cool, and there's interesting stuff in this general direction, even if I find that the particular methodology he's developed doesn't justify anything and instead engages in obfuscated question-begging of existing ideas (like Kelly, and like D Bernoulli's proposed solution to St Petersburg).

Also, I could be misunderstanding something about Peters!

I almost like what this post is trying to do, except that Kelly isn't just about repeated bets. It's about multiplicative returns and independent bets. If the returns from your bets add (rather than multiply), then Kelly isn't optimal. This is the case, for instance, for many high-frequency traders - the opportunities they exploit have limited capacity, so if they had twice as much money, they would not actually be able to bet twice as much.

The logarithm in the "maximize expected log wealth" formulation is a reminder of that. If returns are multiplicative and bets are independent, then the long run return will be the product of individual returns, and the log of long-run return will be the sum of individual log returns. That's a sum of independent random variables, so we apply the central limit theorem, and find that long-run return is roughly e^((number of periods)*(average expected log return)). To maximize that, we maximize expected log return each timestep, which is the Kelly rule.

This is my first post, so I would appreciate any feedback. This started out as a comment on one of the other threads but kept on expanding from there.

I'm also tempted to write "Contra Kelly Criterion" or "Kelly is just the start" where I write a rebuttal to using Kelly. (Rough sketch - Kelly is not enough you need to understand your edge, Kelly is too volatile). Or "Fractional Kelly is the true Kelly" (either a piece about how fractional Kelly accounts for your uncertainty vs market uncertainty OR a piece about how fractional Kelly is about power utilities OR a piece about fractional Kelly is optimal in some risk-return sense)

I've only skimmed this post so far, so, I'll need to read it in greater detail. However, I think this is quite related to Nassim Taleb and Ole Peters' critique of Bayesian decision theory [LW · GW]. I've left fairly extensive comments there, although I'm afraid I changed my mind a couple of times as I read deeper into the material, and there's not a nice summary of my final assessment.

I would currently defend the following statements:

• From a Bayesian standpoint, Kelly is all about logarithmic utility, and the arguments about repeated bets don't make very much sense.
• From a frequentist standpoint, the arguments about repeated bets make way more sense. Ole Peters is developing a true frequentist decision theory: it used to be that Bayesianism was the only game in town if you wanted your philosophy of statistics to also give you decision theory, but no longer!
• I ultimately find Peters' decision theory unsatisfying. It promises to justify different strategies for different situations (EG logarithmic utility for Kelly-type scenarios), but really, pulls the quantity to be maximized out of thin air (IE the creativity of the practitioner). It's slight of hand.

Anyway, I'll read the post more closely, and go further into my critique of Peters if it seems relevant.

There's something odd about the tone of this piece. It begins by saying three times (three times!) "Kelly isn't about logarithmic utility. Kelly is about repeated bets." Then, later, it says "Yes, Kelly is maximising log utility. No, it doesn't matter which way you think about this".

I think you have to choose between "You can think about Kelly in terms of maximizing utility = log(wealth), and that's OK" and "Thinking about it that way is sufficiently not-OK that I wrote an article that says not to do it in the title and then reiterates three times in a row that you shouldn't do it".

I do agree that there are excellent arguments for Kelly-betting that assume very little about your utility function. (E.g., if you are going to make a long series of identical bets at given odds then almost certainly the Kelly bet leaves you better off at the end than any different bet.) And I do agree that there's something a bit odd about focusing on details of the formula rather than the underlying principles. (Though I don't think "odd" is the same as "wrong"; if you ever expect to be making Kelly bets, or non-Kelly bets informed by the Kelly criterion, it's not a bad idea to have a memorable form of the formula in your head, as well as the higher-level concept that maximizing expected log wealth is a good idea.) But none of that seems to explain how cross you seem to be about it at the start of the article.

We have a stock which returns 50% or -40%. If we put all our money in this stock, we are going to lose it over time since $(1+.5)\cdot (1-.4)=1+.1-.2<1$ but on any given day, this stock has a positive expected return.

I like this example as a way of thinking about the problem.

This post makes some good points, too strongly. I will now proceed to make some good points, too rambly.

F=ma is a fantastically useful concept, even if in practice the scenario is always more complicated than just "apply this formula and you win". It's short and intuitive and ports the right ideas about energy and mass and stuff into your brain.

Maximizing expected value is a fantastically useful concept.

Maximizing expected value at a time t after many repeated choices is a fantastically useful concept.

(Edit: see comments, there are false statements incoming. Leaving it all up for posterity.) It turns out that maximizing expected magnitude of value at every choice is very close to the optimal way to maximize your expected value at a time t after many repeated choices. Kelly is a nice, intuitive, easy heuristic for doing so.

So yes, what you really want to do is maximize something like "your total wealth at times t0 and t1 and t2 and t3 and... weighted by how much you care about having wealth at those times, or something" and the way to do that is to implement a function which knows there will be choices in the future and remembers to take into account now, on this choice having the power to maximally exploit those future choices, aka think about repeated bets. But also the simple general way to do something very close to that maximization is just "maximize magnitude of wealth", and the intuitions you get from thinking "maximize magnitude of wealth" are more easily ported than the intuitions you get from thinking "I have a universe of future decisions with some distribution and I will think about all of them and determine the slightly-different-from-maximize-log-wealth way to maximize my finnicky weighted average over future selves' wealth".

Do you need to think about whether you should use your naive estimate of the probability you win this bet, when plugging into Kelly to get an idea of what to do? Absolutely. If you're not sure of the ball's starting location, substituting a point estimate with wide variance into the calculation which eventually feeds into F=ma will do bad things and you should figure out what those things are. But starting from F=ma is still the nice, simple concept which will be super helpful to your brain.

Kelly is about one frictionless sphere approximation to [the frictionless sphere approximation which is maximizing magnitude of wealth]. Bits that were rubbed off to form the spheres involve repeated bets, for sure.

I’m not sure what prompted all of this effort, and I’ve rarely heard Kelly described as corresponding to log utility, only ever as an aside about mean-variance optimization. There, log utility corresponds to A=1, which is also the Kelly portfolio. That is the maximally aggressive portfolio, and most people are much, much more risk averse.

If anything, I’d say that the Kelly - log utility connection obviously suggests one point, which is that most people are far too risk-averse (less normatively, most people don’t have log utility functions). The exception is Buffett - empirically he does, subject to leverage constraints.