The Isolation Assumption of Expected Utility Maximization

post by Pedro Oliboni (pedro-oliboni) · 2020-08-06T04:05:25.727Z · LW · GW · 1 comments

Contents

1 comment

In this short essay I will highlight the importance of what I call the “isolation assumption” in expected utility theory. It may be that this has already been named in the relevant literature and I just don’t know it. I believe this isolation assumption is both important to decision-making about doing good and often ignored.

Expected utility theory is here taken as a normative theory of practical rationality. That is, a theory about what is rational to choose given one’s ends (Thoma 2019, 5). Expected utility theory is then the decision theory that says that the best way for an agent to pursue her goals is to choose so as to maximize expected utility.

By utility, I mean not some concept akin to happiness or wellbeing but a measure that represents how much an agent prefers an outcome. For example, for an altruist, having a child not die from drowning in a pond may have significantly higher utility than dining out at a fancy restaurant.

The term “expected” comes from probability theory. It refers to the sum of the products of the probability and value of each outcome. Expected utility is then a property of options in decisions. Say an agent has two options for lunch and the single thing this agent has preferences over is how her lunch goes today. Option A is to eat a veggie burger, which will bring this agent 10 “utils” for certain. Then, Option A has expected utility of 10. Option B, however, is a raffle in which the agent either gets a really fancy clean-meat burger with probability 0.1 or nothing with probability 0.9. If the agent values the clean-meat burger at 20 utils, and not eating lunch at 0, then the expected utility of Option B has 0.1*20 + 0.9*0 = 2 expected utility.

I currently think expected utility theory is reasonable as a theory of practical rationality. Brian Tomasik has proposed a compelling thought experiment for why that is. Consider the following scenario:

suppose we see a number of kittens stuck in trees, and we decide that saving some number n of kittens is n times as good as saving one kitten. Then, if we are faced with the choice of either saving a single kitten with certainty or having a 50-50 shot at saving three kittens (where, if we fail, we save no kittens), then we ought to try to save the three kittens, because doing so has expected value 1.5 (= 3*0.5 + 0*0.5), rather than the expected value of 1 (= 1*1) associated with saving the single kitten. (Tomasik 2016).

In this case, you may have an instinct that it makes more sense to save the single kitten, since this is the only way to guarantee on life is saved. Yet, Tomasik provides a nice line of reasoning for why you should instead maximize expected utility:

Suppose you're one of the kittens, and you're deciding whether you want your potential rescuer to save one of the three or take a 50-50 shot at saving all three. In the former case, the probability is 1/3 that you'll be saved. In the latter case, the probability is 1 that you'll be saved if the rescuer is successful and 0 if not. Since each of these is equally likely, your overall probability of being saved is (1/2)*1 + (1/2)*0 = 1/2, which is bigger than 1/3. (Tomasik 2016)

So, I’ve attempted to make the case for why expected utility theory makes sense. Now I will get to my point that we should be careful not to misuse it. I will thus try to make the case for the importance of what I call the “isolation assumption” and for how easy it is for it to be dangerously ignored.

First, let’s get a bit deeper in the point of expected utility theory. As I said above, this is a theory about how to best go about achieving one’s ends. Let’s suppose our ends are mostly about “making the most good”. Then, especially if we are aspiring Effective Altruists [? · GW], we ideally want to maximize the expected utility of all of the relevant consequences of our actions. I say this in contrast to merely maximizing the expected utility of the immediate consequences of our actions. Notice, however, that scenarios that are commonly discussed when talking about decision theory, such as the one involving kittens above, are focused on the immediate consequences. What is important, then, is that we don’t forget that consequences which are not immediate often matter, and sometimes matter significantly more than the immediate consequences.

This then gives rise to the assertion I want to make clear: that we can only apply expected utility theory when we are justified in assuming that the values associated with the outcomes in our decision problem encompass all of the difference in value in our choice problem. Another way of putting this is to say that the future (beyond the outcomes we are currently considering) is isolated from the outcomes we are currently considering. Yet another way to put this is that the choice we currently face affects nothing but the prospects that we are taking into account.

Notice how this point is even more important if we are longtermists [EA · GW]. Longtermism entails that consequences matter regardless of when they happen. This means we care about consequences extending as far into the future as the end of history. Then, if we are to choose by maximizing expected utility, we must be able to assume that whatever outcomes we are considering, choosing one way or another does not negatively affect what options are available in the rest of history.

Here is an example to illustrate my point, this time adapted and slightly modified from a thought experiment provided by Tomasik:

Suppose (if necessary) that you are an altruist. Now assume (because it will make it easier to make my point) that you are a longtermist. Suppose you are the kind of longtermist that thinks it is good if people who will lead great lives are added to the world, and bad if such great lives are prevented from existing. Suppose that there are 10,000 inhabitants in an island. This is a special island with a unique culture in which everyone is absurdly happy and productive. We can expect that if this culture continues into the future, many of the most important scientific discoveries of humanity will be made by them. However, all of the inhabitants recently caught a deadly disease. You, the decision maker, has two options. Drug A either saves all of the islanders with 50% chance or saves none of them with 50% chance. Drug B saves 4,999 of them with complete certainty.

If we consider only the survival of the inhabitants, the expected utility of Drug A is higher (10,000*0.5 + 0*0.5 = 5,000 > 4,999). However, saving these inhabitants right now is not the only thing you care about. As a longtermist, you care about all potential islanders that could exist in the future. This implies the appropriate expected utility calculation includes more than what we have just considered.

Suppose that combining the value of the scientific discoveries these islanders would make and the wellbeing their future descendants would experience if this civilization continues is worth 1,000,000 utils. Then, the complete expected utility of Drug B is the value of each life saved directly (4,999) plus the 1,000,000 of the continuation of this civilization (total = 1,004,999). The expected utility of Drug A, however, is merely (10,000 + 1,000,000)*0.5 + 0*0.5 = 505,000. So, Drug B is now the one with highest expected value. I hope this makes clear how if you can’t assume that the choices in the present do not affect the long-term consequences, you cannot use expected value!

As I understand it, the upshot is the following. If you make a decision based on maximizing expected utility, you have two possibilities. You can incorporate all the relevant consequences (which may extend until the end of human civilization). Or you have to be able to assume that the value of the consequences that you are not considering do not change which of your current options is best. However, it seems to me now that the only way you can assume a set of consequences of your choice won’t affect which option is better is if you know this set is isolated from the current choice. Otherwise you would have incorporated these consequences in the decision problem.

1 comments

Comments sorted by top scores.