# Agents which are EU-maximizing as a group are not EU-maximizing individually

post by Mlxa · 2023-12-04T18:49:08.708Z · LW · GW · 2 comments

## Contents

  Introduction
Specific example
Increasing number of agents
Violation of Independence axiom
Conclusion
None


## Introduction

Why Subagents? [LW · GW] and Why Not Subagents? [LW · GW] explore whether a group of expected utility maximizers is itself a utility maximizer. Here I want to discuss the converse: if a group wants to maximize some utility function as a whole, what can be said about the individual agents? Of course, if they could make decisions together, they will just compute what each agent needs to do, but what if the only thing they have is a common algorithm that each of them uses independently?

It seems that such agents, in general, don't make decisions by multiplying utilons with probabilities and instead they need to consider the whole distribution of outcomes to evaluate a choice. A similar idea was already presented in Against Expected Utility [LW · GW], though without the focus on the number of agents.

## Specific example

Imagine two traders, who select trades independently, but pool their returns together and optimize for the expected logarithm of their total wealth (as in Kelly betting).  Also I will assume for simplicity that they select the same trade for both of them, though the outcomes are still sampled independently.

So if a trade multiplies the wealth by (a random variable) , utility for one trader would be . But for the described group of two traders it becomes , where  are independent random variables with the same distribution as . It is not linear in terms of the outcome probabilities anymore:

## Increasing number of agents

Qualitatively, as the number of agents in the group increases, the agents can afford more risky actions, thanks to the aggregation of returns. So their decision will be somewhere between what an individual agent would do to maximize  and what it would do to maximize . Even more specific example to support this intution: there is a fair coin, and the agent can bet fraction  of the wealth available to them on a certain side, which will turn into  if the coin lands this side and  otherwise, i.e. their wealth will be multiplied by either  or

So as the number of agents increases,  each agent becomes closer to maximizing , but for any finite case there is still some risk-aversion. In particular, any distribution of outcomes that allows the wealth to become zero is still infinitely bad, because if it happens to all agents at the same time, their total wealth will become zero.

## Violation of Independence axiom

As the VNM theorem says that under some assumptions agents can be seen as maximizing expected utility, a natural question is which assumptions don't hold in this case?

I have an example demonstrating that Independence doesn't apply to the agents described above. There will be two lotteries:  which simply preserves the money, and  which multiples them either by  or by  with equal probability. Also consider  and  (i.e. or  with equal probability).

What are the "utilities" here?

Or if you don't trust algebraic manipulations, here is a Python simulation.

Anyway, we see that , but , i.e. a possibility of another outcome reverses the preference.

## Conclusion

I don't know what is the optimal solution to this problem and perhaps it doesn't have a simple form anyway. But I think the problem setup is relevant to EA community, because it is a group of agents who, we might assume, often think in similar way, and it is intractable for them to coordinate what actions each individual should take.

And, at least in some interpretations, Sam Bankman-Fried clearly demonstrated what happens when one starts doing expected utility maximization in a completely risk-neutral way.