Appraising aggregativism and utilitarianism

post by Cleo Nardo (strawberry calm) · 2024-06-21T23:10:37.014Z · LW · GW · 10 comments

Contents

  1. Introduction
  2. Advantages of aggregativism
    2.1 Avoids excessive permissions
    2.2. Avoids excessive obligations.
    2.3. Computationally tractable
    2.4. Retains utilitarian spirit
    2.4. Lower description complexity
    2.6. Avoids counterintuitive implications
      2.5.1. Repugnant conclusion
      2.5.2. Extreme suffering
    2.7. More concrete and relatable
  3. Objections to aggregativism
    3.1. Inherits human irrationality
    3.2. Requires model of human behaviour
  3. Conclusion
None
10 comments

“My problem is: What are those objects we are adding up? I have no objection to adding them up if there's something to add.” — Kenneth Arrow

1. Introduction

Aggregative principles state that a social planner should make decisions as if they will face the aggregated personal outcomes of every individual in the population. Different modes of aggregation generate different aggregative principles. In general, a mode aggregation provides some method for reducing a collection of personal outcomes to a single personal outcome.

There are three notable aggregative principles:

• Live Every Life Once, stating that a social planner should make decisions as if they face the concatenation of every individual's life in sequence.^[1]
• Harsanyi's Lottery, stating that a social planner should make decisions as if they face a uniform lottery over the individuals in population.^[2]
• Rawls' Original Position, stating that a social planner should make decisions as if they face ignorance about which individual's life they will live, with no basis for assigning probabilities.^[3]

This article follows on from my previous articles on this topic:

• Aggregative principles of social justice [LW · GW]
• Aggregative principles approximate utilitarian principles [LW · GW]

In this article I compare aggregativism with an alternative strategy for specifying principles of social justice: namely, utilitarianism. Aggregativism avoids many theoretical pitfalls that plague utilitarianism, but faces its own objections. Ultimately, I conclude that aggregativism's advantages outweigh the objections; it is superior to utilitarianism as a strategy for specifying principles of social justice, though the objections are serious and worth addressing in future work.

The rest of the article is organized as follows. Section 2 discusses seven advantages of aggregativism over utilitarianism, in descending importance:

1. Avoids excessive permissions. If an option strongly defies human nature, then aggregativism will never permit choosing it, unlike utilitarianism.
2. Avoids excessive obligations. If an option strongly conforms to human nature, then aggregativism will never forbid choosing it, unlike utilitarianism.
3. Computationally tractable. Aggregativism can be implemented with realistic computational resources, unlike utilitarianism.
4. Retains utilitarian spirit. Under reasonable conditions, aggregativism approximates utilitarianism.
5. Lower description complexity. The aggregative approach sidesteps the complexity of defining a social utility function.
6. Avoids counterintuitive implications. Aggregativism resolves Parfit's Repugnant Conclusion, the dismissal of extreme suffering, and similar paradoxes that beset utilitarianism.
7. More concrete and relatable. Aggregativism grounds ethics in personal experiences, which are easier to reason about than abstract numbers. 

Section 3 discusses two objections to aggregativism, in descending severity:

1. Inherits human irrationality. Aggregativism inherits the irrationalities of human decision-making.
2. Requires model of human behaviour. Aggregativism depends an accurate model of human choice across all contexts, both mundane and exotic.

2. Advantages of aggregativism

2.1 Avoids excessive permissions

Aggregativism is more robust than utilitarianism to 'going off the rails'. The crucial difference is that, under aggregativism, the ultimate arbiter of social choice is human behaviour, providing a safeguard against bizarre or inhuman choices. Utilitarianism, by contrast, outsources social choices to an abstract optimization process, ${\text{argmax}}_X$, which is susceptible to extreme or bizarre results.

To see why aggregativism is more robust than utilitarianism, let's quickly review the formal framework introduced in the previous article [? · GW]. Feel free to skim this if you're already familiar:

• Let $P$ be the space of personal outcomes, a full description of the state-of-affairs for a single individual; and let $S$ be the space of social outcomes, a full description of the state-of-affairs for society as a whole. If $I$ denotes the set of individuals in society, then there is a function $\gamma :I\times S\to P$ such that, if the social outcome $s\in S$ obtains, then each individual $i\in I$ faces the personal outcome $\gamma (i,s)\in P$.
• Suppose a social planner may choose from a set of options $X=\{x_1,\dots ,x_n\}$, and the social consequences of each option are described by a function $f:X\to S$ assigning a social outcome $s\in S$ to each option $x\in X$. For instance, $X$ might be a set of possible tax rates, and $S$ the set of possible societal wealth distributions. We call $f$ the social context.
• A social choice principle is any function $\Psi :(X\to S)\to P\left(X\right)$ which takes a social context $f:X\to S$ as input and returns the set of 'permissible' options $\Psi \left(f\right)\subseteq X$ as output. For example, the principle $\Psi \left(f\right):=\{x\in X\mid \#[x{]}_f>\frac{\#X}{2}\}$
  will permit choosing an option if and only if most other options would've led to the same social outcome. The task of social ethics is specifying a normatively compelling social choice principle.
• A utilitarian principle has the form $\Psi \left(f\right):={\text{argmax}}_X(u\circ f)$ where $u:S\to R$ is a social utility function mapping social outcomes to real values, and ${\text{argmax}}_X:(X\to R)\to P\left(X\right)$ returns the maximisers of any given real-valued function over $X$. For instance, if $u\left(s\right)\in R$ is the gross world product then the utilitarian principle $\Psi \left(f\right):={\text{argmax}}_X(u\circ f)$ endorsing maximising gross world product. This strategy for specifying social choice principles is called utilitarianism.
• An aggregative principle has the form $\Psi \left(f\right)=\Pi (\zeta \circ f)$ where $\zeta :S\to P$ is a function mapping social outcomes to personal outcomes, and $\Pi :(X\to P)\to P\left(X\right)$ models the behavior of a self-interested human, identifying the options $\Pi \left(g\right)\subseteq X$ they might choose in a personal context $g:X\to P$. For instance, if $\zeta \left(s\right)\in P$ is the personal outcome of living every individual's life in sequence, then the aggregative principle $\Psi \left(f\right):=\Pi (\zeta \circ f)$ is MacAskill's LELO. We call a function $\zeta :S\to P$ a social zeta function. This strategy for specifying social choice principles is called aggregativism.

Using this formal framework, we can prove that aggregative principles satisfy a built-in consistency property. This property, which we call 'always-forbidden consistency' can be stated as follows: if a human will never choose a particular option, regardless of the personal context, then the social planner is forbidden to choose that option, regardless of the personal context. Formally: if $x\notin \Pi \left(g\right)$ for all personal contexts $g:X\to P$, then $x\notin \Psi \left(f\right)$ for all social contexts $f:X\to S$.

Clearly, any aggregative principle $\Psi \left(f\right):=\Pi (\zeta \circ f)$ satisfies this property. For example, suppose a human would never choose to torture an innocent person, no matter the personal consequences. Then a social planner following any aggregative principle is always forbidden to torture an innocent person, no matter the social consequences. Aggregativism is therefore constrained by the limits of human decision-making.

Why might an option $x\in X$ be entirely ruled out by $\Pi$? There are three main reasons, all covered by this consistency property.

1. Violates moral constraints. The option may transgress inviolable boundaries, like prohibitions on murder, torture, or slavery. If human behaviour, as modelled by $\Pi$, precludes such options, then the aggregative principle will forbid those options.
2. Logical, metaphysical, or nomological impossibility. The option may be logically incoherent or incompatible with the laws of nature. For example, $x$ might involve traveling faster than light, or being in two places at once, or making $2+2=5$. Our model of the option space $X$ might have been too inclusive, including some pseudo-options that weren't actually possible.
3. Unthinkability. The option may be so alien to a person's character that they would never ever entertain it. It's not something they would recognise as an eligible option, even though it doesn't violate the laws of nature or transgress ethical boundaries.^[4]

The aggregative principles forbid all such options. Crucially, this consistency property holds regardless of how the social zeta function $\zeta :S\to P$ is defined. Even if $\zeta$ is poorly specified, or fails to represent a normatively compelling notion of impartial aggregation, the social planner would nonetheless never choose an option that a human would never choose, if they followed the aggregative principle generated by $\zeta$.

Utilitarianism violates 'always-forbidden consistency'. Utilitarianism can permit literally any option, no matter how bizarre, in some social context. This hold for any non-constant social utility function $u:S\to R$; there will exists some pair of outcomes $s^{-},s^{+}\in S$ with $u\left(s^{-}\right)<u\left(s^{+}\right)$. We can therefore construct a social context $f:X\to S$ where the options permitted by the utilitarian principle are precisely those we want the utilitarian principle to permit.

To prove this, let $\varnothing ⊊A\subseteq X$ be any nonempty subset of $X$. Define the social context $f:X\to S$ as follows:

$f\left(x\right)=\{\begin{array}{cc}{s}^{+}& \text{if}x\in A\\ s^{-}& \text{if}x\notin A\end{array}$

Then for the utilitarian principle $\Psi \left(f\right):={\text{argmax}}_X(u\circ f)$, we have $x\in \Psi \left(f\right)$ if and only if $x\in A$. Therefore, the utilitarian planner maximizing $u$ would be permitted to choose precisely the options in $A$, even if $A$ contains options which violate moral constraints,  are logically, metaphysically, or nomological impossible, or are otherwise unthinkable.^[5]

Under utilitarianism, everything is potentially permitted. The root problem is that utilitarianism delegates social choice to ${\text{argmax}}_X$, an abstract optimization process deviating wildly from realistic human behaviour. Without this anchor of human decision-making, utilitarianism can go arbitrarily awry.

2.2. Avoids excessive obligations.

In addition, aggregativism satisfies 'always-permissible consistency' which can be stated as follows: if a human might always choose a particular option, regardless of the personal context, then the social planner is permitted to choose that option, regardless of the social context. Formally: if $x\in \Pi \left(g\right)$ for all personal contexts $g:X\to P$, then $x\in \Psi \left(f\right)$ for all social contexts $f:X\to S$. This is the dual of the 'always-forbidden consistency' property discussed above.

Clearly, an aggregative principle $\Psi \left(f\right):=\Pi (\zeta \circ f)$ satisfies this property. For example, suppose there's some 'do nothing' option $x_{\text{noop}}\in X$ such that, regardless of the personal consequences, it's always possible that the human would choose this option. Then the social planner following any aggregative principle is always permitted to 'do nothing', no matter the social consequences.

In contrast, utilitarian principles violate always-permissible consistency. Utilitarianism can forbid any option, no matter how inoffensive, in some social contexts. To prove this, we again assume that $u:S\to R$ is a non-constant social utility function, with $u\left(s^{-}\right)<u\left(s^{+}\right)$. If $\varnothing \subseteq B⊊X$ is any proper subset of $X$, then we set $A:=X\setminus B$ and define the social context $f:X\to S$ as we did before:

$f\left(x\right)=\{\begin{array}{cc}{s}^{+}& \text{if}x\in A\\ s^{-}& \text{if}x\notin A\end{array}$

Then for the utilitarian principle $\Psi \left(f\right):={\text{argmax}}_X(u\circ f)$, we have $x\in \Psi \left(f\right)$ if and only if $x\notin B$. Now, for any option $x\in X$, there will exist some proper subset $\varnothing \subseteq B⊊X$ containing $x$, provided that $|X|\ge 2$. The utilitarian planner maximizing $u$ would be forbidden to choose $x$ in this social context, even if $x$ is inoffensive.

Under utilitarianism, everything is potentially forbidden. This situation is the basis of the demandingness objection to utilitarianism: any option, such as saving a child from a burning building, donating one's income to AMF, or dedicating one's to esoteric research, is potentially obligated by the utilitarian principle.

2.3. Computationally tractable

Consider the following scenario:

A social planner must choose a 1000-bit proof, so their option space is $X=\{0,1{\}}^{1000}$. Let $\Theta$ denote all 1000-bit mathematical statements, in a formal language such as Peano Arithmetic. If $\theta \in \Theta$ then the social context $f_{\theta }:X\to S$ is defined as follows:

• If $x=0$, then $f_{\theta }\left(x\right)=s_{\text{okay}}$, meaning everyone gains £100.
• If $x\ne 0$ and $x$ encodes a valid proof of statement $\theta$, then $f_{\theta }\left(x\right)=s_{\text{great}}$, meaning everyone gains £200.
• If $x\ne 0$ and $x$ doesn't encode a valid proof of statement $\theta$, then $f_{\theta }\left(x\right)=s_{\text{bad}}$, meaning everyone gains £0.

For the sake of concreteness, take $\theta$ to be Goldbach's conjecture. The key property of this class of social contexts is that finding a proof of statement $\theta \in \Theta$ is computationally intractable, but verifying a claimed proof is easy.^[6]

Now, any reasonable social utility function $u:S\to R$ would rank the social outcomes as $u\left(s_{\text{great}}\right)>u\left(s_{\text{okay}}\right)>u\left(s_{\text{bad}}\right)$. Let's examine what is demanded of a social planner by the utilitarian principle $f\mapsto {\text{argmax}}_X(u\circ f)$. When the social planner faces the social context $f_{\theta }:X\to S$ then, if $\theta \in \Theta$ is a mathematical statement with a 1000-bit proof, then the social planner is obligated to provide it. Choosing 0 would be impermissible, as would choosing any nonzero string that doesn't encode a valid proof of $\theta$. The catch is that the social planner has bounded computational resources, so they cannot actually implement this principle.^[7] The utilitarian principle is not action-guiding for agents with realistic computational limits.

By contrast, the aggregative principle $f\mapsto \Pi (\zeta \circ f)$ would regularly endorse choosing zero when facing the context $f_{\theta }:X\to S$. This is because a self-interested human has little chance of proving a mathematical statement such as Goldbach's conjecture. Hence, if a self-interested human anticipated facing the aggregated personal outcomes of each individual, then they would play it safe by choosing zero. Unlike utilitarianism, aggregativism provides clear guidance which doesn't exceed the social planner's computational resources.

The root problem is that the ${\text{argmax}}_X$ operator, which underlies utilitarianism, is computationally unbounded. Formally, there exist easy functions $F:\Theta \times X\to R$ such that there exist no similarly easy functions $g:\Theta \to X$ satisfying $g\left(\theta \right)\in {\text{argmax}}_X\left(F\right(\theta ,-\left)\right)$ for all $\theta \in \Theta$. Hence, if $X$ denotes the social planner's options, and $\Theta$ parameterizes a class of social contexts, then the ${\text{argmax}}_X$ operator can search over the entire option-space for a maximiser of $F(\theta ,-):X\to R$, although no physical implementation could find a maximiser efficiently.

One may object that the utilitarian principle is not meant to be a computationally tractable decision procedure, but merely to define what counts as right or to ground normative judgments more abstractly.^[8] And that's a fair point. However, the fact that utilitarian principles fail to guide action in cases like this, while aggregative principles succeed, does count in favour of aggregativism.^[9]

2.4. Retains utilitarian spirit

With a well-chosen social zeta function $\zeta :S\to P$, aggregativism retains the most appealing features of utilitarianism. Like utilitarianism, aggregativism is sensitive to outcomes: whether an option is permitted typically depends on its social consequences, as captured by the function $f:X\to S$. Of course, aggregativism may consider nonconsequentialist factors also (see Section 2.1). Nonetheless, it preserves the consequentialist spirit of utilitarianism; directing the social planner to choose options that lead to desirable personal outcomes for individuals, considered impartially.

Consider the famous trolley problem: a runaway trolley will kill five people unless diverted to a side track, where it will kill only one person. The naïve utilitarian verdict is clear: you must divert the trolley. Doing so leads to one death rather than five, and the former social outcome has higher social utility. An aggregative principle would likely agree, assuming the social zeta function $\zeta$ is well-chosen. For example, suppose $\zeta$ maps each social outcome to a lottery over the fates of the affected individuals. In that case, most people would prefer the lottery corresponding to diverting the trolley over the lottery corresponding to not diverting. The odds of death are lower in the former lottery.

So in 'ordinary' cases, aggregativism and utilitarianism tend to agree. The two approaches diverge only in more extreme scenarios. Suppose the only way to save the five people is to push a large man in front of the trolley, killing him but stopping the trolley due to his mass. Utilitarianism still endorses sacrificing the one to save the five, as this leads to fewer total deaths. But aggregativism likely rejects pushing the man. Most humans would refrain from murder, even if doing so improves their odds of survival. Importantly, the aggregative framework need not explicitly specify the social planner's moral constraints; it simply imports them from the human behavior model.

Indeed, the previous article established a pertinent theorem [? · GW]: under reasonable conditions, aggregativism and utilitarianism are mathematically equivalent, in the sense that the aggregative principle $f\mapsto \Pi (\zeta \circ f)$ and utilitarian principle $f\mapsto {\text{argmax}}_X(u\circ f)$ will always permit the same subset of options in all contexts $f:X\to S$.

While these conditions don't hold exactly, they are close approximations, so aggregativism and utilitarianism will yield similar prescriptions in in practice. In particular, the Live Every Life Once (LELO) will approximate longtermist total utilitarianism. Similarly, Harsanyi's Lottery (HL) will approximate average utilitarianism. Finally, Rawls' Original Position (ROI) will approximate his difference principle.

2.4. Lower description complexity

The most glaring problem with utilitarianism is the immense description complexity of any suitable social utility function $u:S\to R$. That is, fully defining the function $u$ would require an impractically vast amount of information, if we limit ourselves to a basic, physicalist language without high-level concepts. By 'description complexity', we mean the quantity of information needed to uniquely identify an object within a given language.

We can establish a lower bound on the complexity of $u$ as follows: First, an adequate social choice principle $\Psi$ must encompass all the ethical considerations and tradeoffs in real-world social decision-making. This includes the full complexity of axiology, i.e. determining which outcomes are valuable and to what degree. It also includes the complexity of decision theory, such as accounting for long-term consequences, dealing with uncertainty, and considering acausal effects. For utilitarianism to yield a suitable principle, all this complexity must be encoded in the utility function $u$. This is because a utilitarian principle $\Psi$ is entirely determined by $u$ and the ${\text{argmax}}_X$ operator. But ${\text{argmax}}_X$ is a simple mathematical object, contributing no complexity, so almost all the complexity of $\Psi$ must come from $u$ itself.

The unavoidable complexity of any suitable social utility function has been a major criticism of utilitarianism.^[10] Moreover, the unavoidable complexity of $u$ makes it infeasible to design a superintelligent AI system by specifying an appropriate utility function.^[11]

In contrast, aggregativism sidesteps this prohibitive complexity. Much of the intricacy of social choice may already be captured by the intricacy of self-interested human behaviour, which is 'imported' into the social choice principle via the social zeta function. Recall that an aggregative principle $\Psi$ is fully determined by two components: the social zeta function $\zeta :S\to P$ and the model of human behaviour $\Pi :(X\to P)\to P\left(X\right)$. While an adequate $\Psi$ must be highly complex, this complexity is split between $\zeta$ and $\Pi$. Crucially, $\Pi$ models human decision-making across all personal contexts, so it necessarily encodes a huge amount of information about human preferences, values, and reasoning. Given the complexity of $\Pi$, even a fairly simple $\zeta$ can yield a suitable $\Psi$.

2.6. Avoids counterintuitive implications

2.5.1. Repugnant conclusion

The repugnant conclusion, first formulated by Derek Parfit, poses a serious challenge to total utilitarianism and population ethics. Namely, total utilitarianism suggests that a world filled with an enormous number of people leading lives barely worth living is better than one with a much smaller population of very happy individuals.

Let's consider how the three utilitarian principles resolve this:

1. Longtermist total utilitarianism, as defended William MacAskill, seeks to maximize the total sum of personal utility across all individuals — past, present, and future. However, this leads to the repugnant conclusion.
2. Average utilitarianism, as proposed by John Harsanyi, seeks to maximize the average personal utility across all individuals. This avoids the repugnant conclusion, but results in its own counterintuitive implication: a social planner should refrain from adding additional lives which are worth living, if they are below the average personal utility.
3. The difference principle, defended by John Rawls, seeks to maximize the minimal personal utility across all individuals. This avoids the repugnant conclusion, but results in the most bizarre implication: a social planner has no incentive for adding any lives, as this can only reduce the minimum personal utility.

The aggregative principle LELO handles the repugnant conclusion more successfully than any utilitarian principle discussed. Under LELO, a society $s$ with individuals facing personal outcomes $p_1,\dots ,p_n$ is concatenated to a single personal outcome, i.e. $\zeta \left(s\right):=p_1▹\cdots ▹p_n$. This reframes the population ethics dilemma in terms of a personal choice between quality and duration of life.

Formally, when comparing a population of $n$ individuals with personal utilities $p_1,\dots ,p_n$ to an alternative population of $m$ individuals with utilities $q_1,\dots ,q_m$, LELO ranks the first population as better if and only if a self-interested human would prefer to live the combined lifespan $p_1▹\cdots ▹p_n$ over $q_1▹\cdots ▹q_m$. Do people generally prefer a longer life with moderate quality, or a shorter but sublimely happy existence? Most people's preferences likely lie somewhere in between the extremes. This is is because personal utility of a concatenation of personal outcomes is not precisely the sum of the personal utilities of the outcomes being concatenated.

For example, exponential time-discounting is a common assumption in economics, which states that the personal utility function $v:P\to R$ obeys the equation $v(p_1▹p_2)=v\left(p_1\right)+(1+\delta {)}^{-\text{duration}\left(p_1\right)}\cdot v(p_2)$. Here $\text{duration}:P\to R^{\ge 0}$ gives the duration of each outcome and $\delta >0$ is the discount rate. This discounting formula weights the first outcome $p_1$ more than the second outcome $p_2$, with the difference growing exponentially with the duration of $p_1$. If human's maximise a personal utility function with this property, then the value gained by adding an additional life will exponentially decay in the total sum of existing lifespans.

Hence, LELO endorses a compromise between total and average utilitarianism, better reflecting our normative intuitions. While not decisive, it is a mark in favour of aggregative principles as a basis for population ethics.

2.5.2. Extreme suffering

Intuitively, some personal outcomes are so horrific that no miniscule benefit to others, no matter how many beneficiaries there are, could ever justify them. The following example illustrates this: Imagine a social outcome $s$ where $n$ people live comfortable lives. Now compare that to an alternative outcome $s^{\prime }$ where where $n-1$ people receive a miniscule additional benefit, but one unfortunate person faces a life of unrelenting agony, subjected to the most horrific physical and psychological torture imaginable.

For many social utility functions $u:S\to R$, there exists a sufficiently large $n$ such that $u\left(s\right)<u\left(s^{\prime }\right)$. In other words, the utilitarian principle would not only permit the social planner to switch from $s$ to $s^{\prime }$, but in fact obligate it. This holds for both total and average utilitarianism; the social planner would be morally obligated to torture an innocent person for the sake of a trivial benefit to others. This conclusion seems to defy moral common sense; surely there are some personal outcomes which are so awful that no number of miniscule beneficiaries could justify it.^[12] Only Rawls' difference principle avoids it, since it equates social utility with the minimum personal utility.

How do aggregative principles handle extreme suffering to a small minority?

According to Harsanyi's Lottery (HL), the social planner should choose $s^{\prime }$ over $s$ only if a human would prefer a uniform lottery over the individuals in $s^{\prime }$ to a uniform lottery over the individuals in $s$. That is, only if a self-interested human would accept a $\frac{1}{n}\approx 0$ likelihood of torture in exchange for the $(1-\frac{1}{n})\approx 1$ chance of the miniscule benefit. But would they choose this trade? Perhaps not. Humans do exhibit an extreme aversion to even small risks of catastrophic outcomes.

Formally, let $p_{\text{good}}$, $p_{\text{improved}}$, and $p_{\text{torture}}$ denote a good life, a slightly improved life, and a horrifically torturous life respectively. Let $p_{\text{torture}}{+}_{\lambda }{p}_{\text{improved}}$ denote the lottery that yields $p_{\text{torture}}$ with probability $\lambda$ and $p_{\text{improved}}$ with probability $1-\lambda$. Plausibly, a self-interested human would choose $p_{\text{good}}$ over $p_{\text{torture}}{+}_{\lambda }{p}_{\text{improved}}$, because they place substantial value on the 'zero-likelihood-ness' of torture.

Note that this behaviour is inconsistent with the following assumptions of human behaviour.

• Humans maximise a personal utility function $v:P\to R$.
• The outcomes are ranked as $v\left(p_{\text{improved}}\right)>v\left(p_{\text{good}}\right)>v\left(p_{\text{torture}}\right)$.
• Personal utility $v:P\to R$ is continuous in the underlying likelihoods, in the sense that $\underset{\lambda \to 0}{lim}v\left(p_{\text{torture}}{+}_{\lambda }{p}_{\text{improved}}\right)$ equals $v\left(p_{\text{improved}}\right)$ and $\underset{\lambda \to 1}{lim}v\left(p_{\text{torture}}{+}_{\lambda }{p}_{\text{improved}}\right)$ equals $v\left(p_{\text{torture}}\right)$.

To be clear, whether humans actually exhibit this discontinuous preference is an empirical question. But it seems plausible given the limits of human reasoning. Humans don't represent probabilities with infinite precision, so there may be some small probability $ϵ$ such that they treat any nonzero probability less than $ϵ$ in the same way they treat $ϵ$. If there are personal outcomes with this property, then Harsanyi's Lottery would endorse choosing $s$ over $s^{\prime }$, conforming to my moral intuitions.

Compared to HL, I think that LELO is even less tolerant of the extreme suffering of a small minority. Facing the uniform lottery proposed by HL, a human may find it easy to dismiss the small likelihood of extreme suffering, and hence a social planner following the principle would dismiss the extreme suffering of the small minority. But under LELO, the social planner must imagine facing the concatenation of the individual lives, rather than a uniform lottery. Hence, $s$ is associated with the concatenation of $n+1$ comfortable lives, and $s^{\prime }$ with the concatenation of a life of horrific torture followed by $n$ slightly improved lives. But it's impossible for a human to ignore the period of torture, no matter how many comfortable lives succeed it, since it is experienced with certainty.^[13]

There are other puzzling cases where the paradigm aggregative principles conform more to our moral intuitions than the paradigm utilitarian principles, such as distant future generations, the terminal value of tradition, and infinite ethics.

2.7. More concrete and relatable

Aggregativism is more concrete than utilitarianism. Aggregative principles like LELO, HL, and ROI promote outcomes that are graspable by the social planner. The outcomes promoted by utilitarian principles, in contrast, are far more opaque.

Recall that each aggregative principle is specified by a social zeta function $\zeta :S\to P$, which maps social outcomes to personal outcomes. Assuming the social planner is human, they will be familiar with these personal outcomes. They will have a rich set of prior attitudes towards them, including intuitions, preferences, experiences, historical analogues, legal precedents, and moral convictions.

The social zeta function extends these prior attitudes to social outcomes. For instance, consider a social outcome $s$ with a billion healthy people and a billion sick people. The planner may lack prior attitude towards this social outcome, because the population contains strangers the planner has no interest in. But if $\zeta$ maps $s$ to the personal outcome of an even lottery between health and sickness, the planner will have strong prior attitudes about it, helping them reason about the social outcome via analogy to something concrete and familiar.

By contrast, each utilitarian principle is specified by a social utility function $u:S\to R$ mapping social outcomes to real numbers. But the planner has no prior attitudes towards these abstract numbers, so $u$ does nothing to help the social planner understand the situation. They may know that $540>450$, but how does this mathematical fact help them reason: is an outcome with utility 540 slightly better than one with 450, or vastly better? Does the difference in social utility warrant violating property rights? Does the difference warrant violating body autonomy? It's not clear.

Indeed, a common objection of utilitarianism is that it has an austere and impersonal quality, seeking to maximize an abstract metric ('utility') rather than anything recognizably valuable to humans. Aggregativism replaces these abstract numbers with personal outcomes, avoiding this objection.

There are limits to the concreteness of personal outcomes. Some social outcomes may map to 'exotic' personal outcomes that are unfamiliar to humans; for instance, in LELO, $\zeta \left(s\right)$ is the concatenation of billions of individual lives. Nonetheless, even these exotic personal outcomes remain more tangible than abstract utility numbers such as "3.27e10". Humans can imagine the prospect, reason about what it would be like, feel excited or horrified by it, judge whether it's more desirable than some other concatenation of lives, and so on.

The concreteness of aggregativism explains its appeal to moral philosophers, such as MacAskill, Harsanyi, and Rawls, who have used aggregative thought experiments to motivate their principles of social choice.

3. Objections to aggregativism

3.1. Inherits human irrationality

Aggregativism inherits the inconsistencies, irrationalities, and biases of human decision-making, since it is based on $\Pi$, the model of human behaviour.

For a concrete example, consider the case of intransitive preferences. If humans exhibit intransitive preferences, then a social planner following the aggregative principle may also exhibit intransitive preferences. This renders them vulnerable to 'ethical money pumps', a cycle of trades that exploit the social planner's intransitivity. By contrast, utilitarianism is based on the ${\text{argmax}}_X$ operator, which never exhibits intransitive preferences. Hence, a social planner following the utilitarian principle will not be exploitable in this way.^[14] 

Similar arguments apply to other forms of irrationality that $\Pi$ might exhibit, such as: incomplete preferences, dynamic inconsistency, framing effects, menu effects, and so on. Aggregativism inherits these defects; utilitarianism doesn't.

That said, I suspect aggregativism would still perform adequately in practice. Human irrationality, while real, is not so severe or easily exploited as to undermine their overall competence. Most of the time, they muddle through okay. By extension, a social planner following the aggregative principle should be roughly as resistant to exploitation as a typical human, avoiding the most egregious errors.

Furthermore, it may be possible to mitigate the biases and inconsistencies in the model of human behavior used by aggregativism. The aggregative framework does not demand that $\Pi$ perfectly model a realistic human, only that it is some function of type $(X\to P)\to P\left(X\right)$. That is, $\Pi$ need not model realistic human behaviour, it may instead model idealised human behaviour. This idealisation may correct for certain biases, expand available information, remove inconsistencies, or otherwise improve on ordinary human decision-making, while still retaining human values and cognition.

However, we must be cautious not to idealize $\Pi$ too far: the appeal of aggregativism lies in its proximity to actual human decision-making. If $\Pi$ diverges too radically from human behaviour then aggregativism is more likely to 'go off the rails' like utilitarianism, because the underlying choice criterion approaches the alienness of argmax. The further we stray from human psychology, the less obvious the moral authority of the resulting entity. Moreover, as Joe Carlsmith argues in "On the limits of idealized values [LW · GW]", there are myriad ways to idealise human behaviour. The choice a particular method feels arbitrary and subject to scrutiny itself — we can always wonder whether we'd actually endorse the choices of the idealised agent. Therefore, I endorse employing a $\Pi$ which models realistic human behaviour.

3.2. Requires model of human behaviour

In order for the social planner to implement an aggregative principle, they need an accurate model of how a self-interested human would behave in different personal contexts. This model is captured by the operator $\Pi :(X\to P)\to P\left(X\right)$ which, as discussed in Section 2.4, is an immensely complicated object. The social planner's uncertainty about self-interested human behaviour will hinder applying the principle in practice.

By contrast, utilitarianism relies on the simple, well-defined ${\text{argmax}}_X$ operator. (As discussed before, the complexity of utilitarianism lies in the social utility function $u:S\to R$.) The social planner has no analogous uncertainty about the nature of ${\text{argmax}}_X$.

This is a serious objection to aggregativism, but I do think that grounding a normative theory on $\Pi$, the model of human behaviour, has several advantages over grounding it on $u$, the social utility function.

Firstly, $\Pi$ is a possibilistic model of human behavior, meaning it specifies which options a human might choose in different contexts, without assigning probabilities. As such, the object $\Pi$ supervenes on natural facts: it is simply a question of which options the human might choose in given personal contexts. In contrast, defining a utility function $u$ requires making value judgments compare different social outcomes.

Secondly, we can draw on a rich body of existing empirical data and methods to inform the specification of $\Pi$. We can leverage behavioral experiments, economical models, neuroimaging studies, and so on. We could train a deep neural network on a large dataset of human choices to predict which options humans would choose in novel contexts. Specifying $\Pi$ through a combination of empirical study and machine learning seems more scientifically grounded and tractable than defining $u$ from first principles.

Thirdly, the social planner could consult their own intuitions about what they would do in the hypothetical personal context. For example, if the social planner feels they would choose option $x_1$ over option $x_2$ in the personal context $\zeta \circ f:X\to P$, this provides a reason for them to choose option $x_1$ over option $x_2$ in the social context $f:X\to S$. Of course, people's self-predictions are often inaccurate,^[15] but the social planner's intuitions provide a useful starting point that can be refined with empirical data.

3. Conclusion

The central feature of aggregativism, for better and worse, is that it conforms to the contours of human decision-making in ways that utilitarianism does not.

On the plus side, it never permits an option that the human would never choose, thereby avoiding utilitarianism's excessive permissions. It also never forbids an option that the human might always choose, thereby avoiding utilitarianism's excessive obligations. Moreover, it doesn't require utilitarianism's superhuman computational resources. And it sidesteps the prohibitive complexity of defining a social utility function, by importing the complexity of the human behavior model. Aggregativism also enables more concrete moral reasoning, by dealing in personal experiences rather than abstract utility.

On the other hand, by hewing so closely to human behavior, aggregativism inherits the messy irrationalities and inconsistencies of human decision-making. And it requires a model of how humans act in a vast array of hypothetical scenarios.

1. ^{^}

   The term LELO originates in Loren Fryxell (2024), "XU", which is where I first encountered the concept. I think Fryxell offers the first formal treatment of the LELO principle. MacAskill (2022), "What We Owe the Future", says this thought experiment comes from Georgia Ray (2018), “The Funnel of Human Experience”, and that the short story Andy Weir (2009), "The Egg", shares a similar premise. But Roger Crisp attributes LELO to C.I. Lewis, which would predate both Ray and Weir, but I haven't traced that reference.

2. ^{^}

   John C. Harsanyi "Cardinal Utility in Welfare Economics and in the Theory of Risk-Taking" (1953) and "Cardinal Welfare, Individualistic Ethics, and Interpersonal Comparisons of Utility" (1955)

3. ^{^}

   See John Rawls (1971), "A Theory of Justice" and Samuel Freeman (2023) "Original Position".

4. ^{^}

   Bernard Williams discusses the notion of "unthinkable" options in his critique of utilitarianism.

   It could be a feature of a man’s moral outlook that he regarded certain courses of action as unthinkable, in the sense that he would not entertain the idea of doing them: and the witness to that might, in many cases, be that they simply would not come into his head. Entertaining certain alternatives, regarding them indeed as alternatives, is itself something that he regards as dishonourable or morally absurd.

    (Bernard Williams, 1973, "Utilitarianism: For and Against"). 

   This is distinct from options that are ruled out by moral side constraints or physical impossibility. As Williams puts it, "it is perfectly consistent, and it might be thought a mark of sense, to believe, while not being a consequentialist, that there was no type of action which satisfied [the condition of being morally prohibited whatever the consequences]" (Williams, 1973).

5. ^{^}

   On the flip-side, even if the social context $f:X\to S$ is fixed, we can nonetheless concoct for any option $x\in X$ a utility function $u:S\to R$ such that $x$ is permitted by the utilitarian principle $f\mapsto {\text{argmax}}_X(u\circ f)$. That is, any option will be permitted in any social context, provided the social utility function is sufficiently misspecified, no matter how ludicrous that choice would be.

   To prove this, define $u:S\to R$ as the indicator function for $\left\{f\right(x\left)\right\}\subseteq S$:

   $u\left(s\right)=\{\begin{array}{cc}1& \text{if}s=f\left(x\right)\\ 0& \text{if}s\ne f\left(x\right)\end{array}$

   This $u$ assigns utility 1 to the social outcome of choosing $x$, and 0 to all other outcomes, so a utilitarian planner maximizing this $u$ would be permitted to choose $x$, or any other option that leads to the same outcome as $x$. That is, $x^{\prime }\in {\text{argmax}}_X(u\circ f)⟺f\left(x\right)=f\left(x^{\prime }\right)$.

6. ^{^}

   Determining whether a statement $\theta$ has a proof that is less than $k$ bits long is an NP-complete problem. Even $k=1000$ would exceed the computational resources of the observable universe.

   Firstly, this problem belongs to the complexity class NP because, given a proof $x$ that is less than $k$ bits in length, it is possible to verify each step of the proof to ensure that it adheres to the rules of Peano Arithmetic (PA). The verification process can be completed in polynomial time with respect to the size of the proof.

   Moreover, this problem is NP-hard, as it is possible to reduce the Boolean Satisfiability Problem (SAT), which is known to be NP-hard, to our problem. To demonstrate this reduction, consider an instance of SAT with variables $x_1,\dots ,x_n$ and a Boolean formula $\varphi (x_1,\dots ,x_n)$. We can construct a statement $\theta$ in the following manner:

   $\exists x_1,\dots ,x_n:x_1\le 1\wedge \dots \wedge x_n\le 1\wedge \varphi (x_1,\dots ,x_n)$

   If the original Boolean formula is satisfiable, then this newly constructed formula is provable with a proof that requires only a polynomial number of bits. Furthermore, this reduction can be performed in polynomial time.

   Our problem is both in NP and NP-hard, and hence is NP-complete.

7. ^{^}

   Strictly speaking, the claim that a social planner cannot implement the utilitarian principle in this scenario relies on two key assumptions:

   (1) The social planner's decision-making process is instantiated by a physical system, such as a machine or computer, that exists in our universe and is bound by the laws of physics.

   (2) No physically realizable machine can efficiently solve NP-complete problems. In other words, the time required to find a solution grows exponentially with the size of the problem, quickly becoming infeasible for even moderately large instances.

   For a discussion of (1), see Scott Garrabrant and Abram Demski's 2018 article "Embedded Agency [LW · GW]". For a compelling defence of (2), see Scott Aaronson's 2005 paper "NP-complete Problems and Physical Reality".

8. ^{^}

   See Bales, R. E. (1971), ‘"Act utilitarianism: Account of right-making characteristics or decision making procedure", which "stress[es] the importance of maintaining a sharp distinction between (a) decision-making procedures, and (b) accounts of what makes right acts right."

9. ^{^}

   We've seen how utilitarianism demands superhuman computational resources from the social planner, in contrast to aggregativism. As I demonstrate below, a similar point can be made about noncomputational resources.

   Most humans cannot, I presume, jump exactly 45 cm. It's practically impossible for a typical human to reliably distinguish between jumping 45 cm and 46 cm, as the difference is too small to accurately control or perceive. Hence, in some circumstances, a human might either jump 45 cm or jump 46 cm; in other circumstances, they will surely do neither; but there are no circumstance where a human might jump 45 cm but surely won't jump 46 cm.

   Formally, let $X=\{0\text{cm},1\text{cm},\dots ,60\text{cm}\}$ denote all the possible heights that a human might jump. To say that the human cannot distinguish between 45 cm and 46 cm, we mean that $45\text{cm}\in \Pi \left(g\right)⟺46\text{cm}\in \Pi \left(g\right)$ for all personal contexts $g:X\to P$.

   Now, the aggregative principles satisfy a property called 'indistinguishable-options consistency'. Namely, If the aggregative principle permits (resp. forbids) jumping 45 cm in some social context, then it must also permit (resp. forbid) jumping 46 cm in that same context. The social planner is never permitted to jump 45 cm while forbidden to jump 46 cm, nor vice-versa. 

   More generally, if $x_1\in \Pi \left(g\right)⟺x_2\in \Pi \left(g\right)$ for all personal contexts $g:X\to P$, then $x_1\in \Pi (\zeta \circ f)⟺x_2\in \Pi (\zeta \circ f)$ for all social contexts $f:X\to S$.

   In contrast, utilitarian principles violate indistinguishable-options consistency. If $u:S\to R$ is any non-constant utility function, with $u\left(s^{-}\right)<u\left(s^{+}\right)$, then we can define the social context $f:X\to S$ as follows:

   $f\left(x\right)=\{\begin{array}{cc}{s}^{+}& \text{if}x=45\text{cm}\\ s^{-}& \text{if}x\ne 45\text{cm}\end{array}$

   The utilitarian planner maximizing $u$ would be obligated to jump exactly 45 cm, and forbidden to jump 46 cm, even though distinguishing between these two options is physically impossible.

10. ^{^}

    Philosophers like Bernard Williams (1981) rejected the codification of ethics into simple theories such as Kantianism or utilitarianism. “There cannot be any very interesting, tidy or self-contained theory of what morality is… nor… can there be an ethical theory, in the sense of a philosophical structure which, together with some degree of empirical fact, will yield a decision procedure for moral reasoning.”

11. ^{^}

    See Eliezer Yudkowsky on The Hidden Complexity of Wishes [LW · GW], Not for the Sake of Happiness (Alone) [LW · GW], and Fake Utility Functions [LW · GW].

12. ^{^}

    I've been persuaded by Brian Tomasik's writings, in particular "The Horror of Suffering" (2017) and "Preventing Extreme Suffering Has Moral Priority" (2016, video presentation, warning: disturbing content).

13. ^{^}

    In "Three Types of Negative Utilitarianism", Brian Tomasik uses a LELO-esque argument to support lexical-threshold negative utilitarianism. This position states that a small minority facing extreme suffering cannot be compensated by a miniscule benefit to a sufficiently large majority. He justifies this on the grounds that a self-interested human wouldn't desire the concatenation of those lives:

    A day in hell could not be outweighed by happiness:
    I would not accept a day in hell in exchange for any number of days in heaven. Here I'm thinking of hell as, for example, drowning in lava but with my pain mechanisms remaining intact for the whole day. Heaven just wouldn't be worth it, no matter how long. It seems like there's no comparison.

14. ^{^}

    To formalize this, let $\Phi :(X\to R)\to P\left(X\right)$ be any $R$-choice principle and let $(R,\prec )$ be any binary relation over the payoffs $R$. We say that $\Phi$ respects $\prec$ if, for all contexts $f:X\to R$ and all options $x\in \Phi \left(f\right)$, there exists no $x^{\prime }\in X$ such that $f\left(x\right)\prec f\left(x^{\prime }\right)$. In plain terms: if $\prec$ represents strict preference, then $\Phi$ never permits choosing a strictly dispreferred option. Moreover, we say that $\Phi$ has transitive preferences if it respects some transitive relation $\prec$.

    It's straightforward to show that ${\text{argmax}}_X:(X\to R)\to P\left(X\right)$ has transitive preferences: it respects the usual 'less than' ordering $(R,<)$ on real numbers, which is transitive. Furthermore, if $u:S\to R$ is any function and the $R$-choice principle $\Phi$ has transitive preferences, then so does the composite principle $\Psi :(X\to S)\to P\left(X\right)$ defined by $\Psi \left(f\right):=\Phi (u\circ f)$. Indeed, if $\Phi$ respects a relation $(R,\prec )$, then $\Psi$ respects the relation $(S,\dot{\prec })$ defined by $s\dot{\prec }{s}^{\prime }⟺u\left(s\right)\prec u\left(s^{\prime }\right)$, and $\dot{\prec }$ is transitive if $\prec$ is. Combining these observations: since ${\text{argmax}}_X$ has transitive preferences, so does a social planner following any utilitarian principle $f\mapsto {\text{argmax}}_X(u\circ f)$.

    However, the human behaviour model $\Pi :(X\to P)\to P\left(X\right)$ may lack transitive preferences. If so, then a social planner following the aggregative principle $f\mapsto \Pi (\zeta \circ f)$, for some social zeta function $\zeta :S\to P$, may also lack transitive preferences. This exposes the planner to 'ethical money pumps': a sequence of choices that leads to a strictly worse outcome than where they started, by exploiting their intransitive preferences. For example, the planner might trade policy A for B, B for C, and C back to A, each time accepting a small 'ethical cost' that compounds to a large overall loss.

15. ^{^}

    See e.g. "Affective Forecasting" (Gilbert and Wilson, 2003)

10 comments

Comments sorted by top scores.