Example population ethics: ordered discounted utility

post by Stuart_Armstrong · 2019-03-11T16:10:43.458Z · score: 14 (5 votes) · LW · GW · 13 comments

This article is a stub. Alas, you can't help Wikipedia (or LessWrong) by expanding it. Except through good comments.

Here I'll present an old idea for a theory of population ethics. This post exists mainly so that I can have something to point to when I need this example.

Given a total population , each with total individual utility over the whole of their lives, order them from lowest utility to the highest so that implies . These utilities are assumed to have a natural zero point (the "life worth living" standard, or similar).

Then pick some discount factor , and define the total utility of the world with population (which is the total population of the world across all time) as

This is a prioritarian utility that gives greater weight to those least well off. It is not average utilitarianism, and would advocate creating a human with utility larger than than all other humans (as long as it was positive), and would advocate against creating a human with negative utility (for a utility in between, it depends on the details). In the limit , it's total utilitarianism. Increasing someone's individual utility always improves the score. It (sometimes) accepts the "sadistic conclusion", but I've argued that that conclusion is misnamed (the conclusion is a choice between two negative outcomes, meaning that calling it "sadistic" is a poor choice - the preferred outcome is not a good one, just a less bad one). Killing people won't help, unless they will have future lifetime utility that is negative (as everyone that ever lived is included in the sum). Note that this sets up a minor asymmetry between not-creating people and killing them.

Do I endorse this? No; I think a genuine population ethics will be more complicated, and needs a greater asymmetry between life and death. But it's good enough for an example in many situations that come up.

13 comments

Comments sorted by top scores.

comment by rohinmshah · 2019-03-11T16:22:22.026Z · score: 4 (3 votes) · LW · GW

It does recommend against creating humans with lives barely worth living, and equivalently painlessly killing such people as well. If your population is a single person with utility 1000 and γ=.99, then this would recommend against creating a person with utility 1.

comment by Stuart_Armstrong · 2019-03-11T18:09:27.305Z · score: 3 (2 votes) · LW · GW

EDIT: I realised I wasn't clear that the sum was over everyone that ever lived. I've clarified that in the post.

Actually, it recommends killing only people who's future lifetime utility is about going to go negative, as the sum is over all humans in the world in total.

You're correct on the "not creating" incentives.

Now, this doesn't represent what I'd endorse (I prefer more asymmetry between life and death), but it's good enough as an example for most cases that come up.

comment by Joe_Collman · 2019-03-12T13:22:56.405Z · score: 1 (1 votes) · LW · GW

It's interesting. A few points:

Is there a natural extension for infinite population? It seems harder than most approaches to adapt.

I'm always suspicious of schemes that change what they advocate massively based on events a long time ago in a galaxy far, far away - in particular when it can have catastrophic implications. If it turns out there were 3^^^3 Jedi living in a perfect state of bliss, this advocates for preventing any more births now and forever.

Do you know a similar failure case for total utilitarianism? All the sadistic/repugnant/very-repugnant... conclusions seem to be comparing highly undesirable states - not attractor states. If we'd never want world A or B, wouldn't head towards B from A, and wouldn't head towards A from B (since there'd always be some preferable direction), does an A-vs-B comparison actually matter at all?

Total utilitarianism is an imperfect match for our intuitions when comparing arbitrary pairs of worlds, but I can't recall seeing any practical example where it'd lead to clearly bad decisions. (perhaps birth-vs-death considerations?)

In general, I'd be interested to know whether you think an objective measure of per-person utility even makes sense. People's take on their own situation tends to adapt to their expectations (as you'd expect, from an evolutionary fitness point of view). A zero-utility life from our perspective would probably look positive 1000 years ago, and negative (hopefully) in 100 years. This is likely true even if the past/future people were told in detail how the present-day 'zero' life felt from the inside: they'd assume our evaluation was simply wrong.

Or if we only care about (an objective measure of) subjective experience, does that mean we'd want people who're all supremely happy/fulfilled/... with their circumstances to the point of delusion?

Measuring personal utility can be seen as an orthogonal question, but if I'm aiming to match my intuitions I need to consider both. If I consider different fixed personal-utility-metrics, it's quite possible I'd arrive at a different population ethics. [edited from "different population utilities", which isn't what I meant]

I think you're working in the dark if you try to match population ethics to intuition without fixing some measure of personal utility (perhaps you have one in mind, but I'm pretty hazy myself :)).

comment by Stuart_Armstrong · 2019-03-13T12:49:02.542Z · score: 2 (1 votes) · LW · GW

Is there a natural extension for infinite population? It seems harder than most approaches to adapt.

None of the population ethics have decent extensions to infinite populations. I have a very separate idea for infinite populations here [LW · GW]. I suppose the extension of this method to infinite population would use the same method as in that post, but use instead of (where and are the limsup and liminf of utilities, respectively).

I'm always suspicious of schemes that change what they advocate massively based on events a long time ago in a galaxy far, far away - in particular when it can have catastrophic implications. If it turns out there were 3^^^3 Jedi living in a perfect state of bliss, this advocates for preventing any more births now and forever.

You can always zero out those utilities by decree, and only consider utilities that you can change. There are other patches you can apply. By talking this way, I'm revealing the principle I'm most willing to sacrifice: elegance.

Do you know a similar failure case for total utilitarianism? All the sadistic/repugnant/very-repugnant... conclusions seem to be comparing highly undesirable states - not attractor states. If we'd never want world A or B, wouldn't head towards B from A, and wouldn't head towards A from B (since there'd always be some preferable direction), does an A-vs-B comparison actually matter at all?

If A is repugnant and C is now, you can get from C to A by doing improvements (by the standard of total utilitarianism) every step of the way. Similarly, if B is worse than A on that standard, there is a hypothetical path from B to A which is an "improvement" at each step (most population ethics have this property, but not all - you need some form of "continuity").

It's possible that the most total-ut distribution of matter in the universe is a repugnant way; in that case, a sufficiently powerful AI may find a way to reach that.

In general, I'd be interested to know whether you think an objective measure of per-person utility even makes sense.

a) I don't think it makes sense in any strongly principled way, b) I'm trying to build one anyway [LW · GW] :-)

comment by Joe_Collman · 2019-03-13T16:00:10.454Z · score: 1 (1 votes) · LW · GW

Thanks. I'll check out the infinite idea.

On repugnance, I think I've been thinking too much in terms of human minds only. In that case there really doesn't seem to be a practical problem: certainly if C is now, continuous improvements might get us to a repugnant A - but my point is that that path wouldn't be anywhere close to optimal. Total-ut prefers A to C, but there'd be a vast range of preferable options every step of the way - so it'd always end up steering towards some other X rather than anything like A.

I think that's true if we restrict to human minds (the resource costs of running a barely content one being a similar order of magnitude to those of running a happy one).

But of course you're right as soon as we're talking about e.g. rats (or AI-designed molecular scale minds...). I can easily conceive of metrics valuing 50 happy rats over 1 happy human. I don't think rat-world fits most people's idea of utopia.

I think that's the style of repugnance that'd be a practical danger: vast amounts of happy-but-simple minds.

comment by Stuart_Armstrong · 2019-03-14T13:01:52.751Z · score: 2 (1 votes) · LW · GW

I think that's the style of repugnance that'd be a practical danger: vast amounts of happy-but-simple minds.

Yep, that does seem a risk. I think that's what the "muzak and potatoes" formulation of repugnance is about.

comment by Evan Clark (evan-clark) · 2019-03-11T20:04:41.409Z · score: 1 (1 votes) · LW · GW

It seems odd to me that it is so distribution-dependent. If there is a large number of people, with a large gap between the highest and the lowest, then it's worth killing (potentially most people) just to move the high utility individual down the preference ordering. One solution might be to fix the highest power of γ (for any population), and approach it across the summation in a way weighted by the flatness of the distribution.

Another issue is that two individuals with the same unweighted utility can become victims of the ordering, although that could be patched by grouping individuals by equal unweighted utility, and then summing over the weighted sums of the group utilities.

comment by Stuart_Armstrong · 2019-03-11T20:47:24.142Z · score: 4 (2 votes) · LW · GW

EDIT: I realised I wasn't clear that the sum was over everyone that ever lived. I've clarified that in the post.

Killing people with future lifetime non-negative utility won't help, as they will still be included in the sum.

Another issue is that two individuals with the same unweighted utility can become victims of the ordering

No. If , then . The ordering between identical utilities won't matter for the total sum, and the individual that is currently behind will be prioritised.

comment by Evan Clark (evan-clark) · 2019-03-11T21:20:14.231Z · score: 1 (1 votes) · LW · GW

My mistake with respect to the sum being over all time, thank you for clarifying.

No. If a=b, then a+γb=b+γa. The ordering between identical utilities won't matter for the total sum, and the individual that is currently behind will be prioritised.

While the ordering between identical utilities does not affect the total sum, it does affect the individual valuation. a can be prioritized over b just by the ordering, even though they have identical utility. Unless I am missing something obvious.

comment by Stuart_Armstrong · 2019-03-13T12:21:29.381Z · score: 2 (1 votes) · LW · GW

a can be prioritized over b just by the ordering, even though they have identical utility.

Nope. Their ordering is only arbitrary as long as they have exactly the same utility. As soon as a policy would result in one of them having higher utility than the other, their ordering is no longer arbitrary. So if we ignore other people means the term in the sum is . If , it's . If , it can be either term (and they are equal).

(I can explain in more detail if that's not enough?)

comment by Evan Clark (evan-clark) · 2019-03-13T19:30:51.145Z · score: 3 (2 votes) · LW · GW

I have realized that I am coming off like I don't understand algebra, which is a result of my failure to communicate. As unlikely as I am making it sound, I understand what you are saying and already knew it.

What I mean is this:

Despite a = b, it could "look like" a < b or b > a if you didn't have access to the world but only to the (expanded) sum. If you can ask for the difference between the total sum and the sum ignoring a, but not for the actual value of a.

I can't think of a non-pathological case where this would actually matter, but it seems like a desirable desideratum that a = b will always "look like" a = b regardless of what kind of (sufficiently fine-grained) information that you have.

EDIT : After reading your above comment about willingness to sacrifice elegance, I kind of wish I hadn't said anything at all, considering my comments are all in the interest of what I would consider elegance. To be sure, I think elegance is a legitimate practical concern, but I wouldn't have engaged with you initially had I known your view.

comment by Stuart_Armstrong · 2019-03-14T12:57:30.806Z · score: 2 (1 votes) · LW · GW

Hum, not entirely sure what you're getting at...

I'd say that always "looks like ", in the sense that there is a continuity in the overall ; small changes to our knowledge of and make small changes to our estimate of .

I'm not really sure what stronger condition you could want; after all, when , we can always write

as:

  • .

We could equivalently define that way, in fact (it generalises to larger sets of equal utilities).

Would that formulation help?

comment by Evan Clark (evan-clark) · 2019-03-14T14:27:11.976Z · score: 3 (2 votes) · LW · GW

My internal visualization is that all the individuals in the world are disjoint line segments of a certain length which laid end to end correspond to the world-segment, and that when the weighting-fairy (or whatever) passes through, sets of segments which were all previously the same length ought to still be sets of segments of the same length.

Honestly, I do apologize for spending so much of your time running you around in verbal circles because something didn't correspond to my internal model. Thank you for trying to understand/help.