Posts
Comments
My main objection for the simplified utility functions is that they are presented as depending only upon the current external state of the world in some vaguely linear and stable way. Every adjective in there corresponds to discarding a lot of useful information about preferences that people actually have.
The main argument I've heard for this kind of simplification is that your altruistic, morality-type preferences ought to be about the state of the external world because their subject is the wellbeing of other people, and the external world is where other people live. The linearity part is sort of an extension of the principle of treating people equally. I might be steelmanning it a little, a lot of times the argument is less that and more that having preferences that are in any way weird or complex is "arbitrary." I think this is based on the mistaken notion that "arbitrary" is a synonym for "picky" or "complicated."
I find this argument unpersuasive because altruism is also about respecting the preferences of others, and the preferences of others are, as you point out, extremely complicated and about all sorts of things other than the current state of the external world. I am also not sure that having nonlinear altruistic preferences is the same thing as not valuing people equally. And I think that our preferences about the welfare of others are often some of the most path-dependent preferences that we have.
EDIT: I have sense found this post, which discusses some similar arguments and refutes them more coherently than I do.
Second EDIT: I still find myself haunted by the "scary situation" I linked to and find myself wishing there was a way to tweak a utility function a little to avoid it, or at least get a better "exchange rate" than "double tiny good thing and more-than doubling horrible thing while keeping probability the same." I suppose there must be a way since the article I linked to said it would not work on all bounded utility functions.
In general though, this consideration is likely to be irrelevant. Most universes will be nowhere near the upper or lower bounds, and the chance of any individual's decision being single-handedly responsible for doing a universe scale shifts toward a utility bound is so tiny that even estimating orders of magnitude of the unlikelihood is difficult. These are angels-on-head-of-pin quibbles.
That makes sense. So it sounds like the Egyptology Objection is almost a form of Pascal's Mugging in and of itself. If you are confronted by a Mugger (or some other, slightly less stupid scenario where there is a tiny probability of vast utility or disutility) the odds that you are at a "place" on the utility function that would affect the credibility threshold for the Mugger one way or another are just as astronomical as the odds that the Mugger is giving you. So an agent with a bounded utility function is never obligated to research how much utility the rest of the universe has before rejecting the mugger's offer. They can just dismiss it as not credible and move on.
And Mugging-type scenarios are the only scenarios where this Egyptology stuff would really come up, because in normal situations with normal probabilities of normal amounts of (dis)utility, the rescaling and reshifting effect makes your "proximity to the bound" irrelevant to your behavior. That makes sense!
I also wanted to ask about something you said in an earlier comment:
I suspect most of the "scary situations" in these sorts of theories are artefacts of trying to formulate simplified situations to test specific principles, but accidentally throw out all the things that make utility functions a reasonable approximation to preference ordering. The quoted example definitely fits that description.
I am not sure I understand exactly what you mean by that. How do simplified hypotheticals for testing specific principles make utility functions fail to approximate preference ordering? I have a lot of difficulty with this, where I worry that if I do not have the perfect answer to various simplified hypotheticals it means that I do not understand anything about anything. But I also understand that simplified hypotheticals often causes errors like removing important details and reifying concepts.
The main "protection" of bounded utility is that at every point on the curve, the marginal utility of money is nonzero, and the threat of disutility is bounded. So there always exists some threshold credibility below which no threat (no matter how bad) makes expected utility positive for paying them.
That makes sense. What I am trying to figure out is, does that threshold credibility change depending on "where you are on the curve." To illustrate this, imagine two altruistic agents, A and B, who have the same bounded utility function. A lives in a horrifying hell world full of misery. B lives in a happy utopia. So A is a lot "closer" to the lower bound than B. Both A and B are confronted by a Pascal's Mugger who threatens them with an arbitrarily huge disutility.
Does the fact that agent B is "farther" from lower bound than agent A mean that the two agents have different credibility thresholds for rejecting the mugger? Because the amount of disutility that B needs to receive to get close to the lower bound is larger than the amount that A needs to receive? Or will their utility functions have the same credibility threshold because they have the same lower and upper bounds, regardless of "how much" utility or disutility they happen to "possess" at the moment? Again, I do not know if this is a coherent question or if it is born out of confusion about how utility functions work.
It seems to me that an agent with a bounded utility function shouldn't need to do any research about the state of the rest of the universe before dismissing Pascal's Mugging and other tiny probabilities of vast utilities as bad deals. That is why this question concerns me.
One continuous example of this is an exponential discounter, where the decisions are time-invariant but from a global view the space of potential future utility is exponentially shrinking.
Thanks, that example made it a lot easier to get my head around the idea! I think understand it better now. This might not be technically accurate, but to me having a uniform rescaling and reshifting of utility that preserves future decisions like that doesn't even feel like I am truly "valuing" future utility less. I know that in some sense I am, but it feels more like I am merely adjusting and recalibrating some technical details of my utility function in order to avoid "bugs" like Pascal's Mugging. It feels similar to making sure that all my preferences are transitive to avoid money pumps, the goal is to have a functional decision theory, rather to to change my fundamental values.
TLDR: What I really want to know is:
1. Is an agent with a bounded utility function justified (because of their bounded function) in rejecting any "Pascal's Mugging" type scenario with tiny probabilities of vast utilities, regardless of how much utility or disutility they happen to "have" at the moment? Does everything just rescale so that the Mugging is an equally bad deal no matter what the relative scale of future utility is?
2. If you have a bounded utility function, are your choices going to be the same regardless of how much utility various unchangeable events in the past generated for you? Does everything just rescale when you gain or lose a lot of utility so that the relative value of everything is the same? I expect the answer is going to be "yes" based on our previous discussion, but am a little uncertain because of the various confused thoughts on the subject that I have been having lately.
Full length Comment:
I don't think I explained my issue clearly. Those arguments about Pascal's Mugging are addressing it from the perspective of its unlikeliness, rather than using a bounded utility function against it.
I am trying to understand bounded utility functions and I think I am still very confused. What I am confused about right now is how a bounded utility function protects from Pascal's Mugging at different "points" along the function.
Imagine we have a bounded utility function that has a "S" curve shape. The function goes up and down from 0 and flattens as it approaches the upper and lower bounds.
If someone has utility at around 0, I see how they resist Pascal's Mugging. Regardless of whether the Mugging is a threat or a reward, it approaches their upper or lower bound and then diminishes. So utility can never "outrace" probability.
But what if they have a level of utility that is close to the upper bound and a Mugger offers a horrible threat? If the Mugger offered a threat that would reduce their utility to 0, would they respond differently than they would to one that would send it all the way to the lower bound? Would the threat get worse as the utility being cancelled out by the disutility got further from the bound and closer to 0? Or is the idea that in order for a threat/reward to qualify as a Pascal's Mugging it has to be so huge that it goes all the way down to a bound?
And if someone has a level of utility or disutility close to the bound, does that mean disutility matters more so they become a negative utilitarian close to the upper bound and a positive utilitarian close to the lower one? I don't think that is the case, I think that, as you said, "the relative scale of future utility makes no difference in short-term decisions." But I am confused about how.
I think I am probably just very confused in general about utility functions and about bounded utility functions. While some people have criticized bounded utility functions, I have never come across this specific type of criticism before. It seems far more likely that I am confused than that I am the first person to notice an obvious flaw.
Hi, one other problem occurred to me in regards to short term decisions and bounded utility.
Suppose you are in a situation where you have a bounded utility function, plus a truly tremendous amount of utility. Maybe you're an immortal altruist who has helped quadrillions of people, maybe you're an immortal egoist who has lived an immensely long and happy life. You are very certain that all of that was real, and it is in the past and can't be changed.
You then confront a Pascal's Mugger who threatens to inflict a tremendous amount of disutility unless you give the $5. If you're an altruist they threaten to torture quintillions of people, if you are an egoist they threaten to torture you for a quintillion years, something like that. As with standard Pascal's mugging, the odds of them be able to carry this threat out are astronomically unlikely.
In this case, it still fells like you ought to ignore the mugger. Does that make sense considering that, even though your bounded utility function assigns less disvalue to such a threat, it also assigns less value to the $5 because you have so much utility already? Plus, if they are able to carry out their threat, they would be able to significantly lower your utility so that it is much "further away from the bound" than it was before. Does it matter that as they push your utility further and further "down" away from the bound, utility becomes "more valuable."
Or am I completely misunderstanding how bounded utility is calculated? I've never seen this specific criticism of bounded utility functions before, and much smarter people than me have studied this issue, so I imagine that I must be? I am not sure exactly how adding utility and subtracting disutility is calculated. It seems like if the immortal altruist whose helped quadrillions of people has a choice between gaining 3 utilons, or inflicting 2 disutilons to gain 5 utilitons, that they should be indifferent between the two, even if they have a ton of utility and very little disutility in their past.
Thanks, again for your help :) That makes me feel a lot better. I have the twin difficulties of having severe OCD-related anxiety about weird decision theory problems, and being rather poor at the math required to understand them.
The case of the immortal who becomes uncertain of the reality of their experiences is I think what that "Pascal's Mugging for Bounded Utilities" article I linked to the the OP was getting at. But it's a relief to see that it's just a subset of decisions under uncertainty, rather than a special weird problem.
the importance to the immortal of the welfare of one particular region of any randomly selected planet of those 10^30 might be less than that of Ancient Egypt. Even if they're very altruistic.
Ok, thanks, I get that now, I appreciate your help. The thing I am really wondering is, does this make any difference at all to how that immortal would make decisions once Ancient Egypt is in the past and cannot be changed? Assuming that they have one of those bounded utility functions where their utility is asymptotic to the bound, but never actually reaches it, I don't feel like it necessarily would.
If Ancient Egypt is in the past and can't be changed, the immortal might, in some kind of abstract sense, value that randomly selected planet of those 10^30 worlds less than they valued Egypt. But if they are actually in a situation where they are on that random planet, and need to make altruistic decisions about helping the people on that planet, then their decisions shouldn't really be affected. Even if the welfare of that planet is less valuable to them than the welfare of Ancient Egypt, that shouldn't matter if their decisions don't affect Ancient Egypt and only affect the planet. They would be trading less valuable welfare off against other less valuable welfare, so it would even out. Since their utility function is asymptotic to the bound, they would still act to increase their utility, even if the amount of utility they can generate is very small.
I am totally willing to accept the Egyptology argument if all it is saying is that past events that cannot be changed might affect the value of present-day events in some abstract sense (at least if you have a bounded utility function). Where I have trouble accepting it is if those same unchangeable past events might significantly affect what choices you have to make about future events that you can change. If future welfare is only 0.1x as valuable as past welfare, that doesn't really matter, because future welfare is the only welfare you are able to affect. If it's only possible to make a tiny difference, then you might as well try, because a tiny difference is better than no difference. The only time when the tininess seems relevant to decisions is Pascal's Mugging type scenarios where one decision can generate tiny possibilities of huge utility.
The phrasing here seems to be a confused form of decision making under uncertainty. Instead of the agent saying "I don't know what the distribution of outcomes will be", it's phrased as "I don't know what my utility function is".
I think part of it is that I am conflating two different parts of the Egyptology problem. One part is uncertainty: it isn't possible to know certain facts about the welfare of Ancient Egyptians that might affect how "close to the bound" you are. The other part is that most people have a strong intuition that those facts aren't relevant to our decisions, whether we are certain of them or not. But there's this argument that those facts are relevant if you have an altruistic bounded utility function because they affect how much diminishing returns your function has.
For example, I can imagine that if I was an altruistic immortal who was alive during ancient Egypt, I might be unwilling to trade a certainty of a good outcome in ancient Egypt for an uncertain amazingly terrific outcome in the far future because of my bounded utility function. That's all good, it should help me avoid Pascal's Mugging. But once I've lived until the present day, it feels like I should continue acting the same way I did in the past, continue to be altruistic, but in a bounded fashion. It doesn't feel like I should conclude that, because of my achievements as an altruist in Ancient Egypt, that there is less value to being an altruist in the present day.
In the case of the immortal, I do have all the facts about Ancient Egypt, but they don't seem relevant to what I am doing now. But in the past, in Egypt, I was unwilling to trade certain good outcomes for uncertain terrific ones because my bounded utility function meant I didn't value the larger ones linearly. Now that the events of Egypt are in the past and can't be changed, does that mean I value everything less? Does it matter if I do, if the decrease in value is proportionate? If I treat altruism in the present day as valuable, does that contradict the fact that I discounted that same value back in Ancient Egypt?
I think that's why I'm phrasing it as being uncertain of what my utility function is. It feels like if I have a bounded utility function, I should be unwilling (within limits) to trade a sure thing for a small possibility of vast utility, thereby avoiding Pascal's Mugging and similar problems. But it also feels like, once I have that sure thing, and the fact that I have it cannot be changed, I should be able to continue seeking more utility, and how many sure things I have accumulated in the past should not change that.
I really still don't know what you mean by "knowing how close to the bound you are".
What I mean is, if I have a bounded utility function where there is some value, X, and (because the function is bounded) X diminishes in value the more of it there is, what if I don't know how much X there is?
For example, suppose I have a strong altruistic preference that the universe have lots of happy people. This preference is not restricted by time and space, it counts the existence of happy people as a good thing regardless of where or when they exist. This preference is also agent neutral, it does not matter whether I, personally, am responsible for those people existing and being happy, it is good regardless. This preference is part of a bounded utility function, so adding more happy people starts to have diminishing returns the closer one gets to a certain bound. This allows me to avoid Pascal's Mugging.
However, if adding more people has diminishing returns because the function is bounded, and my preference is not restricted by time, space, or agency, that means that I have no way of knowing what those diminishing returns are unless I know how many happy people have ever existed in the universe. If there are diminishing returns based on how many people there are, total, in the universe, then the value of adding more people in the future might change depending on how many people existed in the past.
That is what I mean by "knowing how close to the bound" I am. If I value some "X", what if it isn't possible to know how much X there is? (like I said before, a version of this for egoistic preferences might be if the X is happiness over your lifetime, and you don't know how much X there is because you have amnesia or something).
I was hoping that I might be able to fix this issue by making a bounded utility function where X diminishes in value smoothly and proportionately. So a million happy people in ancient Egypt has proportional diminishing returns to a billion and so on. So when I am making choices about maximizing X in the present, the amount of X I get is diminished in value, but it is proportionately diminished, so the decisions that I make remain the same. If there was a vast population in the past, the amount of X I can generate has very small value according to a bounded utility function. But that doesn't matter because it's all that I can do.
That way, even if X decreases in value the more of it there is, it will not effect any choices I make where I need to choose between different probabilities of getting different amounts of X in the future.
I suppose I could also solve it by making all of my preferences agent-relative instead of agent-neutral, but I would like to avoid that. Like most people I have a strong moral intuition that my altruistic preferences should be agent-neutral. I suppose it might also get me into conflict with other agents with bounded agent-relative utility functions if we value the same act differently.
If I am explaining this idea poorly, let me try directing you to some of the papers I am referencing. Besides the one I mentioned in the OP, there is this one by Beckstead and Thomas (pages 16, 17, and 18 are where it discusses it).
REA doesn't help at all there, though. You're still computing U(2X days of torture) - U(X days of torture)
I think I see my mistake now, I was treating a bounded utility function using REA as subtracting the "unbounded" utilities of the two choices and then comparing the post-subtraction results using the bounded utility function. It looks like you are supposed to judge each one's utility by the bounded function before subtracting them.
Unfortunately REA doesn't change anything at all for bounded utility functions. It only makes any difference for unbounded ones.
That's unfortunate. I was really hoping that it could deal with the Egyptology scenario by subtracting the unknown utility value of Ancient Egypt and only comparing the difference in utility between the two scenarios. That way the total utilitarian (or some other type of altruist) with a bounded utility function would not need to research how much utility the people of Ancient Egypt had in order to know how good adding happy people to the present day world is. That just seems insanely counterintuitive.
I suppose there might be some other way around the Egyptology issue. Maybe if you have a bounded or nonlinear utility function that is sloped at the correct rate it will give the same answer regardless of how happy the Ancient Egyptians were. If they were super happy then the value of whatever good you do in the present is in some sense reduced. But the value of whatever resources you would sacrifice in order to do good is reduced as well, so it all evens out. Similarly, if they weren't that happy, the value of the good you do is increased, but the value of whatever you sacrifice in order to do that good is increased proportionately. So a utilitarian can go ahead and ignore how happy the ancient Egyptians were when doing their calculations.
It seems like this might work if the bounded function has adding happy lives have diminishing returns at a reasonably steady and proportional rate (but not so steady that it is effectively unbounded and can be Pascal's Mugged).
With the "long lived egoist" example I was trying to come up with a personal equivalent to the Egyptology problem. In the Egyptology problem, a utilitarian does not know how close they are to the "bound" of their bounded utility function because they do not know how happy the ancient Egyptians were. In the long lived egoist example, they do not know how close to the bound they are because they don't know exactly how happy and long lived their past self was. It also seems insanely counterintuitive to say that, if you have a bounded utility function, you need to figure out exactly how happy you were as a child in order to figure out how good it is for you to be happy in the future. Again, I wonder if a solution might be to have a bounded utility function with returns that diminish at a steady and proportional rate.
Thank you for your reply. That was extremely helpful to have someone crunch the numbers. I am always afraid of transitivity problems when considering ideas like this, and I am glad it might be possible to avoid the Egyptology objection without introducing any.
Thanks a lot for the reply. That makes a lot of sense and puts my mind more at ease.
To me this sounds more like any non-linear utility, not specifically bounded utility.
You're probably right, a lot of my math is shaky. Let me try to explain the genesis of the example I used. I was trying to test REA for transitivity problems because I thought that it might have some further advantages to conventional theories. In particular, it seemed to me that by subtracting before averaging, REA could avoid the two examples those articles I references:
1. The total utilitarian with a bounded utility function who needs to research how many happy people lived in ancient Egypt to establish how "close to the bound" they were and therefore how much they should discount future utility.
2. The very long lived egoist with a bounded utility function who vulnerable to Pascal's mugging because they are unsure of how many happy years they have lived already (and therefore how "close to the bound" they were).
It seemed like REA, by subtracting past utility that they cannot change before doing the calculation, could avoid both those problems. I do not know if those are real problems or if a non-linear/bounded utility with a correctly calibrated discount rate could avoid them anyway, but it seemed worthwhile to find ways around them. But I was really worried that REA might create intransitivity issues with bounded utility functions, the lottery example I was using was an example of the kind of intransitivity problem that I was thinking of.
It also occurred to me that REA might avoid another peril of bounded utility functions that I read about in this article. Here is the relevant quote:
"if you have a bounded utility function and were presented with the following scary situation: “Heads, 1 day of happiness for you, tails, everyone is tortured for a trillion days” you would (if given the opportunity) increase the stakes, preferring the following situation: “Heads, 2 days of happiness for you, tails, everyone is tortured forever. (This particular example wouldn’t work for all bounded utility functions, of course, but something of similar structure would.)”
It seems like REA might be able to avoid that. If we imagine that the person is given a choice between two coins, since they have to pick one, the "one day of happiness+trillion days of torture" is subtracted beforehand, so all the person needs to do is weigh the difference. Even if we get rid of the additional complications of computing infinity that "tortured forever" creates, by replacing it with some larger number like "2 trillion days", I think it might avoid it.
But I might be wrong about that, especially if REA always gives the same answers in finite situations. If that's the case it just might be better to find a formulation of an unbounded utility function that does its best to avoid Pascal's Mugging and also the "scary situations" from the article, even if it does it imperfectly.
That aside, relative expected value is purely a patch that works around some specific problems with infinite expected values, and gives exactly the same results in all cases with finite expected values.
That's what I thought as well. But then it occurred to me that REA might not give exactly the same results in all cases with finite expected values if one has a bounded utility function. If I am right, this could result in scenarios where someone could have circular values or end up the victim of a money pump.
For example, imagine there is a lottery that costs $1 to for a ticket and generates x utility for odds of y. The value for x is very large, the value for y is quite small, like in Pascal's mugging. A person with a bounded utility function does not enter it. However, imagine that there is another lottery that costs a penny for a ticket, and generates 0.01x utility for odds of y. Because this person's utility function is bounded, y odds of 0.01x utility is worth a penny to them, even though y odds of x utility is not worth a dollar to them. The person buys a ticket for a penny. Then they are offered a chance to buy another. Because they are using REA, they only count the difference in utility from buying the new ticket, and do not count the ticket they already have, so they buy another. Eventually they buy 100 tickets. Then someone offers to buy the tickets from them for a dollar. Because they have a bounded utility function, y odds of winning x are less valuable than a dollar, so they take the trade. They are now back where they started.
Does that make sense? Or have I made some sort of error somewhere? (maybe pennies become more valuable the less you have, which would interrupt the cycle?) It seems like with a bounded utility function, REA might have transitivity problems like that. Or have I made a mistake and misunderstood how to discount using REA?
I am really concerned about this, because REA seems like a nice way to address the Egyptology objection to bounded utility functions. You don't need to determine how much utility already exists in the world by studying Ancient Egypt, because you only take into account the difference in utility, not the total utility, when calculating where the bound is in your utility function. Ditto for the Pascal's mugging example. So I really want there to be a way to discount the Egyptology stuff, without also generating intransitive preferences.
That can be said about any period in life. It's just a matter of perspective and circumstances. The best years are never the same for different people.
That's true, but I think that for the overwhelming majority of people, their childhoods and young adulthoods were at the very least good years, even if they're not always the best. They are years that contain significantly more good than bad for most people. So if you create a new adult who never had a childhood, and whose lifespan is proportionately shorter, they will have a lower total amount of wellbeing over their lifetime than someone who had a full-length life that included a childhood.
I took a crack at figuring it out here.
I basically take a similar approach to you. I give animals a smaller -u0 penalty if they are less self-aware and less capable of forming the sort of complex eudaimonic preferences that human beings can. I also treat complex eudaimonic preferences as generating greater moral value when satisfied in order to avoid incentivizing creating animals over creating humans.
I think another good way to look at u0 that compliments yours is to look at it as the "penalty for dying with many preferences left unsatisfied." Pretty much everyone dies with some things that they wanted to do left undone. I think most people have a strong moral intuition that being unable to fulfill major life desires and projects is tragic, and think a major reason death is bad is that it makes us unable to do even more of what we want to do with our lives. I think we could have u0 represent that intuition.
If we go back to Peter Singer's original formulation of this topic, we can think of unsatisfied preferences as "debts" that are unpaid. So if we have a choice between creating two people who live x years, or 1 person who lives 2x years, assuming their total lifetime happiness is otherwise the same, we should prefer the one person for 2x years. This is because the two people living x years generate the same amount of happiness, but twice the amount of "debt" from unfulfilled preferences. Everyone will die with some unfulfilled preferences because everyone will always want more, and that's fine and part of being human.
Obviously we need to calibrate this idea delicately in order to avoid any counterintuitive conclusions. If we treat creating a preference as a "debt" and satisfying it as merely "paying the debt" to "break even" then we get anti-natalism. We need to treat the "debt" that creating a preference generates as an "investment" that can "pay off" by creating tremendous happiness/life satisfaction when it is satisfied, but occasionally fails to "pay off" if its satisfaction is thwarted by death or something else.
I think that this approach could also address Isnasene's question below of figuring out the -u0 penatly for nonhuman animals. Drawing from Singer again, since nonhuman animals are not mentally capable of having complex preferences for the future, they generate a smaller u0 penalty. The preferences that they die without having satisfied are not as strong or complex. This fits nicely with the human intuition that animals are more "replaceable" than humans and are of lesser (although nonzero) moral value. It also fits the intuition that animals with more advanced, human-like minds are of greater moral value.
Using that approach for animals also underscores the importance of treating creating preferences as an "investment" that can "pay off." Otherwise it generates the counterintuitive conclusion that we should often favor creating animals over humans, since they have a lower u0 penalty. Treating complex preference creation as an "investment" means that humans are capable of generating far greater happiness/satisfaction than animals, which more than outweighs our greater u0 penalty.
We would also need some sort of way to avoid incentivizing the creation of intelligent creatures with weird preferences that are extremely easy to satisfy, or a strong preference for living a short life as an end in itself. This is a problem pretty much all forms of utilitarianism suffer from. I'm comfortable with just adding some kind of hack massively penalizing the creation of creatures with preferences that do not somehow fit with some broad human idea of eudaimonia.
Point taken, but for the average person, the time period of growing up isn't just a joyless period where they do nothing but train and invest in the future. Most people remember their childhoods as a period of joy and their college years as some of the best of their lives. Growing and learning isn't just preparation for the future, people find large portions of it to be fun. So the "existing" person would be deprived of all that, whereas the new person would not be.
If someone is in a rut and could either commit suicide or take the reprogramming drug (and expects to have to take it four times before randomizing to a personality that is better than rerolling a new one), why is that worse than killing them and allowing a new human to be created?
If such a drug is so powerful that the new personality is essentially a new person, then you have created a new person whose lifespan will be a normal human lifespan minus however long the original person lived before they got in a rut. By contrast, if they commit suicide and you create a new human, you have created a new person who will likely live a normal human lifespan. So taking the drug even once is clearly worse than suicide + replacement since, all else being equal, it is better to create a someone with a longer lifespan than a shorter one (assuming their lifespan is positive, overall, of course).
So if they don't want to be killed, that counts as a negative if we do that, even if we replace them with someone happier.
I have that idea as my "line of retreat." My issue with it is that it is hard to calibrate it so that it leaves as big a birth-death asymmetry as I want without degenerating into full-blown anti-natalism. There needs to be some way to say that the new happy person's happiness can't compensate for the original person's death without saying that the original person's own happiness can't compensate for their own death, which is hard. If I calibrate it to avoid anti-natalism it becomes such a small negative that it seems like it could easily be overcome by adding more people with only a little more welfare.
There's also the two step "kill and replace" method, where in step one you add a new life barely worth living without affecting anyone. Since the new person exists now, they count the same as everyone else, so then in the second step you kill someone and transfer their resources to the new person. If this process gives the new person the same amount of utility as the old one, it seems neutral under total utilitarianism. I suppose under total preference utilitarianism its somewhat worse, since you now have two people dying with unsatisfied preferences instead of one, but it doesn't seem like a big enough asymmetry for me.
I feel like in order to reject the two step process, and to have as big an asymmetry as I want, I need to be able to reject "mere addition" and accept the Sadistic Conclusion. But that in turn leads to "galaxy far far away issues" where it becomes wrong to have children because of happy people in some far off place. Or "Egyptology" issues where its better for the world to end than for it to decline so future people have somewhat worse lives, and we are obligated to make sure the Ancient Egyptians didn't have way better lives than ours before we decide on having children. I just don't know. I want it to stop hurting my brain so badly, but I keep worrying about how there's no solution that isn't horrible or ridiculous.
This has degenerate solutions too - it incentivises producing beings that are very easy to satisfy and that don't mind being killed.
For this one, I am just willing to just decree that creating creatures with a diverse variety of complex human-like psychologies is good, and creating creatures with weird minmaxing unambitious creatures is bad (or at least massively sub-optimal). To put it another way, Human Nature is morally valuable and needs to be protected.
Another resource that helped me on this was Derek Parfit's essay "What Makes Someone's Life Go Best." You might find it helpful, it parallels some of your own work on personal identity and preferences. The essay describes which of our preferences we feel count as part of our "self interest" and which do not. It helped me understand things, like why people general feel obligated to respect people's "self interest" preferences (i.e. being happy, not dying), but not their "moral preferences" (i.e. making the country a theocracy, executing heretics).
Parfit's "Success Theory," as he calls it, basically argues that only preferences that are "about your own life" count as "welfare" or "self interest." So that means that we would not be making the world a better place by adding lives who prefer that the speed of light stay constant, or that electrons keep having negative charges. That doesn't defuse the problem entirely, you could still imagine creating creatures with super unambitious life goals. But it gets it part of the way, the rest, again, I deal with by "defending Human Nature."
I'm more wanting to automate the construction of values from human preferences
I had a question about that. It is probably a silly question since my understanding of decision and game theory is poor. When you were working on that you said that there was no independence of irrelevant alternatives. I've noticed that IIA is something that trips me up a lot when I think about population ethics. I want to be able to say something like "Adding more lives might be bad if there is still the option to improve existing ones instead, but might be good if the existing ones have already died and that option is foreclosed." This violates IIA because I am conditioning whether adding more lives is good on whether there is another alternative or not.
I was wondering if my brain might be doing the thing you described in your post on no IIA, where it is smashing two different values together and getting different results if there are more alternatives. It probably isn't I am probably just being irrational, but reading that post just felt familiar.
You can always zero out those utilities by decree, and only consider utilities that you can change. There are other patches you can apply. By talking this way, I'm revealing the principle I'm most willing to sacrifice: elegance.
It's been a long time since you posted this, but if you see my comment, I'd be curious about what some others patches one could apply are. I have pretty severe scrupulosity issues around population ethics and often have trouble functioning because I can't stop thinking about them. I dislike pure total utilitarianism, but I have trouble rejecting it precisely because of "galaxy far far away" type issues. I spend a lot of time worrying about the idea that I am forced to choose between two alternatives:
1) That (to paraphrase what you said in your critique of total utilitarianism) it is a morally neutral act to kill someone if you replace them with someone whose lifetime utility is equal to the first person's remaining lifetime utility (and on a larger scale, the Repugnant Conclusion), or
2.That the human race might be obligated to go extinct if it turns out there is some utopia in some other branch of the multiverse, or the Andromeda Galaxy, or in some ancient, undiscovered fallen civilization in the past. Or that if the Earth was going to explode and I could press a button to save it, but it would result in future generations living slightly lower quality lives than present generations, I shouldn't push the button.
I'd really like to know some ways that I can reject both 1 and 2. I really admire your work on population ethics and find that your thinking on the subject is really closely aligned with my own, except that you're better at it than me :)
You can get mind states that are ambiguous mixes of awake and asleep.
I am having trouble parsing this statement. Does it mean that when simulating a mind you could also simulate ambiguous awake/asleep in addition to simulating sleep and wakefulness? Or does it mean that a stored, unsimulated mind is ambiguously neither awake or asleep?
That makes a lot of sense, thank you.
Thanks for the reply. It sounds like maybe my mistake was assuming that unsimulated brain data was functionally and morally equivalent to an unconscious brain. From what you are saying it sounds like the data would need to be simulated even to generate unconsciousness.
And much like Vaniver below (above? earlier!), I am unsure how to translate these sorts of claims into anything testable
One thing I consider very suspicious is that deaf people often don't just deny the terminal value of hearing. They also deny its instrumental value. The instrumental values of hearing are obvious. This indicates to me that they are denying it for self-esteem reasons and group loyalty reasons, the same way I have occasionally heard multiculturalists claim behaviors of obvious instrumental value (like being on time) are merely the subjective values of Western culture.
The typical defense of this denial (and other disability-rights type claims) is hearing only has instrumental value because society is structured in a way that makes use of it. But this is obviously false, hearing would be useful on a desert island, and there are some disabilities that society is not technologically capable of solving (there's no way to translate instrumental music into sign language). Plus, structuring society around disabilities is essentially having society pay to enable a person instead of having biology do it for free. Obviously it's better than not accommodating them, but it;s even better to have biology do the accommodation for free if that is possible.
I think another factor is simply my knowledge of the human brain structure, and the psychological unity of humankind. It seems like it would be a much smaller departure from standard brain design to switch the specific target of the "romance" module of the brain, than it would be to completely erase all desire to enjoy the pleasures that a sense of hearing can provide us, and to assign terminal value to being inconvenienced by things like not being able to talk to people who aren't in your visual range.
I think another thing that supports my intuitions is Bostrom's Reversal test. Imagine instead of discussing giving a preexisting sense to people who lack it, we were considering giving people a new sense that no human being has ever had before. Should we do that? If there were no side effects, I would say yes! As I told Vaniver in my reply to them, I really want to be able sense magnetic fields. Seeing infrared and ultraviolet would also be fun. The fact that my intuitions are the same in the Reversal Test provides evidence that they are not based on the Status Quo Bias.
it may be clearer to consider counterfactual mes of every possible sexual orientation, and comparing the justifications they can come up with for why it's egosyntonic to have the orientation that they have.
I think that maybe all of them would be perfectly justifying in saying that their sexual orientation is a terminal value and the buck stops there.
On the other hand, I'm nowhere near 100% sure I wouldn't take a pill to make me bisexual.
If you kept all of my values the same and deleted my sexual orientation, what would regrow?
I think a way to help tease out your intuition would be Bostrom's reversal test. If transhumanist scientists invented a new kind of sexual orientation, a new kind of sexual experiences, and so on, would you want to be modified to be able to enjoy this new, never before seen type of sex. I don't know how you'd reply, for me it would probably be yes or no, depending on specific details of the new kind of sex.
I think the reason I would sometimes say yes is that I have a strong preexisting preference for novelty and novel experiences. So my desire for new sexual preferences would grow out of that.
Incidentally, Bostrom's reversal test also supports my intuitions about deafness. If transhumanists invented new senses that no human has ever had before, would I want to have them if there were no side effects? Of course I would! I especially want to be able to sense magnetic fields like sharks can.
It seems to me that most people lack the ability to be aroused by people--typically, their ability is seriously limited, to half of the population at most.
When I was talking about being queer I wasn't just talking about the experience of being aroused, I was talking about the desire to have that experience, and that experience being egosyntonic. It's fairly easy to rephrase any preference a person has to sound like an ability or lack thereof. For instance, you could say that I lack the ability to enjoy skinning people alive. But that's because I don't want to skin people alive, or to enjoy it! That's a terminal value, the buck stops there.
Some other factors to consider:
- Even if I was to define "being aroused" as an ability, that doesn't perfectly map onto the discuss. In the case of removing deafness we are adding an ability. In the case of changing queerness to heterosexuality, we are either removing an ability to find some people arousing and replacing it with a different one (in the case of homosexuals) or removing an ability and replacing it with nothing (in the case of bisexuals).
- Arousal has very little instrumental value compared to hearing. Even if someone with the power of hearing took no pleasure from music or people's voices they would still benefit from being able to hear people talk outside of their visual range, and to hear cars coming when they cross the street. I can see deaf people denying the terminal benefits of hearing, but denying the instrumental ones seems obviously crazy.
I can't claim that I would choose to be gay for that reason, starting from emptiness.
Starting from emptiness you would be completely indifferent to everything, including changes to your future preferences. To paraphrase Eliezer, you would be a rock, not a perfectly impartial being.
At some point you just have to say "These are my terminal values, the buck stops here."
Now, while I would not say it is impossible to create a creature that assigns a terminal value to deafness, I find it unlikely that humans are such creatures. The way human psychology works makes me assign a much higher probability to their being self-deceived for group status purposes.
But since then, you've concluded that being queer isn't actually something (at least some people, like me) differentially approve of.
I'm not sure what I wrote that gave you this idea. I do think that queer people approve of being queer. What I'm talking about when I say "approval" is preferences that are ego-syntonic, that are line with the kind of person they want to be. Most queer people consider their preference to be ego-syntonic. Being queer is the kind of person they want to be and they would not change it if they could. Those who do not are usually motivated by mistaken religious ideas, rather than clearly reasoned disapproval.
What I am trying to say is that being queer is a statement about what people want to do. When we say that someone is queer that means that they have a desire to engage in romantic and sexual relationships that are different from the heterosexual norm. This desire is ego-syntonic, it is approved of.
Being deaf, by contrast, is a statement about what people are able do. They lack the ability to hear things.
If you removed someone's deafness, none of their desires would change. They would still want everything they wanted before they were deaf. If they were really attached to their current lifestyle they could buy earplugs. By contrast, if you changed a queer person into a straight person, they would stop wanting to have non-heteronormative relationships. They'd be able to continue their current lifestyle (or at least, as able as anyone is in a heteronormative society), but they wouldn't want to.
There are some people who claim that they prefer being deaf to being able to hear, and that being deaf is ego-syntonic. I believe that they are confused. I think what they really value isn't being deaf, it's the community that they have built with other deaf people. They are confusing their preference to display loyalty to their community with with a preference to not be able to hear. In addition I think they are confused for some other reasons:
- Sour grapes. When people are unable to do something, they often convince themselves they didn't want to do it anyway in order to assuage their ego.
- Confusing "life could be better" with "life is not worth living." As I said before, a lot of disability rights advocates seem to think that if you admit that their disability makes their life even slightly worse, that means their life is not worth living at all and they should be euthanized. This is not true.
- If people got hit in the head with a baseball bat every day.....
- Happy death spirals around their community. They love their community and want to say more and more good things about it. So they say that their community is so awesome that living in it is worth being significantly less good at sensing one's surroundings.
To sum it up, I believe that being queer is an ego-syntonic desire. I believe that being deaf is not ego-syntonic, but people say it is out of a desire to have self-esteem and be proud of and loyal to the deaf community.
Rereading your original comment keeping in mind that you're talking mostly about approval rather than desire or preference... so, would you say that Deaf people necessarily disapprove of deafness?
I'd say that a good portion of them do approve of it. There seem to be a lot of disability rights activists who seem to think that being disabled and making more disabled people is okay.
I should also mention, however, that I do think it is possible to mistakenly approve or disapprove of something. For instance I used to disapprove of pornography and voluntary prostitution. However, I eventually realized that the arguments for why those things were bad were wrong/incoherent, and realized that I should never have disapproved of those things. Disapproval of pornography and voluntary prostitution was never my CEV.
I think a large portion of disability-rights activists are also confused in their thinking, and would have different views if their thinking was clearer. For instance, many disability rights activists seem to think that any suggestion that disability is bad implies that the lives of disabled people aren't worth living and that they should all be involuntarily euthanized, which is obviously false. It's possible to believe your life is worth living while simultaneously believing it could be better.
It's not clear to me how this difference justifies the distinction in my thinking I was describing.
I believe the difference is that in the case of deaf people, you are improving their lives by giving them more abilities to achieve the values they have (in this case, an extra sense). By contrast, with queerness you are erasing a value a person has and replacing it with a different value that is easier to achieve. I believe that helping a person achieve their existing values is a laudable goal, but that changing a person's values is usually morally problematic, even if their new values are easier to achieve than their old ones.
Now, keep in mind that I am speaking in principle, not in practice. In the real-life case of deafness this issue is more complicated than the way I just described it. There are other issues, for instance, the value of an extra sense is to some extent tied to the support mechanisms society has developed for it. I think that the deaf community may be voicing a valid concern that society has good set of support mechanism for people who are fully deaf and fully hearing, but not as good mechanisms for people with the kind of mid-range hearing that cochlear implants provide.
But those are concerns of practice, not principle. In principle having extra senses should make it easier to achieve your values. I mean, wouldn't you want super-hearing, microscopic vision, etc if you could get them without any side-effects.
How do we tell whether what I value is to find a mate of Type A, or to find a mate I find attractive?
I think the fact that you are unwilling to have your criteria for attractiveness be modified is good evidence that it is the former and not the latter.
Is this simply a semantic disagreement -- that is, do we just have different understandings of what the phrase "who I am" refers to? Or is there something we'd expect to observe differently in the world were you correct and I mistaken about this?
I think there are two issues, one is semantic, the other is that I did completely understand what you meant by being changed into someone who isn't queer.
First, the semantic issue. I have been trying to approach the issue of Personal Identity by righting a wrong question. Instead of asking "Am I the same person as him?" I instead ask "How desirable would it be for me to change into that person?" I find that this approache generates the same intuitive results as traditional approaches to personal identity (for instance both approaches identify being killed and being wireheaded as very undesirable outcomes) but doesn't get bogged down by the issues of what exactly it means to be "the same."
Saying that you literally wouldn't be the same person was hyperbolic of me. I was trying to draw attention to the fact that our values are an important part of who we are, and that changing our values can change our identity. It would be more accurate to say something like "the new you is only 90% the same person as the previous you."
The other issue is that I don't think I quite understood what you meant when you talked about being changed. To give a framework to what I mean, I call your attention to Yvain's famous post on Wanting, Liking, and Approving. When you talked about being changed to not be queer, I assumed you meant that your Wanting, Liking, and Approving stats had all been changed. You had been changed so that you wanted to not be queer, liked it, and deeply approved of this fact.
However, this does not match your description of what you imagine the subjective experience of being modified to not be attracted to men would be like. You say:
If I woke up tomorrow morning and I was no longer sexually attracted to men, that would be startling, and it would be decidedly inconvenient in terms of my existing marriage, but I wouldn't be someone else, any more than if I stopped being sexually attracted to anyone, or stopped liking the taste of beef, or lost my arm.
That sounds to me like your Wanting and Liking stats have been modified, but your Approving stat has stayed the same.
I consider the "Approving" portion of your personality to be a much bigger part of your personal identity than "Wanting" and "Liking." So if the change left the "Approving" portion of your personality intact, I would completely agree with you that you are still the same person that you were before, regardless of what personal-identity framework I am using.
I acknowledge that life is more difficult in certain readily quantifiable ways for queer people than for straight people, but it doesn't follow that I would use a reliable therapy for making me straight if such a thing existed... and in fact I wouldn't. Nor would I encourage the development of such a therapy, particularly, and indeed the notion of anyone designing such a therapy makes me more than faintly queasy. And if it existed, I'd be reluctant to expose my children to it. And I would be sympathetic to claims that developers and promoters of such a technology are in some way acting 'against' queer folk.
I think there is a fundamental difference between being queer and being deaf. Being queer means you have different values from other people. You are attracted to different types of people than is typical. Being deaf means you have different abilities from other people. You can't hear things a typical person can. If you are struck deaf your fundamental values haven't changed. If your sexual orientation has changed, they have.
And that's not because I want the difficulties themselves; I don't. I want those differential difficulties to disappear; I just don't like the idea of having them disappear by making everyone straight. I want them to disappear by having the culture treat queers and straights in ways that don't create differential difficulties.
If you weren't queer you would have more difficulties fulfilling your values, not less. If your value is to find a mate of Type A, and you modify yourself to like Type B instead, you will be even less likely to find a mate of Type A than you were before, since your new self will be pursuing Type B people. In other words, if you weren't queer you wouldn't be better off because you wouldn't be you, you'd be somebody else.
Now, you might argue that the fact that the new you can fulfill his values more easily can compensate for this. I would disagree. I am not a negative utilitarian, but I do believe there are many circumstances where creating and fulfilling new values/people cannot fully compensate for the destruction of old values/people, even if the new values are easier to fulfill than the old ones. And I believe that changing one's sexual orientation is usually one of those circumstances.
You are, in this very post, questing and saying that your utility function PROBABLY this and that you dont think there's uncertainty about it... That is, you display uncertainty about your utility function. Check mate.
Even if I was uncertain about my utility function, you're still wrong. The factor you are forgetting about is uncertainty. With a bounded utility function infinite utility scores the same as a smaller amount of utility. So you should always assume a bounded utility function, because unbounded utility functions don't offer any more utility than bounded ones and bounded ones outperform unbounded ones in situations like Pascal's Mugging. There's really no point to believing you have an unbounded function.
I just used the same logic you did. But the difference is that I assumed a bounded utility function was the default standard for comparison, whereas you assumed, for no good reason, that the unbounded one was.
I don't know what the proper way to calculate utility when you are uncertain about your utility function. But I know darn well that doing an expected-utility calculation about what utility each function will yield and using one of the two functions that are currently in dispute to calculate that utility is a crime against logic. If you do that you're effectively assigning "having an unbounded function" a probability of 1. And 1 isn't a probability.
Your formulation of "unbounded utility function always scores infinity so it always wins" is not the correct way to compare two utility functions under uncertainty. You could just as easily say "unbounded and bounded both score the same, except in Pascal's mugging where bounded scores higher, so bounded always wins."
I think that using expected utility calculation might be valid for things like deciding whether you assign any utility at all to object or consequence. But for big meta-level questions about what your utility function even is attempting to use them is a huge violation of logic.
It seems to me that the project of transhumanism in general is actually the project of creating artificial utility monsters. If we consider a utility monster a creature that can transmute resources into results more efficiently that's essentially what a transhuman is.
In a world where all humans have severe cognitive and physical disabilities and die at the age of 30 a baseline human would be a utility monster. They would be able to achieve far more of their life goals and desires than all other humans would. Similarly, a transhuman with superhuman cognitive abilities, physical abilities, and indefinite lifespan would be a utility monster from the point of view of modern people.
So to answer the opening question about whether or not effective altruists have ever considered building artificial utility monsters: Any effective altruist who has donated any money to the SIAI, FHI, or other organization has already started doing this. We've been working towards creating artificial utility monsters for decade now.
Now, you might have been meaning something slightly different than that. Maybe you meant to create some creature with an inhuman psychology, like orgasmium. To answer that question I'd have to delve deeper and more personally into my understanding of ethics.
Long story short, I think that would be a terrible idea. My population ethics only considers the creation of entities with complex values that somewhat resemble human ones to be positive. For all other types of creatures I am a negative preference utilitarian, I consider their addition to be a bad thing and that we should make sacrifices to prevent it. And that's even assuming that it is possible to compare their utility functions with ours. I don't think interpersonal utility comparison between two human-like creatures is hard at all. But a creature with a totally alien set of values is likely impossible.
I suspect that calling your utility function itself into question like that isn't valid in terms of expected utility calculations.
I think what you're suggesting is that on top of our utility function we have some sort of meta-utility function that just says "maximize your utility function, whatever it is." That would fall into your uncertainty trap, but I don't think that is the case, I don't think we have a meta-function like that, I think we just have our utility function.
If you were allowed to cast your entire utility function into doubt you would be completely paralyzed. How do you know you don't have an unbounded utility function for paperclips? How do you know you don't have an unbounded utility function for, and assign infinite utility to, the universe being exactly the way it would be if you never made a fully rational decision again and just went around your life on autopilot? The end result is that there are a number of possible courses of action that would all generate infinity utility and no way to choose between them because infinity=infinity. The only reason your argument sounds logical is because you are allowing the questioning of the boundedness of the utility function, but not its contents.
I think that knowledge of your utility function is probably a basic, prerational thing, like deciding to use expected utility maximization and Bayesian updating in the first place. Attempting to insert your utility function itself into your calculations seems like a basic logical error.
This tends to imply the Sadistic Conclusion: that it is better to create some lives that aren't worth living than it is to create a large number of lives that are barely worth living.
I think that the Sadistic Conclusion is correct. I argue here that it is far more in line with typical human moral intuitions than the repugnant one.
There are several "impossibility" theorems that show it is impossible to come up with a way to order populations that satisfies all of a group of intuitively appealing conditions.
If you take the underlying principle of the Sadistic Conclusion, but change the concrete example to something smaller scale and less melodramatic than "Create lives not worth living to stop the addition of lives barely worth living," you will find that it is very intuitively appealing.
For instance, if you ask people if they should practice responsible family planning or spend money combating overpopulation they agree. But (if we assume that the time and money spent on these efforts could have been devoted to something more fun) this is the same principle. The only difference is that instead creating a new life not worth living we are instead subtracting an equivalent amount of utility from existing people.
It's worth noting that the question of what is a better way of evaluating such prospects is distinct from the question of how I in fact evaluate them.
Good point. What I meant was closer to "which method of evaluation does the best job of capturing how you intuitively assign value" rather than which way is better in some sort of objective sense. For me #1 seems to describe how I assign value and disvalue to repeating copies better than #2 does, but I'm far from certain.
So I think that from my point of view Omega offering to extend the length of a repeated event so it contains a more even mixture of good and bad is the same as Omega offering to not repeat a bad event and repeat a good event instead. Both options contain zero value, I would rather Omega leave me alone and let me go do new things. But they're better than him repeating a bad event.
I think I understand your viewpoint. I do have an additional question though, which is what you think about how to to evaluate moments that have a combination of good and bad.
For instance, let's suppose you have the best day ever, except that you had a mild pain in your leg for the most of the day. All the awesome stuff you did during the day more than made up for that mild pain though.
Now let's suppose you are offered the prospect of having a copy of you repeat that day exactly. We both agree that doing this would add no additional value, the question is whether it would be valueless, or add disvalue?
There are two possible ways I see to evaluate this:
You could add up all the events of the day and decide they contain more good than bad, therefore this was a "good" day. "Good" things have no value when repeated, so you would assign zero value to having a copy relive this day. You would not pay to have it happen, but you also wouldn't exert a great effort to stop it.
You could assign value to the events first before adding them up, assigning zero value to all the good things and a slight negative value to the pain in your leg. Therefore you would assign negative value to having a copy relive this day and would pay to stop it from happening.
To me (1) seems to be an intuitively better way of evaluating the prospect of a copy reliving the day than (2). It also lines up with my intuition that it wouldn't be bad news if MWI was true. But I wonder if you would think differently?
For my own part, I share your #1 and #2, don't share your #3 (that is, I'd rather Omega not reproduce the bad stuff, but if they're going to do so, it makes no real difference to me whether they reproduce the good stuff as well)
One thing that makes me inclined towards #3 is the possibility that the multiverse is constantly reproducing my life over and over again, good and bad. I do not think that I would consider it devastatingly bad news if it turns out that the Many-Worlds interpretation is correct.
If I really believed that repeated bad experiences could not ever be compensated for by repeated good ones, I would consider the Many Worlds Interpretation to be the worst news ever, since there were tons of me out in the multiverse having a mix of good and bad experiences, but the good ones "don't count" because they already happened somewhere else. But I don't consider it bad news. I don't think that if there was a machine that could stop the multiverse from splitting that I would pay to have it built.
One way to explain my preferences in this regard would be that I believe that repeated "good stuff" can compensate for repeated "bad stuff," but that it can't compensate for losing brand new "good stuff" or experiencing brand new "bad stuff."
However, I am not certain about this. There may be some other explanation for my preferences. Another possibility that I think is likely is that I think that repeated "good stuff" only loses its value for copies that have a strong causal connection to the current me. Other mes who exist somewhere out in the multiverse have no connection to this version of me whatsoever, so my positive experiences don't detract from their identical ones. But copies that I pay to have created (or to not be) are connected to me in such a fashion, so I (and they) do feel that their repeated experiences are less valuable.
This second explanation seems a strong contender as well, since I already have other moral intuitions in regards to causal connection (for instance, if there was a Matrioshka brain full of quintillions environmentalists in a part of the multiverse so far off they will never interact with us, I would not consider their preferences to be relevant when forming environmental policy, but I would consider the preferences of environmentalists here on Earth right now to be relevant). This relates to that "separability" concept we discussed a while ago.
Or maybe both of these explanations are true. I'm not sure.
Also, I'm curious, why are you indifferent in case 4? I think I might not have explained it clearly. What I was going for was that Omega say "I'm making a copy of you in a bad time of your life. I can either not do it at all, or extend the copy's lifespan so that it is now a copy of a portion of your life that had both good and bad moments. Both options cost $10." I am saying that I think I might be indifferent about what I spend $10 on in that case.
I don't see anything inconsistent about believing that a good life loses values with repetition, but a bad life does not lose disvalue. It's consistent with the Value of Boredom, which I thoroughly endorse.
Now, there's a similar question where I think my thoughts on the subject might get a little weird. Imagine you have some period of your life that started out bad, but then turned around and then became good later so that in the end that period of life was positive on the net. I have the following preferences in regards to duplicating it:
I would not pay to have a simulation that perfectly relived that portion of my life.
If Omega threatened to simulate the bad first portion of that period of life, but not the good parts that turned it around later, I would pay him not to.
If Omega threatened to simulate the bad first portion of that period of life, but not the good parts that turned it around later, I would probably pay him to extend the length of the simulation so that it also encompassed the compensating good part of that period of life.
If the cost of 2 and 3 was identical I think would probably be indifferent. I would not care whether the simulation never occurred, or if it was extended.
So it seems like I think that repeated good experiences can sometimes "make up for" repeated bad ones, at least if they occur in the same instance of simulation. But all they can do is change the value I give to the simulation from "negative" to "zero." They can't make it positive.
These preferences I have do strike me as kind of weird. But on the other hand, the whole situation is kind of weird, so maybe any preferences I have about it will end up seeming weird no matter what they are.
It seems like there's an easy way around this problem. Praise people who are responsible and financially well-off for having more kids. These traits are correlated with good genes and IQ, so it'll have the same effect.
It seems like we already do this to some extent. I hear others condemning people with who are irresponsible and low-income for having too many children fairly frequently. It's just that we fail to extend this behavior in the other direction, to praising responsible people for having children.
I'm not sure why this is. It could be for one of the reasons listed in the OP. Or it could just be because the tendency to praise and the tendency to condemn are not correlated.
I'm very unfamiliar with it, but intuitively I would have assumed that the preferences in question wouldn't be all the preferences that the agent's value system could logically be thought to imply, but rather something like the consciously held goals at some given moment
I don't think that would be the case. The main intuitive advantage negative preference utilitarianism has over negative hedonic utilitarianism is that it considers death to be a bad thing, because it results in unsatisfied preferences. If it only counted immediate consciously held goals it might consider death a good thing, since it would prevent an agent from developing additional unsatisfied preferences in the future.
However, you are probably onto something by suggesting some method of limiting which unsatisfied preferences count as negative. "What a person is thinking about at any given moment" has the problems I pointed out earlier, but another formulation could well work better.
Otherwise total preference utilitarianism would seem to reduce to negative preference utilitarianism as well, since presumably the unsatisfied preferences would always outnumber the satisfied ones.
I believe Total Preference Utilitarianism typically avoids this by regarding the creation of at most types of unsatisfied preferences as neutral rather than negative. While there are some preferences whose dissatisfaction typically counts as negative, such as the preference not to be tortured, most preference creations are neutral. I believe that under TPU, if a person spends the majority of their life not preferring to be dead then their life is considered positive no matter how many unsatisfied preferences they have.
At least I personally find it very difficult to compare experiences of such differing magnitudes.
I feel like I could try to get some sort of ballpark by figuring how much I'm willing to pay to avoid each thing. For instance, if I had an agonizing migraine I knew would last all evening, and had a choice between paying for an instant cure pill, or a device that would magically let me avoid traffic for the next two months, I'd probably put up with the migraine.
I'd be hesitant to generalize across the whole population, however, because I've noticed that I don't seem to mind pain as much as other people, but find boredom far more frustrating than average.
I guess I see a set of all possible types of sentient minds with my goal being to make the universe as nice as possible for some weighted average of the set.
I used to think that way, but it resulted in what I considered to be too many counterintuitive conclusions. The biggest one, that I absolutely refuse to accept, being that we ought to kill the entire human race and use the resources doing that would free up to replace them with creatures whose desires are easier to satisfy. Paperclip maximizers or wireheads for instance. Humans have such picky, complicated goals, after all..... I consider this conclusion roughly a trillion times more repugnant than the original Repugnant Conclusion.
Naturally, I also reject the individual form of this conclusion, which is that we should kill people who want to read great books, climb mountains, run marathons, etc. and replace them with people who just want to laze around. If I was given a choice between having an ambitious child with a good life, or an unambitious child with a great life, I would pick the ambitious one, even though the total amount of welfare in the world would be smaller for it. And as long as the unambitious child doesn't exist, never existed, and never will exist I see nothing wrong with this type of favoritism.
A bounded utility function does help matters, but then everything depends on how exactly it's bounded, and why one has chosen those particular parameters.
Yes, and that is my precise point. Even if we assume a bounded utility function for human preferences, I think it's reasonable assume that it's a pretty huge function. Which means that antinatalism/negative preference utilitarianism would be willing to inflict massive suffering on existing people to prevent the birth of one person who would have a better life than anyone on Earth has ever had up to this point, but still die with a lot of unfulfilled desires. I find this massively counter-intuitive and want to know how the antinatalist community addresses this.
I take it you mean to say that they don't spend all of their waking hours convincing other people not to have children, since it doesn't take that much effort to avoid having children yourself.
If the disutility they assign to having children is big enough they should still spend every waking hour doing something about it. What if some maniac kidnaps them and forces them to have a child? The odds of that happening are incredibly small, but they certainly aren't zero. If they really assign such a giant negative to having a child they should try to guard even against tiny possibilities like that.
Also, are they all transhumanists? For the typical person (or possibly even typical philosopher), infinite lifespans being a plausible possibility might not even occur as something that needs to be taken into account
Yes, but from a preference utilitarian standpoint it doesn't need to actually be possible to live forever. It just has to be something that you want.
Does any utilitarian system have a good answer to questions like these? If you ask a total utilitarian something like "how much morning rush-hour frustration would you be willing to inflict to people in order to prevent an hour of intense torture, and how exactly did you go about calculating the answer to that question", you're probably not going to get a very satisfying answer, either.
Well, of course I'm not expecting an exact answer. But a ballpark would be nice. Something like "no more than x, no less than y." I think, for instance, that a total utilitarian could at least say something like "no less than a thousand rush hour frustrations, no more than a million."
Speaking personally, I don't negatively weigh non-aversive sensory experiences. That is to say, the billions of years of unsatisfied preferences are only important for that small subset of humans for whom knowing about the losses causes suffering.
If I understand you correctly, the problem with doing this with negative utilitarianism is that it suggests we should painlessly kill everyone ASAP. The advantage of negative preference utilitarianism is that it avoids this because people have a preference to keep on living that killing would thwart.
It's worth pointing out that negative utilitarianism is incoherent.
Why? For the reason I pointed out, or for a different one? I'm not a negative utilitarian personally, but I think a few aspects of it have promise and would like to see them sorted out.
Not relevant because we are considering bringing these people into existence at which point they will be able to experience pain and pleasure.
Yes, but I would argue that the fact that they can't actually do that yet makes a difference.
Imagine you know that one week from now someone will force you to take heroin and you will become addicted. At this point you will be able to have an OK life if given a regular amount of the drug but will live in permanent torture if you never get any more of the substance. Would you pay $1 today for the ability to consume heroin in the future?
Yes, if I was actually going to be addicted. But it was a bad thing that I was addicted in the first place, not a good thing. What I meant when I said I "do not care in the slightest" was that the strength of that desire was not a good reason to get addicted to heroin. I didn't mean that I wouldn't try to satisfy that desire if I had no choice but to create it.
Similarly, in the case of adding lots of people with short lives, the fact that they would have desires and experience pain and pleasure if they existed is not a good reason to create them. But it is a good reason to try to help them extend their lives, and lead better ones, if you have no choice but to create them.
Thinking about it further, I realized that you were wrong in your initial assertion that "we have to introduce a fudge factor that favors people (such as us) who are or were alive." The types of "fudge factors" that are being discussed here do not, in fact do that.
To illustrate this, imagine Omega presents you with the following two choices:
Everyone who currently exists receives a small amount of additional utility. Also, in the future the amount of births in the world will vastly increase, and the lifespan and level of utility per person will vastly decrease. The end result will be the Repugnant Conclusion for all future people, but existing people will not be harmed, in fact they will benefit from it.
Everyone who currently exists loses a small amount of their utility. In the future far fewer people will be born than in Option 1, but they will live immensely long lifespans full of happiness. Total utility is somewhat smaller than in Option 1, but concentrated in a smaller amount of people.
Someone using the fudge factor Kaj proposes in the OP would choose 2, even though it harms every single existing person in order to benefit people who don't exist yet. It is not biased towards existing persons.
I basically view adding people to the world in the same light as I view adding desires to my brain. If a desire is ego-syntonic (i.e. a desire to read a particularly good book) then I want it to be added and will pay to make sure it is. If a desire is ego-dystonic (like using heroin) I want it to not be added and will pay to make sure it isn't. Similarly, if adding a person makes the world more like my ideal world (i.e. a world full of people with long eudaemonic lives) then I want that person to be added. If it makes it less like my ideal world (i.e. Repugnant Conclusion) I don't want that person to be added and will make sacrifices to stop it (for instance, I will spend money on contraceptives instead of candy).
As long as the people we are considering adding are prevented from ever having existed, I don't think they have been harmed in the same way that that discriminating against an existing person for some reason like skin color or gender harms someone, and I see nothing wrong with stopping people from being created if it makes the world more ideal.
Of course, needless to say, if we fail and these people are created anyway, we have just as much moral obligation towards them as we would towards any preexisting person.
For me, however, it doesn't seem all that far from someone saying "I'm a utilitarian but my intuition strongly tells me that people with characteristic X are more important than everyone else so I'm going to amend utilitarianism by giving greater weight to the welfare of X-men."
There is a huge difference between discriminatory favoritism, and valuing continued life over adding new people,
In discriminatory favoritism people have a property that makes them morally valuable (i.e the ability to have preferences, or to feel pleasure and pain). They also have an additional property that does not affect their morally valuable property in any significant way (i.e skin color, family relations). Discriminatory favoritism argues that this additional property means that the welfare of these people is less important, even though that additional property does not affect the morally valuable property in any way.
By contrast, in the case of valuing continuing life over creating new people, the additional property (nonexistance) that the new people have does have a significant effect on their morally significant property. Last I checked never having existed had a large effect on your ability to have preferences, and your ability to feel please and pain. If the person did exist in the past, or will exist in the future, that will change, but if they never existed, don't exist, and never will exist, then I think that is significant. Arguing that it shouldn't be is like arguing you shouldn't break a rock because "if the rock could think, it wouldn't want you to."
We can illustrate it further by thinking about individual preferences instead of people. If I become addicted to heroin I will have a huge desire to take heroin far stronger than all the desires I have now. This does not make me want to be addicted to heroin. At all. I do not care in the slightest that the heroin addicted me would have a strong desire for heroin. Because that desire does not exist and I intend to keep it that way. And I see nothing immoral about that.
Though now that you point it out, it is a problem that, under this model, creating a person who you don't expect to live forever has a very high (potentially infinite) disutility. Yeah, that breaks this suggestion. Only took a couple of hours, that's ethics for you. :)
Oddly enough, right before I noticed this thread I posted a question about this on the Stupid Questions Thread.
My question, however, was whether this problem applies to all forms of negative preferences utilitarianism. I don't know what the answer is. I wonder if SisterY or one of the other antinatalists who frequents LW does.
What amount of disutility does creating a new person generate in Negative Preference Utilitarian ethics?
I need to elaborate in order to explain exactly what question I am asking: I've been studying various forms of ethics, and when I was studying Negative Preference Utilitarianism (or anti-natalism, as I believe it's often also called) I came across what seems like a huge, titanic flaw that seems to destroy the entire system.
The flaw is this: The goal of negative preference utilitarianism is to prevent the existence of unsatisfied preferences. This means that negative preference utilitarians are opposed to having children, as doing so will create more unsatisfied preferences. And they are opposed to people dying under normal circumstances, because someone's death will prevent them from satisfying their existing preferences.
So what happens when you create someone who is going to die, and has an unbounded utility function? The amount of preferences they have is essentially infinite, does that mean that if such a person is created it is impossible to do any more harm, since an infinite amount of unsatisfied preferences have just been created? Does that mean that we should be willing to torture everyone on Earth for a thousand years if doing so will prevent the creation of such a person?
The problem doesn't go away if you assume humans have bounded utility functions. Suppose we have a bounded utility function, so living an infinite number of years, or a googolplex number of years, is equivalent to living a mere hundred billion years for us. That still means that creating someone who will live a normal 70 year lifespan is a titanic harm, a harm that everyone alive on Earth today should be willing to die to prevent it, as it would create 99,999,999,930 years worth of unsatisfied preferences!
My question is, how do negative preference utilitarians deal with this? The ones I've encountered online make an effort to avoid having children, but they don't devote every waking minute of their lives to it. And I don't think akrasia is the cause, because I've heard some of them admit that it would be acceptable to have a child if doing so reduced the preference frustration/suffering of a very large amount of existing people.
So with that introduction out of the way, my questions, on a basic level are:
How much suffering/preference frustration would an antinatalist be willing to inflict on existing people in order to prevent a birth? How much suffering/preference frustration would a birth have to stop in order for it to be justified? For simplicity's sake, let's assume the child who is born has a normal middle class life in a 1st world country with no exceptional bodily or mental health problems.
How exactly did they go about calculating the answer to question 1?
There has to be some answer to this question, there wouldn't be whole communities of anti-natalists online if their ideology could be defeated with a simple logic problem.
It is also worth noting that average utilitarianism has also its share of problems: killing off anyone with below-maximum utility is an improvement.
No it isn't. This can be demonstrated fairly simply. Imagine a population consisting of 100 people. 99 of those people have great lives, 1 of those people has a mediocre one.
At the time you are considering doing the killing the person with the mediocre life, he has accumulated 25 utility. If you let him live he will accumulate 5 more utility. The 99 people with great lives will accumulate 100 utility over the course of their lifetimes.
If you kill the guy now average utility will be 99.25. If you let him live and accumulate 5 more utility average utility will be 99.3. A small, but definite improvement.
I think the mistake you're making is that after you kill the person you divide by 99 instead of 100. But that's absurd, why would someone stop counting as part of the average just because they're dead? Once someone is added to the population they count as part of it forever.
It is also worth noting that average utilitarianism has also its share of problems: killing off anyone with below-maximum utility is an improvement.
It's true that some sort of normalization assumption is needed to compare VNM utility between agents. But that doesn't defeat utilitarianism, it just shows that you need to include a meta-moral obligation to make such an assumption (and to make sure that assumption is consistent with common human moral intuitions about how such assumptions should be made).
As it happens, I do interpersonal utility comparisons all the time in my day-to-day life using the mental capacity commonly referred to as "empathy." The normalizing assumption I seem to be making is to assume that others people's minds are similar to mine, and match their utility to mine on a one to one basis, doing tweaks as necessary if I observe that they value different things than I do.
I wonder what a CEV-implementing AI would do with such cases.
Even if it does turn out that my current conception of personal identity isn't the same as my old one, but is rather I similar concept I adopted after realizing my values were incoherent, the AI might still find that the CEVs of my past and present selves concur. This is because, if I truly did adopt a new concept of identity because of it's similarity to my old one, this suggests I possess some sort of meta-value that values taking my incoherent values and replacing them with coherent ones that are as similar as possible to the original. If this is the case the AI would extrapolate that meta-value and give me a nice new coherent sense of personal identity, like the one I currently possess.
Of course, if I am right and my current conception of personal identity is based on my simply figuring out what I meant all along by "identity," then the AI would just extrapolate that.
Granted, negative utilitarians would prefer to add a small population of beings with terrible lives over a very large beings with lives that are almost ideal, but this would not be a proper instance of the Sadistic Conclusion. See the formulation:
When I read the formulation of the Sadistic Conclusion I interpreted "people with positive utility" to mean either a person whose life contained no suffering, or a person whose satisfied preferences/happiness outweighed their suffering. So I would consider adding a small population of terrible lives instead of a large population of almost ideal lives to be the Sadistic Conclusion.
If I understand you correctly, you are saying that negative utilitarianism technically avoids the Sadistic Conclusion because it considers a life with any suffering at all to be a life of negative utility, regardless of how many positive things that life also contains. In other words, it avoid the SC because it's criterion for what makes a life positive and negative are different than the criterion Arrenhius used when he first formulated the SC. I suppose that is true. However, NU does not avoid the (allegedly) unpleasant scenario Arrenhius wanted to avoid (adding a tortured life instead of a large amount of very positive lives).
Negative utilitarians try to minimize the total amount of preference-frustrations, or suffering....(Also note that being killed is only a problem if you have a preference to go on living, and that even then, it might not be the thing considered worst that could happen to someone.)
Right, but if someone has a preference to live forever does that mean that infinite harm has been done if they die? In which case you might as well do whatever afterwards, since infinite harm has already occurred? Should you torture everyone on Earth for decades to prevent such a person from being added? That seems weird.
The best solution I can currently think of is to compare different alternatives, rather than try to measure things in absolute terms. So if a person who would have lived to 80 dies at 75 that generates 5 years of unsatisfied preferences, not infinity, even if the person would have preferred to live forever. But that doesn't solve the problem of adding people who wouldn't have existed otherwise.
What I'm trying to say is, people have an awful lot of preferences, and generally only manage to satisfy a small fraction of them before they die. So how many unsatisfied preferences should adding a new person count as creating? How big a disutility is it compared to other disutilities, like thwarting existing preferences and inflicting pain on people.
A couple possibilities that occurs to me off the top of my head. One would be to find the difference in satisfaction between the new people and the old people, and then compare it to the difference in satisfaction between the old people and the counter-factual old people in the universe where the new people were never added.
Another possibility would be to set some sort of critical level based on what the maximum level of utility it is possible to give the new people given our society's current level of resources, without inflicting greater disutilities on others than you give utility to the new people. Then weigh the difference between the new peoples actual utility and their "critical possible utility" and compare that to the dissatisfaction the existing people would suffer if the new people are not added.
Do either of these possibilities sound plausible to you, or do you have another idea?