Upgrading moral theories to include complex values

I do expect it to change in the far future as the human race (barring some extinction event) expands into space.

I am also a little skeptical of one of the author's premises, I would not give up a significant portion of my lifespan (probably less than a week at most) to avoid a painful, but relatively brief death. I am concerned about the suffering wild animals feel in their day-to-day life, but I don't think any painful deaths they experience are as significant as the author implies. I'm not expert enough to know how frequent predator encounters, starvation and other such things are among animals to know whether the average of their day-to-day life is mostly pain, I'm guessing it's closer to neutral, but I can't be sure.

I have also read some studies that suggest fear may be much more harmful than pain to animals, I have no idea what that implies.

Then there's this, although I wouldn't take it seriously at all, and neither does the author.

Another weird idea I don't think anyone has considered before, what about the wants of animals, are they significant at all? It's well known that humans can want things that do not give them pleasure (i.e. not wanting to be told a comforting lie). It seems like that is true of animals as well. If I knock out the part of a rat's brain that likes food, and it still tries to get food (because it wants it) am I morally obligated to give it food?

Generally when I want things I don't enjoy I can divide those wants into ego-syntonic wants that I consider part of my "true self" (i.e. wanting to be told the truth, even if it's upsetting) versus ego-dystonic wants that I consider an encroachment on my true self I want to eliminate (like wanting to eat yet another potatoe chip). Since animals are not sapient, and so lack any reflective "true self" does that mean none of their wants matter, or all of them? If an animal gets what it wants does that make up for pain it has experienced, or not?

Still, you make a good point, maybe I should revise my opinion on this.

Thanks so much! I can't believe I forgot about that. Fixed.

Allowing yourself to be swayed by thought experiments of the form "If you value x at all, then if offered enough x, you would give up y, and you value one y so much more than one x!" is a recipe for letting scope insensitivity rewrite your utility function.

Be careful about making an emotional judgement on numbers bigger than you can intuitively grasp. If possible, scale down the numbers involved (Instead of 1 million x vs 1 thousand y, imagine 1 thousand x vs 1 y) before imagining each alternative as viscerally as possible.

In my own experience, not estimating a number, just thinking "Well, it's really big" basically guarantees I will not have the proper emotional response.

You would only create these viruses if the total utility of the viruses you can create with the resources at your disposal exceeds the utility of the humans you could make with these same resources. For instance, if you give a utility of 1 to a steel paperclip weighing 1 gram, then assuming a simple additive model (which I wouldn't, but that's besides the point) making one metric ton of paperclips has an utility of 1,000,000. If you give an utility of 1,000,000,000 to a steel sculpture weighing a ton, it follows that you will never make any paperclips unless you have less than a ton of iron. You will always make the sculpture, because it gives 1,000 times the utility for the exact same resources.

I have tried to get around this by creating a two step process. When considering whether or not to create a creature the first step is asking "Does it have humane values?" The second step is asking "Will it live a good life, without excessively harming others in the process?" If the answer to either of those questions is "no," then it is not good to create such a creature.

Now, it doesn't quite end there. If the benefits to others are sufficiently large then the goodness of creating a creature that fails the process may outweigh the badness of creating it. Creating an animal without humane values may still be good if such a creature provides companionship or service to a human, and the value of that outweighs the cost of caring for it. However, once such a creature is created we have a responsibility to include it in utility calculations along with everyone else. We have to make sure it lives a good life, unless there's some other highly pressing concern that outweighs it in our utility calculations.

Now, obviously creating a person with humane values who will not live a good life may still be a good thing as well, if they invent a new vaccine or something like that.

I think this process can avoid mandating we create viruses and mice, while also preserving our intuition that torturing cats is bad.

If you assign any positive utility at all, no matter how small, to creating happy low-complexity life, you end up having to create lots of happy viruses (they are happy when they can replicate).

You are using a rather expansive definition of "happy." I consider happiness to be a certain mental process that only occurs in the brains of sufficiently complex creatures. I consider it to not be synonymous with utility, which includes both happiness, and other desires people have.

You're right to say that treating actual people that way seems pretty unpleasant. But the examples I gave involved creating new people and animals, not differentially treating existing people and animals in some fashion.

I don't see refusing to create a new person because of a certain set of priorities as morally equivalent to disrespectfully treating existing persons because of that set of priorities. If you say "I care about you, but I don't really CARE, you know?" to a person you've decided to not create, who are you talking to? Who have you been rude to? Who have you been offensive to? Nobody, that's who.

I agree with you that we have a duty to treat people with care and respect in real life. That's precisely why I oppose creating new people in certain circumstances. Because once they're created there is no turning back, you have to show them that respect and care, and you have to be genuine about it, even if doing so messes up your other priorities. I want to make sure that my duty to others remains compatible with my other priorities. And I don't see anything wrong with creating slightly less people in order to do so.

Or to put it another way, it seems like I have a switch that, when flipped, makes me go from considering a person in the sort of cynical way you described to considering them a fellow person that I love and care for and respect with all my heart. And what flips that switch from "cynical" to "loving and compassionate" is an affirmative answer to the question "Does this person actually exist, or are they certain to exist in the future?" I don't see this as a moral failing. Nonexistant people don't mind if you don't love or respect them.

Although the reason it may be hard to justify is that I have to explain to others who are somewhat distant, "Well, I care about you, but I don't really CARE care, you know? Because of X, Y, and Z you see. You understand." That's not really something that's polite to say to someone's face, so if you say in public "Yeah, I don't really care about everyone who isn't my friends and family, I just want to be sufficiently nice to all of you that I don't get hurt out of spite or guilt." That's offensive.

I see it as an unfortunate fact of limited resources. Caring about the entire world in enough detail is impossible because each of us only has a few neurons for every other person on Earth. Until we can engineer ourselves for more caring we will have to be satisfied with putting most other people into classes such that we care about the class as a whole and leave individual caring up to other individuals in that class. Being sufficiently nice to any particular individual of a class is probably the most you can rationally do for them barring extenuating evidence. If they are obviously out of the norm for their class (starving, injured, in danger, etc.) and you can immediately help them significantly more efficiently than another member of the class then you should probably spend more caring on them. This avoids the bystander effect by allowing you to only care about overall accidents and injuries in the class in general but also care specifically about an individual if you happen to be in a good position to help.

Otherwise try to maximize the utility of the class as a whole with regard to your ability to efficiently affect their utility, weighted against your other classes and individuals appropriately.

By the way, 3^^3 = 3^27 is "only" 7625597484987, which is less than a quadrillion. If you want a really big number, you should add a third arrow (or use a higher number than three).

1) 100 A 2) 1H + 100A - 50A 3) 0 4) 1H - 50A

I wouldn't actually condone the move from 1 to 2. I would not condone inflicting huge harms on animals to create a moderately well-off human. But I would condone never creating some happy animals in the first place. Not creating is not the same as harming. The fact that TU treats not creating an animal with 50 utility to be equivalent to inflicting 50 points of disutility on an animal that has 50 utility is a strike against it. If we create an animal we have a responsibility to make it happy, if we don't we're free to make satisfied humans instead (to make things simpler I'm leaving the option of painlessly killing the animals off the table).

My argument is existing animals and existing humans have equal moral significance (or maybe humans have somewhat more if you're actually right about humans being capable of greaters level of welfare), but when deciding which to create, human creation is superior to animal (or paperclipper or wirehead) creation.

FWIW, I think the best fix is empirical: just take humans to be capable of much greater positive welfare states than animals, such that you have to make really big trades of humans to animals to be worth it. Although these leaves open the possibility we'd end up with little complex value, it does not seem a likely possibliity

I lack your faith in this fix. I consider it almost certain that if we were to create a utilitarian AI it would kill the entire human race and replace it with creatures whose preferences are easier to satisfy. And by "easier to satisfy" I mean "simpler and less ambitious," not that the creatures are more mentally and physically capable of satisfying humane desires.

In addition to this, total utilitarianism has several other problems that need fixing, the most obvious being that it considers it a morally good act to kill someone destined to live a good long life and replace them with a new person whose overall utility is slightly higher than the utility of the remaining years of that person's life would have been (I know in practice doing this is impossible without side-effects that create large dis-utilities, but that shouldn't matter, it's bad because it's bad, not because it has bad side-effects).

Quite frankly I consider the Repugnant Conclusion to be far less horrible than the "Kill and Replace" conclusion. I'd probably accept total utilitarianism as a good moral system if the RC was all it implied. The fact that TU implies something strange like the RC in an extremely unrealistic toy scenario isn't that big a deal. But that's not all TU implies. The "Kill and Replace" conclusion isn't a toy scenario, there are tons of handicapped people that could be killed and replaced with able people right now. But we don't do that, because it's wrong.

I don't spend my days cursing the fact that various disutilitous side effects prevent me from killing handicapped people and creating able people to replace them. Individual people, once they are created, matter. Once someone has been brought into existence we have a greater duty to make sure they stay alive and happy then we do to create new people. There may be some vastly huge amount of happy people that it's okay to kill one slightly-less-happy-person in order to create, but that number should be way, way, way, way, bigger than 1.

As a second recommendation (with pre-emptive apologies for offense: I can't find a better way of saying this), I'd recommend going back to the population ethics literature (and philosophy generally) rather than trying to reconstitute ethical theory yourself.

I've looked at some of the literature, and I've noticed there does not seem to be much interest in the main question I am interested in, which is, "Why make humans and not something else?" Peter Singer mentions it a few times in one essay I read, but didn't offer any answers, he just seemed to accept it as obvious. I thought it was a fallow field that I might be able to do some actual work in.

Peter Singer also has the decency to argue that human beings are not replaceable, that killing one person to replace them with a slightly happier one is bad. But I have trouble seeing how his arguments work against total utilitarianism.