Would a halfway copied brain emulation be at risk of having different values/identity? 2020-07-30T05:43:30.772Z
Upgrading moral theories to include complex values 2013-03-27T18:28:46.983Z
Population Ethics Shouldn't Be About Maximizing Utility 2013-03-18T02:35:56.727Z
Desires You're Not Thinking About at the Moment 2013-02-20T09:41:09.182Z
Some scary life extension dilemmas 2013-01-01T18:41:21.421Z
Dying in Many Worlds 2012-12-21T11:48:17.319Z
Is Equality Really about Diminishing Marginal Utility? 2012-12-04T23:03:31.297Z
The Mere Cable Channel Addition Paradox 2012-07-26T07:20:05.081Z
The Creating Bob the Jerk problem. Is it a real problem in decision theory? 2012-06-12T21:36:43.668Z
Alan Carter on the Complexity of Value 2012-05-10T07:23:07.227Z


Comment by Ghatanathoah on Deminatalist Total Utilitarianism · 2021-09-19T21:17:52.433Z · LW · GW

If someone is in a rut and could either commit suicide or take the reprogramming drug (and expects to have to take it four times before randomizing to a personality that is better than rerolling a new one), why is that worse than killing them and allowing a new human to be created?

If such a drug is so powerful that the new personality is essentially a new person, then you have created a new person whose lifespan will be a normal human lifespan minus however long the original person lived before they got in a rut.  By contrast, if they commit suicide and you create a new human, you have created a new person who will likely live a normal human lifespan.  So taking the drug even once is clearly worse than suicide + replacement since, all else being equal, it is better to create a someone with a longer lifespan than a shorter one (assuming their lifespan is positive, overall, of course).

Comment by Ghatanathoah on Example population ethics: ordered discounted utility · 2021-09-15T00:39:06.369Z · LW · GW

So if they don't want to be killed, that counts as a negative if we do that, even if we replace them with someone happier.

I have that idea as my "line of retreat."  My issue with it is that it is hard to calibrate it so that it leaves as big a birth-death asymmetry as I want without degenerating into full-blown anti-natalism. There needs to be some way to say that the new happy person's happiness can't compensate for the original person's death without saying that the original person's own happiness can't compensate for their own death, which is hard.  If I calibrate it to avoid anti-natalism it becomes such a small negative that it seems like it could easily be overcome by adding more people with only a little more welfare.

There's also the two step "kill and replace" method, where in step one you add a new life barely worth living without affecting anyone.   Since the new person exists now, they count the same as everyone else, so then in the second step you kill someone and transfer their resources to the new person. If this process gives the new person the same amount of utility as the old one, it seems neutral under total utilitarianism. I suppose under total preference utilitarianism its somewhat worse, since you now have two people dying with unsatisfied preferences instead of one, but it doesn't seem like a big enough asymmetry for me.

I feel like in order to reject the two step process, and to have as big an asymmetry as I want, I need to be able to reject "mere addition" and accept the Sadistic Conclusion. But that in turn leads to "galaxy far far away issues" where it becomes wrong to have children because of happy people in some far off place. Or "Egyptology" issues where its better for the world to end than for it to decline so future people have somewhat worse lives, and we are obligated to make sure the Ancient Egyptians didn't have way better lives than ours before we decide on having children.  I just don't know. I want it to stop hurting my brain so badly, but I keep worrying about how there's no solution that isn't horrible or ridiculous. 

This has degenerate solutions too - it incentivises producing beings that are very easy to satisfy and that don't mind being killed. 

For this one, I am just willing to just decree that creating creatures with a diverse variety of complex human-like psychologies is good, and creating  creatures with weird minmaxing unambitious creatures is bad (or at least massively sub-optimal). To put it another way, Human Nature is morally valuable and needs to be protected.

Another resource that helped me on this was Derek Parfit's essay "What Makes Someone's Life Go Best."  You might find it helpful, it parallels some of your own work on personal identity and preferences. The essay describes which of our preferences we feel count as part of our "self interest" and which do not. It helped me understand things, like why people general feel obligated to respect people's "self interest" preferences (i.e. being happy, not dying), but not their "moral preferences" (i.e. making the country a theocracy, executing heretics).  

Parfit's "Success Theory," as he calls it, basically argues that only preferences that are "about your own life" count as "welfare" or "self interest."  So that means that we would not be making the world a better place by adding lives who prefer that the speed of light stay constant, or that electrons keep having negative charges. That doesn't defuse the problem entirely, you could still imagine creating creatures with super unambitious life goals. But it gets it part of the way, the rest, again, I deal with by "defending Human Nature."

I'm more wanting to automate the construction of values from human preferences

I had a question about that. It is probably a silly question since my understanding of decision and game theory is poor. When you were working on that you said that there was no independence of irrelevant alternatives.  I've noticed that IIA is something that trips me up a lot when I think about population ethics.  I want to be able to say something like "Adding more lives might be bad if there is still the option to improve existing ones instead, but might be good if the existing ones have already died and that option is foreclosed." This violates IIA because I am conditioning whether adding more lives is good on whether there is another alternative or not.  

I was wondering if my brain might be doing the thing you described in your post on no IIA, where it is smashing two different values together and getting different results if there are more alternatives. It probably isn't I am probably just being irrational, but reading that post just felt familiar.

Comment by Ghatanathoah on Example population ethics: ordered discounted utility · 2021-09-11T02:10:18.553Z · LW · GW

You can always zero out those utilities by decree, and only consider utilities that you can change. There are other patches you can apply. By talking this way, I'm revealing the principle I'm most willing to sacrifice: elegance.

It's been a long time since you posted this, but if you see my comment, I'd be curious about what some others patches one could apply are.  I have pretty severe scrupulosity issues around population ethics and often have trouble functioning because I can't stop thinking about them.  I dislike pure total utilitarianism, but I have trouble rejecting it precisely because of "galaxy far far away" type issues.  I spend a lot of time worrying about the idea that I am forced to choose between two alternatives: 

1) That (to paraphrase what you said in your critique of total utilitarianism) it is a morally neutral act to kill someone if you replace them with someone whose lifetime utility is equal to the first person's remaining lifetime utility (and on a larger scale, the Repugnant Conclusion), or

2.That the human race might be obligated to go extinct if it turns out there is some utopia in some other branch of the multiverse, or the Andromeda Galaxy, or in some ancient, undiscovered fallen civilization in the past. Or that if the Earth was going to explode and I could press a button to save it, but it would result in future generations living slightly lower quality lives than present generations, I shouldn't push the button.

I'd really like to know some ways that I can reject both 1 and 2. I really admire your work on population ethics and find that your thinking on the subject is really closely aligned with my own, except that you're better at it than me :)

Comment by Ghatanathoah on Would a halfway copied brain emulation be at risk of having different values/identity? · 2020-07-30T23:06:24.454Z · LW · GW
You can get mind states that are ambiguous mixes of awake and asleep.

I am having trouble parsing this statement. Does it mean that when simulating a mind you could also simulate ambiguous awake/asleep in addition to simulating sleep and wakefulness? Or does it mean that a stored, unsimulated mind is ambiguously neither awake or asleep?

Comment by Ghatanathoah on Would a halfway copied brain emulation be at risk of having different values/identity? · 2020-07-30T16:15:26.976Z · LW · GW

That makes a lot of sense, thank you.

Comment by Ghatanathoah on Would a halfway copied brain emulation be at risk of having different values/identity? · 2020-07-30T16:15:06.116Z · LW · GW

Thanks for the reply. It sounds like maybe my mistake was assuming that unsimulated brain data was functionally and morally equivalent to an unconscious brain. From what you are saying it sounds like the data would need to be simulated even to generate unconsciousness.

Comment by Ghatanathoah on Disability Culture Meets the Transhumanist Condition · 2015-09-16T19:35:18.439Z · LW · GW

And much like Vaniver below (above? earlier!), I am unsure how to translate these sorts of claims into anything testable

One thing I consider very suspicious is that deaf people often don't just deny the terminal value of hearing. They also deny its instrumental value. The instrumental values of hearing are obvious. This indicates to me that they are denying it for self-esteem reasons and group loyalty reasons, the same way I have occasionally heard multiculturalists claim behaviors of obvious instrumental value (like being on time) are merely the subjective values of Western culture.

The typical defense of this denial (and other disability-rights type claims) is hearing only has instrumental value because society is structured in a way that makes use of it. But this is obviously false, hearing would be useful on a desert island, and there are some disabilities that society is not technologically capable of solving (there's no way to translate instrumental music into sign language). Plus, structuring society around disabilities is essentially having society pay to enable a person instead of having biology do it for free. Obviously it's better than not accommodating them, but it;s even better to have biology do the accommodation for free if that is possible.

I think another factor is simply my knowledge of the human brain structure, and the psychological unity of humankind. It seems like it would be a much smaller departure from standard brain design to switch the specific target of the "romance" module of the brain, than it would be to completely erase all desire to enjoy the pleasures that a sense of hearing can provide us, and to assign terminal value to being inconvenienced by things like not being able to talk to people who aren't in your visual range.

I think another thing that supports my intuitions is Bostrom's Reversal test. Imagine instead of discussing giving a preexisting sense to people who lack it, we were considering giving people a new sense that no human being has ever had before. Should we do that? If there were no side effects, I would say yes! As I told Vaniver in my reply to them, I really want to be able sense magnetic fields. Seeing infrared and ultraviolet would also be fun. The fact that my intuitions are the same in the Reversal Test provides evidence that they are not based on the Status Quo Bias.

Comment by Ghatanathoah on Disability Culture Meets the Transhumanist Condition · 2015-09-16T19:23:09.203Z · LW · GW

it may be clearer to consider counterfactual mes of every possible sexual orientation, and comparing the justifications they can come up with for why it's egosyntonic to have the orientation that they have.

I think that maybe all of them would be perfectly justifying in saying that their sexual orientation is a terminal value and the buck stops there.

On the other hand, I'm nowhere near 100% sure I wouldn't take a pill to make me bisexual.

If you kept all of my values the same and deleted my sexual orientation, what would regrow?

I think a way to help tease out your intuition would be Bostrom's reversal test. If transhumanist scientists invented a new kind of sexual orientation, a new kind of sexual experiences, and so on, would you want to be modified to be able to enjoy this new, never before seen type of sex. I don't know how you'd reply, for me it would probably be yes or no, depending on specific details of the new kind of sex.

I think the reason I would sometimes say yes is that I have a strong preexisting preference for novelty and novel experiences. So my desire for new sexual preferences would grow out of that.

Incidentally, Bostrom's reversal test also supports my intuitions about deafness. If transhumanists invented new senses that no human has ever had before, would I want to have them if there were no side effects? Of course I would! I especially want to be able to sense magnetic fields like sharks can.

Comment by Ghatanathoah on Disability Culture Meets the Transhumanist Condition · 2015-09-16T18:44:31.540Z · LW · GW

It seems to me that most people lack the ability to be aroused by people--typically, their ability is seriously limited, to half of the population at most.

When I was talking about being queer I wasn't just talking about the experience of being aroused, I was talking about the desire to have that experience, and that experience being egosyntonic. It's fairly easy to rephrase any preference a person has to sound like an ability or lack thereof. For instance, you could say that I lack the ability to enjoy skinning people alive. But that's because I don't want to skin people alive, or to enjoy it! That's a terminal value, the buck stops there.

Some other factors to consider:

  • Even if I was to define "being aroused" as an ability, that doesn't perfectly map onto the discuss. In the case of removing deafness we are adding an ability. In the case of changing queerness to heterosexuality, we are either removing an ability to find some people arousing and replacing it with a different one (in the case of homosexuals) or removing an ability and replacing it with nothing (in the case of bisexuals).
  • Arousal has very little instrumental value compared to hearing. Even if someone with the power of hearing took no pleasure from music or people's voices they would still benefit from being able to hear people talk outside of their visual range, and to hear cars coming when they cross the street. I can see deaf people denying the terminal benefits of hearing, but denying the instrumental ones seems obviously crazy.

I can't claim that I would choose to be gay for that reason, starting from emptiness.

Starting from emptiness you would be completely indifferent to everything, including changes to your future preferences. To paraphrase Eliezer, you would be a rock, not a perfectly impartial being.

At some point you just have to say "These are my terminal values, the buck stops here."

Now, while I would not say it is impossible to create a creature that assigns a terminal value to deafness, I find it unlikely that humans are such creatures. The way human psychology works makes me assign a much higher probability to their being self-deceived for group status purposes.

Comment by Ghatanathoah on Disability Culture Meets the Transhumanist Condition · 2015-09-16T14:43:08.001Z · LW · GW

But since then, you've concluded that being queer isn't actually something (at least some people, like me) differentially approve of.

I'm not sure what I wrote that gave you this idea. I do think that queer people approve of being queer. What I'm talking about when I say "approval" is preferences that are ego-syntonic, that are line with the kind of person they want to be. Most queer people consider their preference to be ego-syntonic. Being queer is the kind of person they want to be and they would not change it if they could. Those who do not are usually motivated by mistaken religious ideas, rather than clearly reasoned disapproval.

What I am trying to say is that being queer is a statement about what people want to do. When we say that someone is queer that means that they have a desire to engage in romantic and sexual relationships that are different from the heterosexual norm. This desire is ego-syntonic, it is approved of.

Being deaf, by contrast, is a statement about what people are able do. They lack the ability to hear things.

If you removed someone's deafness, none of their desires would change. They would still want everything they wanted before they were deaf. If they were really attached to their current lifestyle they could buy earplugs. By contrast, if you changed a queer person into a straight person, they would stop wanting to have non-heteronormative relationships. They'd be able to continue their current lifestyle (or at least, as able as anyone is in a heteronormative society), but they wouldn't want to.

There are some people who claim that they prefer being deaf to being able to hear, and that being deaf is ego-syntonic. I believe that they are confused. I think what they really value isn't being deaf, it's the community that they have built with other deaf people. They are confusing their preference to display loyalty to their community with with a preference to not be able to hear. In addition I think they are confused for some other reasons:

  • Sour grapes. When people are unable to do something, they often convince themselves they didn't want to do it anyway in order to assuage their ego.
  • Confusing "life could be better" with "life is not worth living." As I said before, a lot of disability rights advocates seem to think that if you admit that their disability makes their life even slightly worse, that means their life is not worth living at all and they should be euthanized. This is not true.
  • If people got hit in the head with a baseball bat every day.....
  • Happy death spirals around their community. They love their community and want to say more and more good things about it. So they say that their community is so awesome that living in it is worth being significantly less good at sensing one's surroundings.

To sum it up, I believe that being queer is an ego-syntonic desire. I believe that being deaf is not ego-syntonic, but people say it is out of a desire to have self-esteem and be proud of and loyal to the deaf community.

Comment by Ghatanathoah on Disability Culture Meets the Transhumanist Condition · 2014-12-11T19:16:20.985Z · LW · GW

Rereading your original comment keeping in mind that you're talking mostly about approval rather than desire or preference... so, would you say that Deaf people necessarily disapprove of deafness?

I'd say that a good portion of them do approve of it. There seem to be a lot of disability rights activists who seem to think that being disabled and making more disabled people is okay.

I should also mention, however, that I do think it is possible to mistakenly approve or disapprove of something. For instance I used to disapprove of pornography and voluntary prostitution. However, I eventually realized that the arguments for why those things were bad were wrong/incoherent, and realized that I should never have disapproved of those things. Disapproval of pornography and voluntary prostitution was never my CEV.

I think a large portion of disability-rights activists are also confused in their thinking, and would have different views if their thinking was clearer. For instance, many disability rights activists seem to think that any suggestion that disability is bad implies that the lives of disabled people aren't worth living and that they should all be involuntarily euthanized, which is obviously false. It's possible to believe your life is worth living while simultaneously believing it could be better.

Comment by Ghatanathoah on Disability Culture Meets the Transhumanist Condition · 2014-12-11T18:18:37.959Z · LW · GW

It's not clear to me how this difference justifies the distinction in my thinking I was describing.

I believe the difference is that in the case of deaf people, you are improving their lives by giving them more abilities to achieve the values they have (in this case, an extra sense). By contrast, with queerness you are erasing a value a person has and replacing it with a different value that is easier to achieve. I believe that helping a person achieve their existing values is a laudable goal, but that changing a person's values is usually morally problematic, even if their new values are easier to achieve than their old ones.

Now, keep in mind that I am speaking in principle, not in practice. In the real-life case of deafness this issue is more complicated than the way I just described it. There are other issues, for instance, the value of an extra sense is to some extent tied to the support mechanisms society has developed for it. I think that the deaf community may be voicing a valid concern that society has good set of support mechanism for people who are fully deaf and fully hearing, but not as good mechanisms for people with the kind of mid-range hearing that cochlear implants provide.

But those are concerns of practice, not principle. In principle having extra senses should make it easier to achieve your values. I mean, wouldn't you want super-hearing, microscopic vision, etc if you could get them without any side-effects.

How do we tell whether what I value is to find a mate of Type A, or to find a mate I find attractive?

I think the fact that you are unwilling to have your criteria for attractiveness be modified is good evidence that it is the former and not the latter.

Is this simply a semantic disagreement -- that is, do we just have different understandings of what the phrase "who I am" refers to? Or is there something we'd expect to observe differently in the world were you correct and I mistaken about this?

I think there are two issues, one is semantic, the other is that I did completely understand what you meant by being changed into someone who isn't queer.

First, the semantic issue. I have been trying to approach the issue of Personal Identity by righting a wrong question. Instead of asking "Am I the same person as him?" I instead ask "How desirable would it be for me to change into that person?" I find that this approache generates the same intuitive results as traditional approaches to personal identity (for instance both approaches identify being killed and being wireheaded as very undesirable outcomes) but doesn't get bogged down by the issues of what exactly it means to be "the same."

Saying that you literally wouldn't be the same person was hyperbolic of me. I was trying to draw attention to the fact that our values are an important part of who we are, and that changing our values can change our identity. It would be more accurate to say something like "the new you is only 90% the same person as the previous you."

The other issue is that I don't think I quite understood what you meant when you talked about being changed. To give a framework to what I mean, I call your attention to Yvain's famous post on Wanting, Liking, and Approving. When you talked about being changed to not be queer, I assumed you meant that your Wanting, Liking, and Approving stats had all been changed. You had been changed so that you wanted to not be queer, liked it, and deeply approved of this fact.

However, this does not match your description of what you imagine the subjective experience of being modified to not be attracted to men would be like. You say:

If I woke up tomorrow morning and I was no longer sexually attracted to men, that would be startling, and it would be decidedly inconvenient in terms of my existing marriage, but I wouldn't be someone else, any more than if I stopped being sexually attracted to anyone, or stopped liking the taste of beef, or lost my arm.

That sounds to me like your Wanting and Liking stats have been modified, but your Approving stat has stayed the same.

I consider the "Approving" portion of your personality to be a much bigger part of your personal identity than "Wanting" and "Liking." So if the change left the "Approving" portion of your personality intact, I would completely agree with you that you are still the same person that you were before, regardless of what personal-identity framework I am using.

Comment by Ghatanathoah on Disability Culture Meets the Transhumanist Condition · 2014-12-11T06:56:32.384Z · LW · GW

I acknowledge that life is more difficult in certain readily quantifiable ways for queer people than for straight people, but it doesn't follow that I would use a reliable therapy for making me straight if such a thing existed... and in fact I wouldn't. Nor would I encourage the development of such a therapy, particularly, and indeed the notion of anyone designing such a therapy makes me more than faintly queasy. And if it existed, I'd be reluctant to expose my children to it. And I would be sympathetic to claims that developers and promoters of such a technology are in some way acting 'against' queer folk.

I think there is a fundamental difference between being queer and being deaf. Being queer means you have different values from other people. You are attracted to different types of people than is typical. Being deaf means you have different abilities from other people. You can't hear things a typical person can. If you are struck deaf your fundamental values haven't changed. If your sexual orientation has changed, they have.

And that's not because I want the difficulties themselves; I don't. I want those differential difficulties to disappear; I just don't like the idea of having them disappear by making everyone straight. I want them to disappear by having the culture treat queers and straights in ways that don't create differential difficulties.

If you weren't queer you would have more difficulties fulfilling your values, not less. If your value is to find a mate of Type A, and you modify yourself to like Type B instead, you will be even less likely to find a mate of Type A than you were before, since your new self will be pursuing Type B people. In other words, if you weren't queer you wouldn't be better off because you wouldn't be you, you'd be somebody else.

Now, you might argue that the fact that the new you can fulfill his values more easily can compensate for this. I would disagree. I am not a negative utilitarian, but I do believe there are many circumstances where creating and fulfilling new values/people cannot fully compensate for the destruction of old values/people, even if the new values are easier to fulfill than the old ones. And I believe that changing one's sexual orientation is usually one of those circumstances.

Comment by Ghatanathoah on St. Petersburg Mugging Implies You Have Bounded Utility · 2014-07-14T19:11:53.304Z · LW · GW

You are, in this very post, questing and saying that your utility function PROBABLY this and that you dont think there's uncertainty about it... That is, you display uncertainty about your utility function. Check mate.

Even if I was uncertain about my utility function, you're still wrong. The factor you are forgetting about is uncertainty. With a bounded utility function infinite utility scores the same as a smaller amount of utility. So you should always assume a bounded utility function, because unbounded utility functions don't offer any more utility than bounded ones and bounded ones outperform unbounded ones in situations like Pascal's Mugging. There's really no point to believing you have an unbounded function.

I just used the same logic you did. But the difference is that I assumed a bounded utility function was the default standard for comparison, whereas you assumed, for no good reason, that the unbounded one was.

I don't know what the proper way to calculate utility when you are uncertain about your utility function. But I know darn well that doing an expected-utility calculation about what utility each function will yield and using one of the two functions that are currently in dispute to calculate that utility is a crime against logic. If you do that you're effectively assigning "having an unbounded function" a probability of 1. And 1 isn't a probability.

Your formulation of "unbounded utility function always scores infinity so it always wins" is not the correct way to compare two utility functions under uncertainty. You could just as easily say "unbounded and bounded both score the same, except in Pascal's mugging where bounded scores higher, so bounded always wins."

I think that using expected utility calculation might be valid for things like deciding whether you assign any utility at all to object or consequence. But for big meta-level questions about what your utility function even is attempting to use them is a huge violation of logic.

Comment by Ghatanathoah on Artificial Utility Monsters as Effective Altruism · 2014-07-14T18:37:13.431Z · LW · GW

It seems to me that the project of transhumanism in general is actually the project of creating artificial utility monsters. If we consider a utility monster a creature that can transmute resources into results more efficiently that's essentially what a transhuman is.

In a world where all humans have severe cognitive and physical disabilities and die at the age of 30 a baseline human would be a utility monster. They would be able to achieve far more of their life goals and desires than all other humans would. Similarly, a transhuman with superhuman cognitive abilities, physical abilities, and indefinite lifespan would be a utility monster from the point of view of modern people.

So to answer the opening question about whether or not effective altruists have ever considered building artificial utility monsters: Any effective altruist who has donated any money to the SIAI, FHI, or other organization has already started doing this. We've been working towards creating artificial utility monsters for decade now.

Now, you might have been meaning something slightly different than that. Maybe you meant to create some creature with an inhuman psychology, like orgasmium. To answer that question I'd have to delve deeper and more personally into my understanding of ethics.

Long story short, I think that would be a terrible idea. My population ethics only considers the creation of entities with complex values that somewhat resemble human ones to be positive. For all other types of creatures I am a negative preference utilitarian, I consider their addition to be a bad thing and that we should make sacrifices to prevent it. And that's even assuming that it is possible to compare their utility functions with ours. I don't think interpersonal utility comparison between two human-like creatures is hard at all. But a creature with a totally alien set of values is likely impossible.

Comment by Ghatanathoah on St. Petersburg Mugging Implies You Have Bounded Utility · 2014-05-29T04:48:05.224Z · LW · GW

I suspect that calling your utility function itself into question like that isn't valid in terms of expected utility calculations.

I think what you're suggesting is that on top of our utility function we have some sort of meta-utility function that just says "maximize your utility function, whatever it is." That would fall into your uncertainty trap, but I don't think that is the case, I don't think we have a meta-function like that, I think we just have our utility function.

If you were allowed to cast your entire utility function into doubt you would be completely paralyzed. How do you know you don't have an unbounded utility function for paperclips? How do you know you don't have an unbounded utility function for, and assign infinite utility to, the universe being exactly the way it would be if you never made a fully rational decision again and just went around your life on autopilot? The end result is that there are a number of possible courses of action that would all generate infinity utility and no way to choose between them because infinity=infinity. The only reason your argument sounds logical is because you are allowing the questioning of the boundedness of the utility function, but not its contents.

I think that knowledge of your utility function is probably a basic, prerational thing, like deciding to use expected utility maximization and Bayesian updating in the first place. Attempting to insert your utility function itself into your calculations seems like a basic logical error.

Comment by Ghatanathoah on The Lifespan Dilemma · 2014-05-29T01:51:15.536Z · LW · GW

This tends to imply the Sadistic Conclusion: that it is better to create some lives that aren't worth living than it is to create a large number of lives that are barely worth living.

I think that the Sadistic Conclusion is correct. I argue here that it is far more in line with typical human moral intuitions than the repugnant one.

There are several "impossibility" theorems that show it is impossible to come up with a way to order populations that satisfies all of a group of intuitively appealing conditions.

If you take the underlying principle of the Sadistic Conclusion, but change the concrete example to something smaller scale and less melodramatic than "Create lives not worth living to stop the addition of lives barely worth living," you will find that it is very intuitively appealing.

For instance, if you ask people if they should practice responsible family planning or spend money combating overpopulation they agree. But (if we assume that the time and money spent on these efforts could have been devoted to something more fun) this is the same principle. The only difference is that instead creating a new life not worth living we are instead subtracting an equivalent amount of utility from existing people.

Comment by Ghatanathoah on Are multiple uploads equivilant to extra life? · 2014-05-07T03:02:45.964Z · LW · GW

It's worth noting that the question of what is a better way of evaluating such prospects is distinct from the question of how I in fact evaluate them.

Good point. What I meant was closer to "which method of evaluation does the best job of capturing how you intuitively assign value" rather than which way is better in some sort of objective sense. For me #1 seems to describe how I assign value and disvalue to repeating copies better than #2 does, but I'm far from certain.

So I think that from my point of view Omega offering to extend the length of a repeated event so it contains a more even mixture of good and bad is the same as Omega offering to not repeat a bad event and repeat a good event instead. Both options contain zero value, I would rather Omega leave me alone and let me go do new things. But they're better than him repeating a bad event.

Comment by Ghatanathoah on Are multiple uploads equivilant to extra life? · 2014-05-06T03:43:43.784Z · LW · GW

I think I understand your viewpoint. I do have an additional question though, which is what you think about how to to evaluate moments that have a combination of good and bad.

For instance, let's suppose you have the best day ever, except that you had a mild pain in your leg for the most of the day. All the awesome stuff you did during the day more than made up for that mild pain though.

Now let's suppose you are offered the prospect of having a copy of you repeat that day exactly. We both agree that doing this would add no additional value, the question is whether it would be valueless, or add disvalue?

There are two possible ways I see to evaluate this:

  1. You could add up all the events of the day and decide they contain more good than bad, therefore this was a "good" day. "Good" things have no value when repeated, so you would assign zero value to having a copy relive this day. You would not pay to have it happen, but you also wouldn't exert a great effort to stop it.

  2. You could assign value to the events first before adding them up, assigning zero value to all the good things and a slight negative value to the pain in your leg. Therefore you would assign negative value to having a copy relive this day and would pay to stop it from happening.

To me (1) seems to be an intuitively better way of evaluating the prospect of a copy reliving the day than (2). It also lines up with my intuition that it wouldn't be bad news if MWI was true. But I wonder if you would think differently?

Comment by Ghatanathoah on Are multiple uploads equivilant to extra life? · 2014-05-02T23:45:21.270Z · LW · GW

For my own part, I share your #1 and #2, don't share your #3 (that is, I'd rather Omega not reproduce the bad stuff, but if they're going to do so, it makes no real difference to me whether they reproduce the good stuff as well)

One thing that makes me inclined towards #3 is the possibility that the multiverse is constantly reproducing my life over and over again, good and bad. I do not think that I would consider it devastatingly bad news if it turns out that the Many-Worlds interpretation is correct.

If I really believed that repeated bad experiences could not ever be compensated for by repeated good ones, I would consider the Many Worlds Interpretation to be the worst news ever, since there were tons of me out in the multiverse having a mix of good and bad experiences, but the good ones "don't count" because they already happened somewhere else. But I don't consider it bad news. I don't think that if there was a machine that could stop the multiverse from splitting that I would pay to have it built.

One way to explain my preferences in this regard would be that I believe that repeated "good stuff" can compensate for repeated "bad stuff," but that it can't compensate for losing brand new "good stuff" or experiencing brand new "bad stuff."

However, I am not certain about this. There may be some other explanation for my preferences. Another possibility that I think is likely is that I think that repeated "good stuff" only loses its value for copies that have a strong causal connection to the current me. Other mes who exist somewhere out in the multiverse have no connection to this version of me whatsoever, so my positive experiences don't detract from their identical ones. But copies that I pay to have created (or to not be) are connected to me in such a fashion, so I (and they) do feel that their repeated experiences are less valuable.

This second explanation seems a strong contender as well, since I already have other moral intuitions in regards to causal connection (for instance, if there was a Matrioshka brain full of quintillions environmentalists in a part of the multiverse so far off they will never interact with us, I would not consider their preferences to be relevant when forming environmental policy, but I would consider the preferences of environmentalists here on Earth right now to be relevant). This relates to that "separability" concept we discussed a while ago.

Or maybe both of these explanations are true. I'm not sure.

Also, I'm curious, why are you indifferent in case 4? I think I might not have explained it clearly. What I was going for was that Omega say "I'm making a copy of you in a bad time of your life. I can either not do it at all, or extend the copy's lifespan so that it is now a copy of a portion of your life that had both good and bad moments. Both options cost $10." I am saying that I think I might be indifferent about what I spend $10 on in that case.

Comment by Ghatanathoah on Are multiple uploads equivilant to extra life? · 2014-05-02T03:56:50.096Z · LW · GW

I don't see anything inconsistent about believing that a good life loses values with repetition, but a bad life does not lose disvalue. It's consistent with the Value of Boredom, which I thoroughly endorse.

Now, there's a similar question where I think my thoughts on the subject might get a little weird. Imagine you have some period of your life that started out bad, but then turned around and then became good later so that in the end that period of life was positive on the net. I have the following preferences in regards to duplicating it:

  1. I would not pay to have a simulation that perfectly relived that portion of my life.

  2. If Omega threatened to simulate the bad first portion of that period of life, but not the good parts that turned it around later, I would pay him not to.

  3. If Omega threatened to simulate the bad first portion of that period of life, but not the good parts that turned it around later, I would probably pay him to extend the length of the simulation so that it also encompassed the compensating good part of that period of life.

  4. If the cost of 2 and 3 was identical I think would probably be indifferent. I would not care whether the simulation never occurred, or if it was extended.

So it seems like I think that repeated good experiences can sometimes "make up for" repeated bad ones, at least if they occur in the same instance of simulation. But all they can do is change the value I give to the simulation from "negative" to "zero." They can't make it positive.

These preferences I have do strike me as kind of weird. But on the other hand, the whole situation is kind of weird, so maybe any preferences I have about it will end up seeming weird no matter what they are.

Comment by Ghatanathoah on Parenting versus career choice thinking in teenagers · 2014-03-15T02:21:16.407Z · LW · GW

It seems like there's an easy way around this problem. Praise people who are responsible and financially well-off for having more kids. These traits are correlated with good genes and IQ, so it'll have the same effect.

It seems like we already do this to some extent. I hear others condemning people with who are irresponsible and low-income for having too many children fairly frequently. It's just that we fail to extend this behavior in the other direction, to praising responsible people for having children.

I'm not sure why this is. It could be for one of the reasons listed in the OP. Or it could just be because the tendency to praise and the tendency to condemn are not correlated.

Comment by Ghatanathoah on Stupid Questions Thread - January 2014 · 2014-01-27T22:00:56.720Z · LW · GW

I'm very unfamiliar with it, but intuitively I would have assumed that the preferences in question wouldn't be all the preferences that the agent's value system could logically be thought to imply, but rather something like the consciously held goals at some given moment

I don't think that would be the case. The main intuitive advantage negative preference utilitarianism has over negative hedonic utilitarianism is that it considers death to be a bad thing, because it results in unsatisfied preferences. If it only counted immediate consciously held goals it might consider death a good thing, since it would prevent an agent from developing additional unsatisfied preferences in the future.

However, you are probably onto something by suggesting some method of limiting which unsatisfied preferences count as negative. "What a person is thinking about at any given moment" has the problems I pointed out earlier, but another formulation could well work better.

Otherwise total preference utilitarianism would seem to reduce to negative preference utilitarianism as well, since presumably the unsatisfied preferences would always outnumber the satisfied ones.

I believe Total Preference Utilitarianism typically avoids this by regarding the creation of at most types of unsatisfied preferences as neutral rather than negative. While there are some preferences whose dissatisfaction typically counts as negative, such as the preference not to be tortured, most preference creations are neutral. I believe that under TPU, if a person spends the majority of their life not preferring to be dead then their life is considered positive no matter how many unsatisfied preferences they have.

At least I personally find it very difficult to compare experiences of such differing magnitudes.

I feel like I could try to get some sort of ballpark by figuring how much I'm willing to pay to avoid each thing. For instance, if I had an agonizing migraine I knew would last all evening, and had a choice between paying for an instant cure pill, or a device that would magically let me avoid traffic for the next two months, I'd probably put up with the migraine.

I'd be hesitant to generalize across the whole population, however, because I've noticed that I don't seem to mind pain as much as other people, but find boredom far more frustrating than average.

Comment by Ghatanathoah on To capture anti-death intuitions, include memory in utilitarianism · 2014-01-24T20:30:22.282Z · LW · GW

I guess I see a set of all possible types of sentient minds with my goal being to make the universe as nice as possible for some weighted average of the set.

I used to think that way, but it resulted in what I considered to be too many counterintuitive conclusions. The biggest one, that I absolutely refuse to accept, being that we ought to kill the entire human race and use the resources doing that would free up to replace them with creatures whose desires are easier to satisfy. Paperclip maximizers or wireheads for instance. Humans have such picky, complicated goals, after all..... I consider this conclusion roughly a trillion times more repugnant than the original Repugnant Conclusion.

Naturally, I also reject the individual form of this conclusion, which is that we should kill people who want to read great books, climb mountains, run marathons, etc. and replace them with people who just want to laze around. If I was given a choice between having an ambitious child with a good life, or an unambitious child with a great life, I would pick the ambitious one, even though the total amount of welfare in the world would be smaller for it. And as long as the unambitious child doesn't exist, never existed, and never will exist I see nothing wrong with this type of favoritism.

Comment by Ghatanathoah on Stupid Questions Thread - January 2014 · 2014-01-24T20:08:18.931Z · LW · GW

A bounded utility function does help matters, but then everything depends on how exactly it's bounded, and why one has chosen those particular parameters.

Yes, and that is my precise point. Even if we assume a bounded utility function for human preferences, I think it's reasonable assume that it's a pretty huge function. Which means that antinatalism/negative preference utilitarianism would be willing to inflict massive suffering on existing people to prevent the birth of one person who would have a better life than anyone on Earth has ever had up to this point, but still die with a lot of unfulfilled desires. I find this massively counter-intuitive and want to know how the antinatalist community addresses this.

I take it you mean to say that they don't spend all of their waking hours convincing other people not to have children, since it doesn't take that much effort to avoid having children yourself.

If the disutility they assign to having children is big enough they should still spend every waking hour doing something about it. What if some maniac kidnaps them and forces them to have a child? The odds of that happening are incredibly small, but they certainly aren't zero. If they really assign such a giant negative to having a child they should try to guard even against tiny possibilities like that.

Also, are they all transhumanists? For the typical person (or possibly even typical philosopher), infinite lifespans being a plausible possibility might not even occur as something that needs to be taken into account

Yes, but from a preference utilitarian standpoint it doesn't need to actually be possible to live forever. It just has to be something that you want.

Does any utilitarian system have a good answer to questions like these? If you ask a total utilitarian something like "how much morning rush-hour frustration would you be willing to inflict to people in order to prevent an hour of intense torture, and how exactly did you go about calculating the answer to that question", you're probably not going to get a very satisfying answer, either.

Well, of course I'm not expecting an exact answer. But a ballpark would be nice. Something like "no more than x, no less than y." I think, for instance, that a total utilitarian could at least say something like "no less than a thousand rush hour frustrations, no more than a million."

Comment by Ghatanathoah on Stupid Questions Thread - January 2014 · 2014-01-24T04:31:43.636Z · LW · GW

Speaking personally, I don't negatively weigh non-aversive sensory experiences. That is to say, the billions of years of unsatisfied preferences are only important for that small subset of humans for whom knowing about the losses causes suffering.

If I understand you correctly, the problem with doing this with negative utilitarianism is that it suggests we should painlessly kill everyone ASAP. The advantage of negative preference utilitarianism is that it avoids this because people have a preference to keep on living that killing would thwart.

It's worth pointing out that negative utilitarianism is incoherent.

Why? For the reason I pointed out, or for a different one? I'm not a negative utilitarian personally, but I think a few aspects of it have promise and would like to see them sorted out.

Comment by Ghatanathoah on To capture anti-death intuitions, include memory in utilitarianism · 2014-01-24T04:22:36.728Z · LW · GW

Not relevant because we are considering bringing these people into existence at which point they will be able to experience pain and pleasure.

Yes, but I would argue that the fact that they can't actually do that yet makes a difference.

Imagine you know that one week from now someone will force you to take heroin and you will become addicted. At this point you will be able to have an OK life if given a regular amount of the drug but will live in permanent torture if you never get any more of the substance. Would you pay $1 today for the ability to consume heroin in the future?

Yes, if I was actually going to be addicted. But it was a bad thing that I was addicted in the first place, not a good thing. What I meant when I said I "do not care in the slightest" was that the strength of that desire was not a good reason to get addicted to heroin. I didn't mean that I wouldn't try to satisfy that desire if I had no choice but to create it.

Similarly, in the case of adding lots of people with short lives, the fact that they would have desires and experience pain and pleasure if they existed is not a good reason to create them. But it is a good reason to try to help them extend their lives, and lead better ones, if you have no choice but to create them.

Thinking about it further, I realized that you were wrong in your initial assertion that "we have to introduce a fudge factor that favors people (such as us) who are or were alive." The types of "fudge factors" that are being discussed here do not, in fact do that.

To illustrate this, imagine Omega presents you with the following two choices:

  1. Everyone who currently exists receives a small amount of additional utility. Also, in the future the amount of births in the world will vastly increase, and the lifespan and level of utility per person will vastly decrease. The end result will be the Repugnant Conclusion for all future people, but existing people will not be harmed, in fact they will benefit from it.

  2. Everyone who currently exists loses a small amount of their utility. In the future far fewer people will be born than in Option 1, but they will live immensely long lifespans full of happiness. Total utility is somewhat smaller than in Option 1, but concentrated in a smaller amount of people.

Someone using the fudge factor Kaj proposes in the OP would choose 2, even though it harms every single existing person in order to benefit people who don't exist yet. It is not biased towards existing persons.

I basically view adding people to the world in the same light as I view adding desires to my brain. If a desire is ego-syntonic (i.e. a desire to read a particularly good book) then I want it to be added and will pay to make sure it is. If a desire is ego-dystonic (like using heroin) I want it to not be added and will pay to make sure it isn't. Similarly, if adding a person makes the world more like my ideal world (i.e. a world full of people with long eudaemonic lives) then I want that person to be added. If it makes it less like my ideal world (i.e. Repugnant Conclusion) I don't want that person to be added and will make sacrifices to stop it (for instance, I will spend money on contraceptives instead of candy).

As long as the people we are considering adding are prevented from ever having existed, I don't think they have been harmed in the same way that that discriminating against an existing person for some reason like skin color or gender harms someone, and I see nothing wrong with stopping people from being created if it makes the world more ideal.

Of course, needless to say, if we fail and these people are created anyway, we have just as much moral obligation towards them as we would towards any preexisting person.

Comment by Ghatanathoah on To capture anti-death intuitions, include memory in utilitarianism · 2014-01-23T20:18:46.715Z · LW · GW

For me, however, it doesn't seem all that far from someone saying "I'm a utilitarian but my intuition strongly tells me that people with characteristic X are more important than everyone else so I'm going to amend utilitarianism by giving greater weight to the welfare of X-men."

There is a huge difference between discriminatory favoritism, and valuing continued life over adding new people,

In discriminatory favoritism people have a property that makes them morally valuable (i.e the ability to have preferences, or to feel pleasure and pain). They also have an additional property that does not affect their morally valuable property in any significant way (i.e skin color, family relations). Discriminatory favoritism argues that this additional property means that the welfare of these people is less important, even though that additional property does not affect the morally valuable property in any way.

By contrast, in the case of valuing continuing life over creating new people, the additional property (nonexistance) that the new people have does have a significant effect on their morally significant property. Last I checked never having existed had a large effect on your ability to have preferences, and your ability to feel please and pain. If the person did exist in the past, or will exist in the future, that will change, but if they never existed, don't exist, and never will exist, then I think that is significant. Arguing that it shouldn't be is like arguing you shouldn't break a rock because "if the rock could think, it wouldn't want you to."

We can illustrate it further by thinking about individual preferences instead of people. If I become addicted to heroin I will have a huge desire to take heroin far stronger than all the desires I have now. This does not make me want to be addicted to heroin. At all. I do not care in the slightest that the heroin addicted me would have a strong desire for heroin. Because that desire does not exist and I intend to keep it that way. And I see nothing immoral about that.

Comment by Ghatanathoah on To capture anti-death intuitions, include memory in utilitarianism · 2014-01-23T20:00:40.024Z · LW · GW

Though now that you point it out, it is a problem that, under this model, creating a person who you don't expect to live forever has a very high (potentially infinite) disutility. Yeah, that breaks this suggestion. Only took a couple of hours, that's ethics for you. :)

Oddly enough, right before I noticed this thread I posted a question about this on the Stupid Questions Thread.

My question, however, was whether this problem applies to all forms of negative preferences utilitarianism. I don't know what the answer is. I wonder if SisterY or one of the other antinatalists who frequents LW does.

Comment by Ghatanathoah on Stupid Questions Thread - January 2014 · 2014-01-23T18:58:21.624Z · LW · GW

What amount of disutility does creating a new person generate in Negative Preference Utilitarian ethics?

I need to elaborate in order to explain exactly what question I am asking: I've been studying various forms of ethics, and when I was studying Negative Preference Utilitarianism (or anti-natalism, as I believe it's often also called) I came across what seems like a huge, titanic flaw that seems to destroy the entire system.

The flaw is this: The goal of negative preference utilitarianism is to prevent the existence of unsatisfied preferences. This means that negative preference utilitarians are opposed to having children, as doing so will create more unsatisfied preferences. And they are opposed to people dying under normal circumstances, because someone's death will prevent them from satisfying their existing preferences.

So what happens when you create someone who is going to die, and has an unbounded utility function? The amount of preferences they have is essentially infinite, does that mean that if such a person is created it is impossible to do any more harm, since an infinite amount of unsatisfied preferences have just been created? Does that mean that we should be willing to torture everyone on Earth for a thousand years if doing so will prevent the creation of such a person?

The problem doesn't go away if you assume humans have bounded utility functions. Suppose we have a bounded utility function, so living an infinite number of years, or a googolplex number of years, is equivalent to living a mere hundred billion years for us. That still means that creating someone who will live a normal 70 year lifespan is a titanic harm, a harm that everyone alive on Earth today should be willing to die to prevent it, as it would create 99,999,999,930 years worth of unsatisfied preferences!

My question is, how do negative preference utilitarians deal with this? The ones I've encountered online make an effort to avoid having children, but they don't devote every waking minute of their lives to it. And I don't think akrasia is the cause, because I've heard some of them admit that it would be acceptable to have a child if doing so reduced the preference frustration/suffering of a very large amount of existing people.

So with that introduction out of the way, my questions, on a basic level are:

  1. How much suffering/preference frustration would an antinatalist be willing to inflict on existing people in order to prevent a birth? How much suffering/preference frustration would a birth have to stop in order for it to be justified? For simplicity's sake, let's assume the child who is born has a normal middle class life in a 1st world country with no exceptional bodily or mental health problems.

  2. How exactly did they go about calculating the answer to question 1?

There has to be some answer to this question, there wouldn't be whole communities of anti-natalists online if their ideology could be defeated with a simple logic problem.

Comment by Ghatanathoah on Skirting the mere addition paradox · 2014-01-20T08:25:40.059Z · LW · GW

It is also worth noting that average utilitarianism has also its share of problems: killing off anyone with below-maximum utility is an improvement.

No it isn't. This can be demonstrated fairly simply. Imagine a population consisting of 100 people. 99 of those people have great lives, 1 of those people has a mediocre one.

At the time you are considering doing the killing the person with the mediocre life, he has accumulated 25 utility. If you let him live he will accumulate 5 more utility. The 99 people with great lives will accumulate 100 utility over the course of their lifetimes.

If you kill the guy now average utility will be 99.25. If you let him live and accumulate 5 more utility average utility will be 99.3. A small, but definite improvement.

I think the mistake you're making is that after you kill the person you divide by 99 instead of 100. But that's absurd, why would someone stop counting as part of the average just because they're dead? Once someone is added to the population they count as part of it forever.

It is also worth noting that average utilitarianism has also its share of problems: killing off anyone with below-maximum utility is an improvement.

It's true that some sort of normalization assumption is needed to compare VNM utility between agents. But that doesn't defeat utilitarianism, it just shows that you need to include a meta-moral obligation to make such an assumption (and to make sure that assumption is consistent with common human moral intuitions about how such assumptions should be made).

As it happens, I do interpersonal utility comparisons all the time in my day-to-day life using the mental capacity commonly referred to as "empathy." The normalizing assumption I seem to be making is to assume that others people's minds are similar to mine, and match their utility to mine on a one to one basis, doing tweaks as necessary if I observe that they value different things than I do.

Comment by Ghatanathoah on 'Effective Altruism' as utilitarian equivocation. · 2014-01-18T05:06:41.980Z · LW · GW

I wonder what a CEV-implementing AI would do with such cases.

Even if it does turn out that my current conception of personal identity isn't the same as my old one, but is rather I similar concept I adopted after realizing my values were incoherent, the AI might still find that the CEVs of my past and present selves concur. This is because, if I truly did adopt a new concept of identity because of it's similarity to my old one, this suggests I possess some sort of meta-value that values taking my incoherent values and replacing them with coherent ones that are as similar as possible to the original. If this is the case the AI would extrapolate that meta-value and give me a nice new coherent sense of personal identity, like the one I currently possess.

Of course, if I am right and my current conception of personal identity is based on my simply figuring out what I meant all along by "identity," then the AI would just extrapolate that.

Comment by Ghatanathoah on 'Effective Altruism' as utilitarian equivocation. · 2014-01-17T22:53:09.195Z · LW · GW

Granted, negative utilitarians would prefer to add a small population of beings with terrible lives over a very large beings with lives that are almost ideal, but this would not be a proper instance of the Sadistic Conclusion. See the formulation:

When I read the formulation of the Sadistic Conclusion I interpreted "people with positive utility" to mean either a person whose life contained no suffering, or a person whose satisfied preferences/happiness outweighed their suffering. So I would consider adding a small population of terrible lives instead of a large population of almost ideal lives to be the Sadistic Conclusion.

If I understand you correctly, you are saying that negative utilitarianism technically avoids the Sadistic Conclusion because it considers a life with any suffering at all to be a life of negative utility, regardless of how many positive things that life also contains. In other words, it avoid the SC because it's criterion for what makes a life positive and negative are different than the criterion Arrenhius used when he first formulated the SC. I suppose that is true. However, NU does not avoid the (allegedly) unpleasant scenario Arrenhius wanted to avoid (adding a tortured life instead of a large amount of very positive lives).

Negative utilitarians try to minimize the total amount of preference-frustrations, or suffering....(Also note that being killed is only a problem if you have a preference to go on living, and that even then, it might not be the thing considered worst that could happen to someone.)

Right, but if someone has a preference to live forever does that mean that infinite harm has been done if they die? In which case you might as well do whatever afterwards, since infinite harm has already occurred? Should you torture everyone on Earth for decades to prevent such a person from being added? That seems weird.

The best solution I can currently think of is to compare different alternatives, rather than try to measure things in absolute terms. So if a person who would have lived to 80 dies at 75 that generates 5 years of unsatisfied preferences, not infinity, even if the person would have preferred to live forever. But that doesn't solve the problem of adding people who wouldn't have existed otherwise.

What I'm trying to say is, people have an awful lot of preferences, and generally only manage to satisfy a small fraction of them before they die. So how many unsatisfied preferences should adding a new person count as creating? How big a disutility is it compared to other disutilities, like thwarting existing preferences and inflicting pain on people.

A couple possibilities that occurs to me off the top of my head. One would be to find the difference in satisfaction between the new people and the old people, and then compare it to the difference in satisfaction between the old people and the counter-factual old people in the universe where the new people were never added.

Another possibility would be to set some sort of critical level based on what the maximum level of utility it is possible to give the new people given our society's current level of resources, without inflicting greater disutilities on others than you give utility to the new people. Then weigh the difference between the new peoples actual utility and their "critical possible utility" and compare that to the dissatisfaction the existing people would suffer if the new people are not added.

Do either of these possibilities sound plausible to you, or do you have another idea?

Comment by Ghatanathoah on 'Effective Altruism' as utilitarian equivocation. · 2013-12-30T19:22:53.123Z · LW · GW

Now, once someone realises this, he/she can either choose to group all the consciousness-moments together that trigger an intuitive notion of "same person" and care about that, even though it is now different from what they thought it was

I don't know if your analysis is right or not, but I can tell you that that isn't what it felt like I was doing when I was developing my concepts of personal identity and preferences. What it felt like I was doing was elucidating a concept I already cared about, and figured out exactly what I meant when I said "same person" and "personal identity." When I thought about what such concepts mean I felt a thrill of discovery, like I was learning something new about myself I had never articulated before.

It might be that you are right and that my feelings are illusory, that what I was really doing was realizing a concept I cared about was incoherent and reaching about until I found a concept that was similar, but coherent. But I can tell you that's not what it felt like.

EDIT: Let me make an analogy. Ancient people had some weird ideas about the concept of "strength." They thought that it was somehow separate from the body of a person, and could be transferred by magic, or by eating a strong person or animal. Now, of course, we understand that that is not how strength works. It is caused by the complex interaction of a system of muscles, bones, tendons, and nerves, and you can't transfer that complex system from one entity to another without changing many of the properties of the entity you're sending it to.

Now, considering that fact, would you say that ancient people didn't want anything coherent when they said they wanted to be strong? I don't think so. They were mistaken about some aspects about how strength works, but they were working from a coherent concept. Once they understood how strength worked better they didn't consider their previous desire for strength to be wrong.

I see personal identity as somewhat analagous to that. We had some weird ideas about it in the past, like that it was detached from physical matter. But I think that people have always cared about how they are going to change from one moment to the next, and had concrete preferences about it. And I think when I refined my concepts of personal identity I was making preferences I already had more explicit, not swapping out some incoherent preferences and replacing them with similar coherent ones.

I'm 100% sure that there is something I mean by "suffering", and that it matters. I'm only maybe 10-20% sure that I'd also want to care about preferences if I knew everything there is to know.

I am 100% certain that there are things I want to do that will make me suffer (learning unpleasant truths for instance), but that I want to do anyway, because that is what I prefer to do.

Suffering seems relevant to me too. But I have to admit, sometimes when something is making me suffer, what dominates my thoughts is not a desire for it to stop, but rather annoyance that this suffering is disrupting my train of thought and making it hard for me to think and get the goals I have set for myself accomplished. And I'm not talking about mild suffering, the example in particular that I am thinking of is throwing up two days after having my entire abdomen cut open and sewn back together.

Comment by Ghatanathoah on 'Effective Altruism' as utilitarian equivocation. · 2013-12-30T19:00:22.981Z · LW · GW

If not, you seem to not intrinsically value the creation of satisfied preferences.

You're right that I do not intrinsically value the creation of all satisfied preferences. This is where my version of Moore's Ideal Utilitarianism comes in. What I value is the creation of people with satisfied preferences if doing so also fulfills certain moral ideals I (and most other people, I think) have about how the world ought to be. In cases where the creation of a person with satisfied preferences would not fulfill those ideals I am essentially a negative preference utilitarian, I treat the creation of a person who doesn't fulfill those ideals the same way a negative preference utilitarian would.

I differ from Moore in that I think the only way to fulfill an ideal is to create (or not create) a person with certain preferences and satisfy those preferences. I don't think, like he did, that you can (for example) increase the beauty in the world by creating pretty objects no one ever sees.

I think a good analogy would again be Parfit's concept of global preferences. If I read a book, and am filled with a mild preference to read more books with the same characters, such a desire is in line with my global preferences, so it is good for it to be created. By contrast, being addicted to heroin would fill me with a strong preference to use heroin. This preference is not in line with my global preferences, so I would be willing to hurt myself to avoid creating it.

Suppose there were ten people, and they would be okay with getting tortured, adding a billion tortured people, plus adding a sufficiently large number of people with preferences more-satisfied-than-not.

I have moral ideals about many things, which include how many people there should be, their overall level of welfare, and most importantly, what sort of preferences they ought to have. It seems likely to me that the scenario with the torture+new people scenario would violate those ideals, so I probably wouldn't go along with it.

To give an example where creating the wrong type of preference would be a negative, I would oppose the creation of a sociopath or a paperclip maximizer, even if their life would have more satisfied preferences than not. Such a creature would not be in line with my ideals about what sort of creatures should exist. I would even be willing to harm myself or others, to some extent, to prevent their creation.

This brings up a major question I have about negative preference utilitarianism, which I wonder if you could answer since you seem to have thought more about the subject of negative utilitarianism than I have. How much harm should a negative preference utilitarian be willing to inflict on existing people to prevent a new person from being born? For instance, suppose you had a choice between torturing every person on Earth for the rest of their lives, or creating one new person who will live the life of a rich 1st world person with a high happiness set point? Surely you wouldn't torture everyone on Earth? A hedonist negative utilitarian wouldn't of course, but we're talking about negative preference utilitarianism.

A similar question I have is, if a creature with an unbounded utility function is created, does that mean that infinite wrong has been done, since such a creature essentially has infinite unsatisfied preferences? How does negative preference utilitarianism address this?

The best thing I can come up with is to give the creation of such a creature a utility penalty equal to "However much utility the creature accumulates over its lifetime, minus x," where x is a moderately sized number. However, it occurs to me that someone whose thought more about the subject than me might have figured out something better.

Comment by Ghatanathoah on The Lifespan Dilemma · 2013-12-17T15:18:42.897Z · LW · GW

Whatever Omega is doing that might kill you might not be tied to the mechanism that divides universes. It might be that the choice is between huge chance of all of the yous in every universe where you're offered this choice dying, vs. tiny chance they'll all survive.

Also, I'm pretty sure that Eliezer's argument is intended to test our intuitions in an environment without extraneous factors like MWI. Bringing MWI into the problem is sort of like asking if there's some sort of way to warn everyone off the tracks so no one dies in the Trolley Problem.

Comment by Ghatanathoah on 'Effective Altruism' as utilitarian equivocation. · 2013-12-15T07:29:05.352Z · LW · GW

I'm leaning towards a hedonistic view, though, and one reason for this has to do with my view on personal identity. I don't think the concept makes any sense.

I consider nearly all arguments of the form "X is not a coherent concept, therefore we ought not to care about it" to be invalid. I don't mean to give offense, but such arguments seem to me to be a form of pretending to be wise. This is especially true if X has predictive power, if knowing something is X can cause you to correctly anticipate your experiences. And you have to admit, knowing someone is the same person as someone you've encountered before makes you more likely to be able to predict their behavior.

Arguments that challenge the coherency of a concept typically function by asking a number of questions that our intuitions about the concept cannot answer readily, creating a sense of dumbfoundment. They then do not bother to think further about the questions and try to answer them, instead taking the inability to answer the question readily as evidence of incoherence. These arguments also frequently appeal to the fallacy of the gray, assuming that because there is no clear-cut border between two concepts or things that no distinction between them must exist.

This fact was brought home to me when I came across discussions of racism that argued that racism was wrong because "race" was not a coherent concept. The argument initially appealed to me because it hit a few applause lights, such as "racism is bad," and "racists are morons." However, I became increasingly bothered because I was knowledgeable about biology and genetics, and could easily see several simple ways to modify the concept of "race" into something coherent. It also seemed to me that the reason racism was wrong was that preference violation and suffering were bad regardless of the race of the person experiencing them, not because racists were guilty of incoherent reasoning. I realized that the argument was a terrible one, and could only be persuasive if one was predisposed to hate racism for other reasons.

The concept of personal identity makes plenty of sense. I've read Parfit too, and read all the questions about the nature of personal identity. I then proceeded to actually answer those questions and developed a much better understanding of what exactly it is I value when I say I value personal identity. To put it (extremely) shortly:

There are entities that have preferences about the future. These preferences include preferences about how the entity itself will change in the future (Parfit makes a similar point when he discusses "global preferences"). These preferences constitute a "personal identity." If an entity changes we don't need to make any references to the changed entity being the "same person" as a past entity. We simply take into account whether the change is desirable or not. I write much more about the subject here.

I don't think my present self has any privileged (normative) authority over my future selves

I don't necessarily think that either. That's why I want to make sure that the future self that I turn into remains similar to my present self in certain ways, especially in his preferences. That way the issue won't ever come up.

because when I just think in terms of consciousness-moments

This might be your first mistake. We aren't just consciousness moments. We're utility functions, memories, personalities and sets of values. Our consciousness-moments are just the tip of the iceberg. That's one reason why it's still immoral to violate a person's preferences when they're unconscious, their values still exist somewhere in their brain, even when they're not conscious.

I find it counterintuitive why preferences (as opposed to suffering) would be what is relevant.

It seems obvious to me that they're both relevant.

Comment by Ghatanathoah on Being Wrong about Your Own Subjective Experience · 2013-12-15T04:42:27.654Z · LW · GW

This would explain why I sometimes dream in cartoons. And I don't mean I dream about popular recognizable cartoon characters, I mean I dream about myself and people I know, but my mind has somehow "drawn" them so they look like cartoon characters.

Comment by Ghatanathoah on The I-Less Eye · 2013-12-15T04:37:05.333Z · LW · GW

You might be able to get a scenario like this without mind-copying by using a variety of Newcomb's Problem.

You wake up without any memories of the previous day. You then see Omega in front of you, holding two boxes. Omega explains that if you pick the first box, you will be tortured briefly now. If you pick the second box, you won't be.

However, Omega informs you that he anticipated which box you would choose. If he predicted you'd pick the first box, the day before yesterday he drugged you so you'd sleep through the whole day. If he predicted you'd pick the second box he tortured you for a very long period of time the previous day and erased your memory of it afterward. He acknowledges that torture one doesn't remember afterwards isn't as bad as torture one does, and assures you that he knows this and extended the length of the previous day's torture to compensate.

It seem to me like there'd be a strong temptation to pick the second box. However, your self from a few days ago would likely pay to be able to stop you from doing this.

Comment by Ghatanathoah on 'Effective Altruism' as utilitarian equivocation. · 2013-12-15T04:03:13.728Z · LW · GW

So would you accept the very repugnant conclusion for total preference utilitarianism?

I did not mention it because I didn't want to belabor my view, but no, I wouldn't. I think that one of the important Ideals that people seem to value is that a smaller population of people with highly satisfied preferences is better than a larger population with lives barely worth living, even if the total amount of preference satisfaction is higher in the large population. That's one reason why the repugnant conclusion is repugnant. This means that sometimes it is good to add people, at other times it is bad.

Of course, this view needs some qualifiers. First of all, once someone is added to a population, they count as being part of it even after they are dead, so you can't arrive at an ideal population size by killing people. This also entails accepting the Sadistic Conclusion, but that is an unavoidable part of all types of Negative Utilitarianism, whether they are of the normal variety, or the weird "sometimes negative sometimes positive depending on the context" variety I employ.

I think a helpful analogy would be Parfit's concept of "global preferences," which he discusses on page 3 of this article. Parfit argues that we have "Global Preferences," which are meta-preferences about what sort of life we should live and what sort of desires we should have. He argues that these Global Preferences dictate the goodness of whether we develop a new preference.

For instance, Parfit argues, imagine someone gets you addicted to a drug, and gives you a lifetime supply of the drug. You now have a strong desire to get more of the drug, which is satisfied by your lifetime supply. Parfit argues that this does not make you life better, because you have a global meta-preference to not get addicted to drugs, which has been violated. By contrast (my example, not Parfit's) if I enter into a romantic relationship with someone it will create a strong desire to spend time with that person, a desire much stronger than my initial desire to enter the relationship. However, this is a good thing, because I do have a global meta-preference to be in romantic relationships.

We can easily scale this up to population ethics. I have Global Moral Principles about the type and amount of people who should exist in the world. Adding people who fulfill these principles makes the world better. Adding people who do not fulfill these principles makes the world worse, and should be stopped.

And it, too, seems to be selfish in a way, although this would have to be argued for further.

Reading and responding to your exchange with other people about this sort of "moral selfishness" has gotten me thinking about what people mean and what concepts they refer to when they use that word. I've come to the conclusion that "selfish" isn't a proper word to use in these contexts. Now, obviously this is something of a case of "disputing defintions", but the word "selfish" and the concepts it refers to are extremely "loaded" and bring a lot of emotional and intuitive baggage with them, so I think it's good mental hygiene to be clear about what they mean.

To me what's become clear is that the word "selfish" doesn't refer to any instance where someone puts some value of theirs ahead of something else. Selfishness is when someone puts their preferences about their own life and about their own happiness and suffering ahead of the preferences, happiness, and suffering of others.

To illustrate this, imagine the case of a racial supremacist who sacrifices his life in order to enable others of his race to continue oppressing different races. He is certainly a very bad person. But it seems absurd to call him "selfish." In my view this is because, while he has certainly put some of his preferences ahead of the preferences of others, none of those preferences were preferences about his own life. They were preferences about the overall state of the world.

Now, obviously what makes a preference "about your own life" is a fairly complicated concept (Parfit discusses it in some detail here). But I don't see that as inherently problematic. Most concepts are extremely complex once we unpack them.

Comment by Ghatanathoah on 'Effective Altruism' as utilitarian equivocation. · 2013-12-12T07:13:37.885Z · LW · GW

This is a form of negative utilitarianism, and inherits the major problems with that theory (such as its endorsement of destroying the universe to stop all the frustrated preferences going on it in it right now)

I'm pretty sure that most forms of negative preference utilitarianism are "timeless." Once a strong "terminal value" type preference is created it counts as always existing, forever. If you destroy the universe the frustrated preferences will still be there, even harder to satisfy than before.

It might, but it would be one that was outweighed by the larger number of preference-satisfactions to be gained from doing so, just like the disutility of torturing someone for 50 years is outweighed by the utility of avoiding 3^^^3 dust-speck incidents (for utilitarian utility functions).

To get around this I employ a sort of "selective negative utilitarianism." To put it bluntly, I count the creation of people with the sort of complex humane values I appreciate to be positive, for the most part,* but consider creating creatures with radically simpler values (or modifying existing complex creatures into them) to count as a negative.

This results in a sort of two-tier system, where I'm basically a preference utilitarian for regular ethics, and an ideal utilitarian for population ethics. In situations where the population is fixed I value all preferences fairly equally.** But when adding new people, or changing people's preferences, I consider it bad to add people who don't have preferences for morally valuable ideals like Truth , Freedom, Justice, etc.

*Of course, I also reject the Repugnant Conclusion. So I also consider adding complex creatures to be negative if it pushes the world in the direction of the RC.

**One exception is that I don't value extremely sadistic preferences at all. I'd rescue the person who is about to be tortured in the Thousand Sadist' Problem.

Comment by Ghatanathoah on 'Effective Altruism' as utilitarian equivocation. · 2013-12-12T06:52:51.611Z · LW · GW

it doesn't address my point that utilitarianism is systematized altruism. Utilitarianism is what results if you apply veil of ignorance type of reasoning without being risk-averse.

Preference Utilitarianism, or Parfit's Success Theory, might be considered systematizied altruism. But classical utilitarianism isn't altruistic at all. It doesn't care about anything or anyone. It only cares about certain types of feelings. It ignores all other personal goals people have in a monstrously sociopathic fashion.

As someone with negative utilitarian inclinations, I sympathize with the "self-centered preference for hedonism" objection against classical utilitarianism.

I occasionally desire to do activities that make me suffer because I value the end result of that activity more than I value not suffering. If you try to stop me you're at least as selfish as a hedonistic utilitarian who makes people suffer in order to generate a large amount of pleasure. (This is of course, if you're a hedonistic negative utilitarian. If you're a negative preference utilitarian I presume you'd want me to do that activity to prevent one of my preferences from not being satisfied.)

In my view, both classical and negative (hedonistic) utilitarianism are sort of "selfish" because being unselfish implies you respect other people's desires for how their lives should go. If you make someone feel pleasure when they don't want to, or not feel pain when they do want to, you are harming them. You are making their lives worse.

In fact, I think that classical and negative (hedonistic) utilitarianism are misnamed, because "utilitarian" is derived from the word "utility," meaning "usefulness." Something is "utilitarian" if it is useful to people in achieving their goals. But classical and negative utilitarians do not consider people's goals to be valuable at all. All they value is Pleasure and NotPain. People are not creatures with lives and goals of their own, they are merely receptacles for containing Pleasure and NotPain.

Preference utilitarianism, Parfit's Success Theory, and other similar theories do deserve the title of "utilitarian." They do place value on the lives and desires of others. So I would say that they are "unselfish." But all pleasure-maximizing/pain-minimizing theories of value are.

Comment by Ghatanathoah on 'Effective Altruism' as utilitarian equivocation. · 2013-12-12T06:30:24.914Z · LW · GW

Every non-sentientist value that you add to your pool of intrinsic values needs an exchange rate (which can be non-linear and complex and whatever) that implies you'd be willing to let people suffer in exchange for said value....If other people value tradition intrinsically, then preference utilitarianism will output that tradition counts to the extent that it satisfies people's preferences for it. This would be the utilitarian way to include "complexity of value".

I am proceeding along this line of thought as well. I believe in something similar to G. E. Moore's "Ideal Utilitarianism." I believe that we should maximize certain values like Truth, Beauty, Curiousity, Freedom, etc. However, I also believe that these values are meaningless when divorced from the existence of sentient creatures to appreciate them. Unlike Moore, I would not place any value at all on a piece of lovely artwork no one ever sees. There would need to be a creature to appreciate it for it to have any value.

So basically, I would shift maximizing complex values from regular ethics to population ethics. I would give "extra points" to the creation of creatures who place intrinsic value on these ideals, and "negative points" to the creation of creatures who don't value them.

Now, you might argue that this does create scenarios where I am willing to create suffering to promote an ideal. Suppose I have the option of creating a wireheaded person that never suffers, or a person who appreciates ideals, and suffers a little (but not so much that their life is not worth living). I would gladly choose the idealistic person over the wirehead.

I do not consider this to be "biting a bullet" because that usually implies accepting a somewhat counterintuitive implication, and I don't find this implication to be counterintuitive at all. As long as the idealistic person's life is not so terrible that they wish they had never been born I can not truly be said to have hurt them.

Comment by Ghatanathoah on Resurrection through simulation: questions of feasibility, desirability and some implications · 2013-12-06T12:46:00.945Z · LW · GW

Continued debate in this thread doesn't seem very productive to me, since all of our disagreement seems to come down to differing sets of moral intuitions / terminal values.

You're probably right.

EDIT: However, I do think you should consider if your moral intuitions really are different, or if you've somehow shut some important intuitions off by use of the "make anything arbitrary" rhetorical strategy I described earlier.

Also, I should clarify that while I disapprove of the normative conclusions you've drawn from personal identity skepticism, I don't see any inherent problem with using it to improve your mental health in the way you described (when you said that it decreased your anxiety about death). If your emotional systems are out of control and torturing you with excessive anxiety I don't see any reason why you shouldn't try a mental trick like that to treat it.

Comment by Ghatanathoah on Put Yourself in Manual Mode (aka Shut Up and Multiply) · 2013-12-06T12:41:51.820Z · LW · GW

So you're saying that by spending resources on not creating the new lives, people are essentially choosing the "create a life with negative welfare" option, but instead of creating a new life with negative welfare, an equivalent amount is subtracted from their own welfare. Am I understanding you correctly?

Yes, that's what I was trying to say.

Comment by Ghatanathoah on An argument that animals don't really suffer · 2013-12-06T01:03:19.224Z · LW · GW

If an important distinction between people and animals is that animals only take pain up to 2, then a human who perceives pain only (mostly?) at the 2 level might be more like an animal.

In terms of experiencing pain, yes (although I do think there are more level 3 animals than Craig does). If I had to choose between torturing a level 2 human or a level 3 human I'd pick a level 2, providing the torture did no lasting damage to the body.

However, a far, far more morally significant distinction between a human and an animal is that humans can foresee the future and have preferences about how it turns out. I think an important part of morality is respecting these preferences, regardless of whether they involve pleasure or pain. So it is still wrong to kill or otherwise inconvenience a level 2 human, because they have many preferences about what they want to accomplish in life, and thwarting such preferences is just as bad, if not worse, then inflicting pain upon them.

It would even be fine to inflict small amounts of pain on a level 3 human if doing so will prevent a major life-goal of a level 2 human from being thwarted.

EDIT: Of course, I'm not saying that no other animals possess the ability to have preferences about the future. I'm sure a few do. But there are a great many that don't and I think that is an important distinction.

Comment by Ghatanathoah on Resurrection through simulation: questions of feasibility, desirability and some implications · 2013-11-30T02:26:43.328Z · LW · GW

"Clearly the better choice" is stating your conclusion rather than making an argument for it.

I assumed that the rest of what I wrote made it clear why I thought it was clearly the better choice.

There's an obvious reason for discounting the preferences of causally unconnected entities: if they really are causally unconnected, that means that they can't find out about our decisions

If that was the reason then people would feel the same about causally connected entities who can't find out about our decisions. But they don't. People generally consider it bad to spread rumors about people, even if they never find out. We also consider it immoral to ruin the reputation of dead people, even though we can't find out.

I think a better explanation for this intuition is simply that we have a bedrock moral principle to discount dissatisfied preferences unless they are about a person's own life. Parfit argues similarly here.

This principle also explains other intuitive reactions people have. For instance, in this problem given by Stephen Landsburg, people tend to think the rape victim has been harmed, but that McCrankypants and McMustardseed haven't been. This can be explained if we consider that the preference the victim had was about her life, whereas the preference of the other two wasn't.

Just as we discount preference violations on a personal level that aren't about someone's own life, so we can discount the existence of distant populations that do not impact the one we are a part of.

and that the extent to which their preferences are satisfied isn't therefore affected by anything that we do.

Just because someone never discovers their preference isn't satisfied, doesn't make it any less unsatisfied. Preferences are about desiring one world state over another, not about perception. If someone makes the world different then the way you want it to be then your preference is unsatisfied, even if you never find out.

Of course, as I said before, if said preference is not about one's own life in some way we can probably discount it.

It's not like the mind that we're seeking to recreate already exists within our part of the universe and has a preference for being (re-)created, while a novel mind that also has a preference for being (re-)created exists in some other part of the universe.

Yes it does, if you think four-dimensionally. The mind we're seeking to recreate exists in our universe's past, whereas the novel mind does not.

People sometimes take actions because a dead friend or relative would have wanted them to. We also take action to satisfy the preferences of people who are certain to exist in the future. This indicates that we do indeed continue to value preferences that aren't in existence at this very moment.

Comment by Ghatanathoah on Resurrection through simulation: questions of feasibility, desirability and some implications · 2013-11-30T02:04:55.242Z · LW · GW

Your wording suggests that I would assume the ITP, which would then imply rejecting the value of identity. But actually my reasoning goes in the other direction: since I don't find personal identity to correspond to anything fundamental, my rejection of it causes me to arrive at something ITP-like. But note that I would not say that my rejection of personal identity necessarily implies ITP: "the total amount of positive and negative experience is all that matters" is a much stronger claim than a mere "personal identity doesn't matter". I have only made the latter claim, not the former.

I have the same reductionist views of personal identity as you. I completely agree that it isn't ontologically fundamental or anything like that. The difference between us is that when you concluded it wasn't ontologically fundamental you stopped caring about it. I, by contrast, just replaced the symbol with what it stood for. I figured out what it was that we meant by "personal identity" and concluded that that was what I had really cared about all along.

That does provide a possible reason to prefer the child with the more ambitious preferences, if the net outcome for the world as a whole could be expected to be positive. But if it can't, then it seems obvious to me that we should prefer creating the non-ambitious child.

I can't agree with this. If I had the choice between a wireheaded child who lived a life of perfect passive bliss, or a child who spent their life scientifically studying nature (but lived a hermitlike existence so their discoveries wouldn't benefit others), I would pick the second child, even if they endured many hardships the wirehead would not. I would also prefer not to be wireheaded, even if the wireheaded me would have an easier life.

When considering creating people who have different life goals, my first objective is of course, making sure both of those people would live lives worth living. But if the answer is yes for both of them then my decision would be based primarily on whose life goals were more in line with my ideals about what humanity should try to be, rather than whose life would be easier.

I suppose I am advocating something like G.E. Moore's Ideal Utilitarianism, except instead of trying to maximize ideals directly I am advocating creating people who care about those ideals and then maximizing their utility.

Even if we accepted IPT, we would still have good reasons to prefer not killing existing people: namely that society works much better and with much lower levels of stress and fear if everyone has strong guarantees that society puts a high value on preserving their lives.

I agree, but I also think killing and replacing is wrong in principle.

I stopped consdering the Repugnant Conclusion a problem after reading John Maxwell's, Michael Sullivan's and Eliezer's comments to your "Mere Cable Channel Addition Paradox" post.

I did too, but then I realized I was making a mistake. I realized that the problem with the RC was in it's premises, not it's practicality. I ultimately realized that the Mere Addition Principle was false, and that that is what is wrong with the RC.

While this phrasing indeed doesn't make any mention of "persons", it still seems to me primarily motivated by a desire to create a moral theory based on persons.

No, it is motivated a desire to create a moral theory that accurately maps what I morally value, and I consider the types of relationships we commonly refer to as "personal identity" to be more morally valuable than pretty much anything. Would you rather I devise a moral theory based on stuff I didn't consider morally valuable?

If not, demanding the "link" criteria seems like an arbitrary decision.

You can make absolutely anything sound arbitrary if you use the right rhetoric. All you have to do is take the thing that I care about, find a category it shares with things I don't care about nearly as much, and then ask me why I am arbitrarily caring for one thing over the other even though they are in the same category.

For instance, I could say "Pain and pleasure are both brain states. It's ridiculously arbitrary to care about one brain state over another, when they are all just states that occur in your brain. You should be more inclusive and less arbitrary. Now please climb into that iron maiden."

I believe personal identity is one of the cornerstones of morality, whether you call it by that name, or replace the name with the things it stands for. I don't consider it arbitrary at all.

Comment by Ghatanathoah on Weak repugnant conclusion need not be so repugnant given fixed resources · 2013-11-30T01:33:04.253Z · LW · GW

Obviously the further we get away from familiar experiences the less reliable our intuitions are. But I think my intuition remains the same, even if the person in question is a hermit in some wilderness somewhere.

Comment by Ghatanathoah on Weak repugnant conclusion need not be so repugnant given fixed resources · 2013-11-30T01:25:36.645Z · LW · GW

Sure, it's fine. Glad I could help!