Being Moral is an end goal.

post by Yair Halberstadt (yair-halberstadt) · 2022-03-09T16:37:15.612Z · LW · GW · 17 comments

Contents

17 comments

Alice wants to go to her best friends party. She has a sore throat, and she usually takes a COVID test when she has a sore throat. But if the test were to come out positive she would feel like she shouldn't go, and she really really wants to go. Solution - she avoids taking the test, leaving her probably COVID free, and ready to party.

I've heard this story in about a thousand variations since COVID restrictions began.

Analysed from a rational perspective this doesn't seem to make much sense.

Let's say that Alice thinks she a 10% chance of having COVID. Tests are perfectly accurate. Alice care about other people, so going to a party whilst infected is worth negative 50 utils. Going to a party is otherwise worth 10 utils.

Then expected utility if Alice takes the test is 0.9 * 10 + 0.1 * 0 = 9.

Expected utility if she doesn't take the test is 0.9 * 10 + 0.1 * -50 = 5.

So taking the test leaves her better off than not taking the test.

This is a special case of the general rule that knowing more information should never be a negative for a rational agent.

So what's going on here?

The answer is that Alice doesn't actually care about other people. Going to a party whilst infectious is worth just as many utils to her as going to a party whilst COVID free.

What Alice cares about is being a moral person. And moral people don't go to parties whilst knowing they're infectious. But moral people do go to parties whilst they might have an infection. So if she avoids finding out if she infectious she gets a guaranteed 10 utils at 0 cost.

One way to model this is that a moral action is one that somebody who genuinely cares about other people will do. We don't genuinely care about other people, but we care about doing moral actions. So we make the minimal changes from pure moral action so that we can mostly take moral actions, whilst still mostly just doing whatever we want. By making the slightly immoral action of not doing a COVID test, we avoid having to make the very immoral action of going to a party whilst infected.

Note that I don't want to claim people don't care about other people at all. I'm reducing the motives in this particular case to a simplified human with only a single motivation, and exploring that particular facet of our motives, but as ever, people are complex and other scenarios can best be explained by assuming that people do genuinely care for others.

17 comments

Comments sorted by top scores.

comment by localdeity · 2022-03-09T18:35:55.365Z · LW(p) · GW(p)

But moral people do go to parties whilst they might have an infection.

Speak for yourself.  Sure, in a vacuous sense, anyone might have an infection at any time, but what we're talking about here is "whilst they have good reason to think they might have an infection".  For me, even before COVID, I've had a general policy of not seeing people when I'm sick, and stayed home from some parties I otherwise would have enjoyed.  At least for me, a sore throat would be rather unusual if I weren't sick, so that would normally be enough evidence for me (unless it was so slight I wasn't sure it was a real symptom; or if I'd overused my voice or had some other non-sickness plausible cause).  If I had a magical test that was free and always told me correctly whether I had an infectious disease, that would be wonderful and I'd use it a lot.

A sensible policy would take into account your estimated probability that you're infectious, and the number of people you'd likely infect.  Also the expected enjoyment you'd get from the party, and the expected negativity an infection would inflict on other people.  (Note that there are circumstances, such as Omicron COVID, where one might reasonably say "If they don't catch it from me, there's still a >80% chance they'll catch it elsewhere", which may downgrade the "expected negativity" estimate.)  Maybe a scaling factor for how you care less about other people than about yourself—I think it's reasonable to have that, and for "people you'd encounter at a party" I think it should normally be between 0 and 1.  I'm not exactly a utilitarian, but I would say that, among friends and people you cooperate with, it generally makes the most sense for policies to be based on utilitarian calculations.

You say that a bunch of people do in fact behave in the manner you've described.  I can only chalk this up to "a bunch of people are disappointing".  Or—along the lines of what TAG says, in this particular case, I suspect that these people have encountered official rules that say "if you test positive, then you must stay home"—and while I'm not sure how common it is for party hosts to explicitly state such rules, these people have internalized the rule and are trying to obey it; but I would say that either (a) they don't fundamentally care about not infecting other people, only about not disobeying those rules (and getting caught?), or (b) they haven't thought things through.  Either seems plausible to me.

Or, I dunno.  Could they have been confident they don't have COVID but worried about a false positive?  Doesn't sound like it, though.

Replies from: yair-halberstadt
comment by Yair Halberstadt (yair-halberstadt) · 2022-03-09T19:13:45.175Z · LW(p) · GW(p)

You say that a bunch of people do in fact behave in the manner you've described. I can only chalk this up to "a bunch of people are disappointing".

For perspective, at least 3 of these people are first degree relatives of me or my wife, at least 2 of those very much pride themselves on being very moral and law abiding. This isn't a cherry picked sample. This is extremely common (when the stakes are high enough), and cannot be dismissed so easily.

Replies from: localdeity
comment by localdeity · 2022-03-10T00:57:44.790Z · LW(p) · GW(p)

I'm not saying that what you describe is rare, just that it disappoints me.

I don't particularly intend to criticize your and your wife's relatives, but I'm afraid that just because a person prides themselves on being very moral and law-abiding doesn't mean that I'll consider them moral by my standards, or that I won't find them disappointing.  I think there are lots of, say, political issues where I can find several tens of percent of the population, sometimes more than 50%, advocating a policy I consider disastrous as well as immoral.  I think most people are not very developed in their moral reasoning (by my standards).

For a relatively non-political example, Jonathan Haidt went around asking people about a hypothetical case of incest—between two consenting adults, in a country where it's legal, taking double precautions against pregnancy—and apparently most people say "this is definitely wrong" while being unable to articulate why (or only coming up with reasons that the scenario rules out).

Google's dictionary definitions for "moral" say "concerned with the principles of right and wrong behavior and the goodness or badness of human character" and "holding or manifesting high principles for proper conduct".  I think that any serious thinker concerned with "principles" here (as opposed to arbitrary isolated rules) would realize that there's something wrong when "moral" people start avoiding information so that they won't feel compelled to act on it.  It follows that these people are not serious thinkers concerned with "principles"; this is a conclusion I unhappily accept.

If I were writing your post, I would put scare quotes around the word "moral" (to avoid claiming that Alice's idea of morality is universally held), thus:

What Alice cares about is being a "moral" person.  And "moral" people don't go to parties whilst knowing they're infectious. But "moral" people do go to parties whilst they might have an infection.

I see you've retitled the post to use "Moral" as a proper noun, which is another approach.

 

Though I would say this: "When the stakes are high enough" sounds like more than just a friend's party.  For example: attending your child's wedding; giving a talk at a conference that you think would be important for your career.  These are probably big enough events that I'd suspect there's a person declaring COVID rules—imposing them on everyone else—who might not respond well to "Yeah, I tested positive, but it's really important for me to be there and I'll try to be careful, so please let me in".

The question I would ask: Do these people agree that, if they had COVID, then they should stay home from these events?  Because if they don't, then it seems like a simple case of "I disagree with this policy so I'm not going to help implement it if no one makes me" or "Given that I'm going to attend anyway, if I did test positive, someone might find out and punish me, and I'd rather not risk that".  If they do, and if it's not about false positives or the cost of testing or whatever (which your premise "she usually takes a COVID test when she has a sore throat" seems to cover), then this becomes more of a case of hypocrisy.

The etymology of "hypocrisy" is interesting.  "Hypo" means "low" (e.g. "hypothermia"), and the "crisy" seems related to "criticism"; thus I take it to mean "insufficiently critical (of one's actions or beliefs)".  And that is the meaning I intend above.

Yet I see claims on the internet that "hypocrite" (or its Greek origin word) meant "stage actor":

It literally translates as “an interpreter from underneath” which reflects that ancient Greek actors wore masks and the actor spoke from underneath that mask. Eventually the Greek word evolved to refer to any person who was wearing a figurative mask and pretending to be someone or something they were not.

Which, I suppose, is the surface-level behavior.

Replies from: yair-halberstadt
comment by Yair Halberstadt (yair-halberstadt) · 2022-03-10T05:19:57.743Z · LW(p) · GW(p)

The question I would ask: Do these people agree that, if they had COVID, then they should stay home from these events?

Yes

comment by Shmi (shminux) · 2022-03-09T18:54:21.238Z · LW(p) · GW(p)

The general pattern of avoiding potentially hedonically harmful knowledge is indeed widespread. This includes your example of "this particular action I want to take could be immoral if I could evaluate its consequences". Other examples are the general flinching from things like checking your weight or sugar or going to see a doctor, or paying bills or...

Replies from: Viliam
comment by Viliam · 2022-03-11T12:47:25.362Z · LW(p) · GW(p)

Which is a general problem of learning by reinforcement. Punishing X not only reduces X but also everything related to X, including "thinking about the possibility of X".

comment by Dagon · 2022-03-09T16:49:53.519Z · LW(p) · GW(p)

Giving a made-up (though believable) story does not justify expanding to what "we" care about.  Ways in which humans are irrational and self-deceiving are many, but that doesn't mean they don't ALSO care about others' feelings and experiences.

Replies from: yair-halberstadt, yair-halberstadt
comment by Yair Halberstadt (yair-halberstadt) · 2022-03-09T17:03:53.673Z · LW(p) · GW(p)

Added this section:

Note that I don't want to claim people don't care about other people at all. I'm reducing the motives in this particular case to a simplified human with only a single motivation, and exploring that particular facet of our motives, but as ever, people are complex and other scenarios can best be explained by assuming that people do genuinely care for others.

Replies from: Dagon
comment by Dagon · 2022-03-09T19:23:45.649Z · LW(p) · GW(p)

Thanks for the edit, but I'm still uncomfortable in pointing out that people often act in immoral and harmful ways, and extending that to a principle of "what people care about" without recognizing the complexity of decision-making and the balance of motives that include factors of public image, peer-group reaction, self-image, and intellectual morality.

Replies from: romeostevensit, tomcatfish
comment by romeostevensit · 2022-03-09T19:34:18.091Z · LW(p) · GW(p)

Agree because morality doesn't refer to a single level of abstraction. I don't necessarily hold people responsible for when their decision theory doesn't reach all the way up to platonic forms of cooperation because most people haven't had that modeled for them such that they can do it. More colloquially, people lie because they are afraid. They are afraid because their ability to meet their basic needs feels fragile. This feeling of fragility is only loosely entangled with reality and much more entangled with a limited range of strategies they know about.

comment by Alex Vermillion (tomcatfish) · 2022-03-11T17:44:50.471Z · LW(p) · GW(p)

If I may, I agree with this through an idea in the vein of "Humans Are Not Automatically Strategic" [LW · GW]. Seeing someone do something that runs counter to their goals isn't proof they don't have those goals; often it's simply a demonstration that you have to do a lot of things correctly to win.

comment by Yair Halberstadt (yair-halberstadt) · 2022-03-09T17:00:18.142Z · LW(p) · GW(p)

This isn't made up. It's an amalgamation of many things people have told me, details changed for privacy.

comment by Idan Arye · 2022-03-09T18:22:57.937Z · LW(p) · GW(p)

Doesn't society already consider it immoral to go to crowded places untested when you suspect you have COVID? This is not just about a specific detail of this specific story - one important feature is morality is preventing humans from convincing themselves that the thing they want to do is the utilitarian choice [? · GW]. We decided that going untested is immoral precisely because people like Alice who avoid testing themselves for such reasons.

Instead of morality, I think what Alice seeks here is deniability. If Alice does not take the test, she can convince herself (and possibly others?) that the probability of her sore throat indicating COVID is as low as she wants. No one else can really tell how bad it was - certainly not days later, when the thing is discovered - she may even claim that she felt nothing unusual. She is still immoral, but she can at least convince herself that she has done nothing wrong.

comment by Pattern · 2022-03-13T01:28:04.054Z · LW(p) · GW(p)

You don't just get -50 if you are sick and you go. It's also negative if you go and someone else who is sick goes.

comment by mukashi (adrian-arellano-davin) · 2022-03-09T21:21:59.467Z · LW(p) · GW(p)

This is a very common behaviour, I agree, but I am not really sure the simplified explanation is useful for understanding anything (I know you are specifically pointing out in the last paragraph that you are perfectly aware that your explanation is a simplification). 

I think unfolding things a bit we get a much better idea of what's really going on:

  1. Alice is a human being, with a mind composed of a society of competing modules
  2. Some of her modules do care about being moral. Some others care about having fun. Some others care about people. Some modules agree with each other, some others contradict each other. 
  3. The society of modules carries some sort of internal voting, that results in Alice taking an action (e.g. going to the party). Not all modules have always the same strength. They are heavily influenced not only by external factors but also internal factors, like hormonal levels
  4. One specific module makes a narrative about why it is justified going to the party and that's the correct thing to do

Reducing Alice to a single entity is a convenient and yet wrong way of thinking of people and motivations

comment by TAG · 2022-03-09T17:25:59.922Z · LW(p) · GW(p)

People care about breaking rules because there are punishments for breaking rules , both formal and informal. If people were unselfishly rational, and moral as a result of being unselfishly rational, there would be no need for punishments to shape behaviour. As it is, rules need to be formulated on the basis that the rule alone generates the desired behaviour, not on the basis that the individual gets the point of the rule.

Replies from: Dagon
comment by Dagon · 2022-03-09T18:55:02.632Z · LW(p) · GW(p)

Include unwritten rules and illegible punishments (changes in the way people interact with you), and then extend to self-modeling (the way you see yourself, as a proxy and template for the way you want others to see you, evolved into a root desire to respect yourself through a mix of behavior and self- and other-deception (or if not active deception, at least framing control)).  Then you have at least some indication of the complexity of "cares about" and a way to avoid simple statements about what people "really" believe.