Unspeakable Morality

Just give up already

I cannot say how many arguments I've had where this would have prevented hurt feelings. Often, after the argument, I discover the other person persisted in arguing for about 10 minutes after they realized they were wrong, all the while getting more angry at me for shooting down ever worse rationalizations.

To be fair, the way this happens isn't that the person persists in arguing for something they know to be false; instead, they drop a subtle hint that maybe they might be wrong and we should stop talking about it now (presumably so they can save face). I invariably miss this hint (well, I'm better now that I know to be looking for it, but not a lot) because it's usually in the form of a ridiculous but hard to disprove objection, to which I (because I'm weird) will come up with a medium-good response. This pisses my interlocutor off, because I missed their social cue, and because I've now forced them to defend a belief (their lousy objection) that they don't actually hold.

This behavior is very understandable; once I noticed others doing it, I noticed a tendency in myself. It's surprisingly hard to say, "Oops, I guess I'm wrong," or, "I can't see a good counter argument to what you're saying; maybe I need to reconsider."

Anyway, I'm saying this because the article linked by the quoted phrase wasn't quite what I was hoping for on the subject. :)

The Story of Bob does have an adequate answer in the "vocabulary of harms". The implicit claim that it does not echoes claims of Jonathan Haidt in much of his work on morality, especially his "five pillars" theory and subsequent extrapolations which have been eagerly seized upon by conservative proponents as evidence that liberals are narrow-minded.

It therefore irritates me a great deal when I see such claims going unchallenged, despite their (to me, anyway) obvious inaccuracy.

Here, then, is my "harm/care-based moral system" take on The Sacrifice of Bob:

I'm going to presume that the fictional culture in the story is reasonably happy and prosperous, otherwise we would have been talking about how terrible their culture is even before the sacrifice had taken place.

• (1) Given that this fictional culture and its president have a good track record at keeping the peace (in diametric opposition to certain other presidents whose similar but much less moral actions are implicitly being referred to), and that Bob's sacrifice probably saved millions of lives, there is nothing wrong with the President's action -- certainly nothing worse than that of a general sending a soldier into a known death trap.
• (2) Given that this culture seems to achieve goals we value (reasonably happy, prosperous people) while using morals we find questionable (highly authoritarian), I should think that we would want to study them intently to see how they do it. Perhaps we can learn some things -- or perhaps the appearance of happiness and prosperity will turn out to have hidden costs.

Having now played a turn by the rules, I have a bone to pick with the basic concept.

The problem with such examples is that you are basing an implicit conclusion -- "harm-based morality is limited" -- on a fiction, a lie. It's a complex form of circular reasoning: "Imagine a world in which this particular form of morality inexplicably produces positive results. Don't you feel silly trying to defend your morality now?" or even "Imagine a universe in which the earth is a flat disc riding on the back of 4 elephants. How can you stick to your narrow-minded idea that Earth is a spheroid orbiting the sun when in some other universe it might not be?"

All that said, I think I agree with the ground ground rules you propose at the end. I have always said that inability to explain one's reasoning doesn't prove one wrong, and that science needs to pay more attention to intuition (reasoning based on summed data for which complete records were not kept) -- but there do need to be guidelines, because intuition should no more trump science unilaterally than the other way round.

since most of them have incoherent explicit metaethics

Is there a coherent metaethical theory specified in a single document somewhere on the Internet? Or does the theory have to be compiled from multiple blog posts? I guess I'm not sure what you're talking about...

I'm curious about the opening line: It is a general and primary principle of rationality, that we should not ... enforce upon our fellows a law which there is insufficient justification to enforce.

On my ordinary understanding of the sentence, it seems to imply that acting justly is necessarily part of what Eliezer means by "acting rationally". Is this right?

More explicitly: the implication is that refraining from "enforcing insufficiently justified laws" is a "general and primary" principle of rationality. Perhaps what is meant is some tautology deriving from (Eliezer's) meanings of "fellows", "justification", and "should". Or perhaps it's just a rhetorical flourish?

I'm going to comment on the presentation (I have no objection to the ideas):

This piece pleased me immensely in the beginning. The "F" grade anecdote was perfect and actually enraged me upon later reflection. In spite of that, I don't feel the persuasion was manipulative.

The reiteration of the linked ideas was good. I especially liked the story of the irrational gambler who could be used as money pump.

The material up to and including the Greene dilemma was good.

Following that, however, is a painfully verbose recapitulation of the opening "I've always felt a bit nervous about demanding that people be able to explain things in words, because, while I happen to be pretty good at that, most people aren't.".

Then we have some clever self-hedging ("shared moral beliefs might suggest an underlying reason, even if there's no satisfactory exposition", "but even unreasonable beliefs sometimes get a seemingly-convincing apology!") that looks like preparation for a debate. Why not let your readers have those thoughts on their own? They just serve to annoy those who are capable of having them, and confuse those who aren't. There's no need to claim priority on such thoughts; let them arise in the comments (post them yourself, even).

Or, in other words, considerations that occurred to you, but don't substantially change the final judgment, can often be omitted or relegated to a comment/footnote.

Finally, the rules. I agree with them. I think it's okay to be as verbose as you are, because there's real danger in having people blindly follow if you're unclear. The arguments for them have already been made (or are obvious). I didn't feel a need to read past the first sentence of each rule, but it's good that detail is there.

The final rule is more hedging like I already complained about. I guess you imagine some of your readers as automatons or idiots :) Or, more likely, you're preparing yourself for expected objections.

I think it's good that you consider the consequences of your ideas, possible misunderstanding of them, and have rebuttals in advance.

I'd like to propose a new form for you:

1. First, all the great stuff. Really original ideas without repetition, plus particularly clever demonstrations or evocative examples/problems. All the formalization, you have, too. Essentially, everything that would inform and please a not-obnoxious and competent person who has no fundamental disconnect from you.

2. A separator to be understood as "you can stop now if you understood everything and are impatient".

3. Now, all the stuff I complained about earlier. Some kind of linking or numbering might be necessary to get the same context as the usual inline form.

Maybe that's too much work. But as a reader, I often find that the required investment to be off-putting. If you don't want "skim and then socialize in the comments" happening, it might be worth the effort, though. (I do think most comments by regulars indicate that they really read the article they comment on).

I see a lot of people in the comments to this post talking about divorcing the "damage report" aspect of pain from the "unpleasant experience" aspect. This won't work -- at least, not in naive form. Emotions are what motivate people to do things. If you take away the negative emotional impact of pain without some kind of situational fine-tuning, you might as well remove pain entirely. Those people with pain asymbolia that I mentioned previously are indifferent to the threat of harm.

I have a question for lesswrong readers. Please excuse any awkwardness in phrasing or diction--I am not formally trained in philosophy. What do you consider to be the "self"? Your physical body, your subconscious and conscious processes combined, consciousness, or something else? Also, do you consider your "past selves" and "future selves" to be part of a whole with your "present self," and to what extent? For an example of why the distinction might be important, let's say that one night, you sleepwalk and steal a thousand dollars. This is the first time something like this has happened. Of course, society will hold you accountable for your actions, but how will you assign the blame in your head? E.g. "I'm such a horrible person" vs. "That pesky subconscious! Always up to no good!" Another example is this. You are offered a deal in which you will live in earthly paradise for 10 years, but at the end of that ten years, you will be tortured in way so that your future self will regret accepting. Do you accept the deal? You might, if you can consider your future self in ten years to be a "different person" than your current self.

and where it came from (evolution? culture?)

Once you know a source, how do you know whether it makes your morals more or less likely to be true? A lazy thinker would just wind up engaging in Bulverism.

only at the point of explaining why pain is a bad thing

Would you adapt that if speaking into the chronophone to a time before negative-utilitarianism?

All moral codes drill down to a rocky core of "ick," though. Suppose A says, "Well it's clearly wrong." And C says, "No, it's not. Make your case." The case is made when A says, "B inevitably leads to D. Does D make you feel icky?" and C says, "It does."

It's true that people in the past had a lot of icky feelings we don't have today. We also have a lot of icky feelings they didn't have. Given that, I would like to see a follow-up article written about, under this framework, how many more letters of the alphabet have to agree with A before A gets to punish C for making him feel icky, depending on the number of letters in agreement, how severe the ick, that kind of thing.

It is a general and primary principle of rationality, that we should not believe that which there is insufficient reason to believe, nor enforce upon our fellows a law which there is insufficient justification to enforce.

Is that a principle, or a tautology?

Nonetheless, I've always felt a bit nervous about demanding that people be able to explain things in words, because, while I happen to be pretty good at that, most people aren't.

I was thinking just yesterday that this is a problem in communications between men and women. Women know a lot of things that they can't explain. Here's a conversation you might overhear between a man and a woman:

She: "Look at that guy. He just doesn't see that she isn't into him."

He: "Huh?"

She: "Well, look at her. Look at how she looks at him. Look at how at how she's sitting."

He: "What about how she looks at him? How does she look at him?"

She: "Just look. It's obvious."

He thinks that She doesn't know what she's talking about, because She can't explain it. But somehow it often turns out that She is right.

So if A says that event B is inherently wrong and awful, and C disagrees on the grounds that it just doesn't seem all that awful to them, then the burden of argument needs to lie on A before any social, legal, public action is brought into play.

Is that the main point of your post, or was there something more? I'm looking at your 4 bullet points, and I agree with them, but they aren't novel or even controversial.

I felt that all of the points made could be said in a post about half the length. The points are something I have noticed, do not need to be persuaded of, and put into better words than I could have before (which illustrates a main point of the post) but I feel that they could persuade a sceptical reader, even if rewritten at half the length.

If you don't mind me asking... How does anyone who believes that pain and/or death is bad NOT conclude that pepperoni pizza is bad?