Morality Isn't Logical

Taboo both "morality" and "logical" and you may find that you and Eliezer have no disagreement.

LessWrongers routinely disagree on what is meant by "morality". If you think "morality" is ambiguous, then stipulate a meaning ('morality₁ is...') and carry on. If you think people's disagreement about the content of "morality" makes it gibberish, then denying that there are moral truths, or that those truths are "logical," will equally be gibberish. Eliezer's general practice is to reason carefully but informally with something in the neighborhood of our colloquial meanings of terms, when it's clear that we could stipulate a precise definition that adequately approximates what most people mean. Words like 'dog' and 'country' and 'number' and 'curry' and 'fairness' are fuzzy (if not outright ambiguous) in natural language, but we can construct more rigorous definitions that aren't completely semantically alien.

Surprisingly, we seem to be even less clear about what is meant by "logic". A logic, simply put, is a set of explicit rules for generating lines in a proof. And "logic," as a human practice, is the use and creation of such rules. But people informally speak of things as "logical" whenever they have a 'logicalish vibe,' i.e., whenever they involve especially rigorous abstract reasoning.

Eliezer's standard use of 'logical' takes the 'abstract' part of logicalish vibes and runs with them; he adopts the convention that sufficiently careful purely abstract reasoning (i.e., reasoning without reasoning about any particular spatiotemporal thing or pattern) is 'logical,' whereas reasoning about concrete things-in-the-world is 'physical.' Of course, in practice our reasoning is usually a mix of logical and physical; but Eliezer's convention gives us a heuristic for determining whether some x that we appeal to in reasoning is logical (i.e., abstract, nonspatial) or physical (i.e., concrete, spatially located). We can easily see that if the word 'fairness' denotes anything (i.e., it's not like 'unicorn' or 'square circle'), it must be denoting a logical/abstract sort of thingie, since fairness isn't somewhere. (Fairness, unlike houses and candy, does not decompose into quarks and electrons.)

By the same reasoning, it becomes clear that things like 'difficulty' and 'the average South African male' and 'the set of prime numbers' and 'the legal system of Switzerland' are not physical objects; there isn't any place where difficulty literally is, as though it were a Large Object hiding someplace just out of view. It's an abstraction (or, in EY's idiom, a 'logical' construct) our brains posit as a tool for thinking, in the same fundamental way that we posit numbers, sets, axioms, and possible worlds. The posits of literary theory are frequently 'logical' (i.e., abstract) in Eliezer's sense, when they have semantic candidates we can stipulate as having adequately precise characteristics. Eliezer's happy to be big-tent here, because he's doing domain-general epistemology and (meta)physics, not trying to lay out the precise distinctions between different fields in academia. And doing so highlights the important point that reasoning about what's moral is not categorically unlike reasoning about what's difficult, or what's a planet, or what's desirable, or what's common, or what's illegal; our natural-language lay-usage may underdetermine the answers to those questions, but there are much more rigorous formulations in the same semantic neighborhood that we can put to very good use.

So if we mostly just mean 'abstract' and 'concrete,' why talk about 'logical' and 'physical' at all? Well, I think EY is trying to constrain what sorts of abstract and concrete posits we take seriously. Various concepts of God, for instance, qualify as 'abstract' in the sense that they are not spatial; and psychic rays qualify as 'concrete' in the sense that they occur in specific places; but based on a variety of principles (e.g., 'abstract things have no causal power in their own right' and 'concrete things do not travel faster than light' or 'concrete things are not irreducibly "mental"'), he seeks to tersely rule out the less realistic spatial and non-spatial posits some people make, so that the epistemic grown-ups can have a more serious discussion amongst themselves.

There's a pseudo-theorem in math that is sometimes given to 1st year graduate students (at least in my case, 35 years ago), which is that

All natural numbers are interesting.

Natural numbers consist of {1, 2, 3, ...} -- actually a recent hot topic of conversation on LW ("natural numbers" is sometimes defined to include 0, but everything that follows will work either way).

The "proof" used the principle of mathematical induction (one version of which is):

If P(n) is true for n=1, and the assertion "m is the smallest integer such that !P(m)" leads to a contradiction, then P(n) is true for all natural numbers.

and also uses the fact (from the Peano construction of the natural numbers?) that every non-empty subset of natural numbers has a smallest element.

PROOF:

1 is interesting.

Suppose theorem is false. Then some number m is the smallest uninteresting number. But then wouldn't that be interesting?

Contradiction. QED.

The illustrates a pitfall of mixing (qualities that don't really belong in a mathematical statement) with (rigorous logic), and in general, if you take a quality that is not rigorously defined, and apply a sufficiently long train of logic to it, you are liable to "prove" nonsense.

(Note: the logic just applied is equivalent to P(1) => P(2) => P(3), ...which is infinite and hence long enough.)

It is my impression that certain contested (though "proven") assertions about economics suffer from this problem, and it's hard, for me at least, to think of a moral proposition that wouldn't risk this sort of pitfall.

For all we know, somebody trying to reason about a moral concept like "fairness" may just be taking a random walk as they move from one conclusion to another based on moral arguments they encounter or think up.

Well. Not a purely random walk. A weighted one.

Isn't this true of all beliefs? And isn't rationality just increasing the weight in the right direction?

The word "morality" needs to be made more specific for this discussion. One of the things you seem to be talking about is mental behavior that produces value judgments or their justifications. It's something human brains do, and we can in principle systematically study this human activity in detail, or abstractly describe humans as brain activity algorithms and study those algorithms. This characterization doesn't seem particularly interesting, as you might also describe mathematicians in this way, but this won't be anywhere close to an efficient route to learning about mathematics or describing what mathematics is.

"Logic" and "mathematics" are also somewhat vague in this context. In one sense, "mathematics" may refer to anything, as a way of considering things, which makes the characterization empty of content. In another sense, it's the study of the kinds of objects that mathematicians typically study, but in this sense it probably won't refer to things like activity of human brains or particular physical universes. "Logic" is more specific, it's a particular way of representing and processing mathematical ideas. It allows describing the things you are talking about and obtaining new information about them that wasn't explicit in the original description.

Morality in the FAI-relevant sense is a specification of what to do with the world, and as such it isn't concerned with human cognition. The question of the nature of morality in this sense is a question about ways of specifying what to do with the world. Such specification would need to be able to do at least these two things: (1) it needs to be given with much less explicit detail than what can be extracted from it when decisions about novel situations need to be made, which suggests that the study of logic might be relevant, and (2) it needs to be related to the world, which suggests that the study of physics might be relevant.

This question about the nature of morality is separate from the question of how to pinpoint the right specification of morality to use in a FAI, out of all possible specifications. The difficulty of finding the right morality seems mostly unrelated to describing what kind of thing morality is. If I put a note with a number written on it in a box, it might be perfectly accurate to say that the box contains a number, even though it might be impossible to say what that number is, precisely, and even if people aren't able to construct any interesting models of the unknown number.

People do all sorts of sloppy reasoning; everyday logic also arrives at both A and ~A ; any sort of fuzziness leads to that. To actually be moral, it is necessary that you can't arrive at both A and ~A at will - otherwise your morality provides no constraint.

In a recent post, Eliezer said "morality is logic"

The actual quote is:

morality is (and should be) logic, not physics

Eliezer said "morality is logic", by which he seems to mean... well, I'm still not exactly sure what, but one interpretation is that a person's cognition about morality can be described as an algorithm, and that algorithm can be studied using logical reasoning. (Which of course is true, but in that sense both math and literary criticism as well as every other subject of human study would be logic.)

Thank you -- I knew I ADBOCed with Eliezer's meta-ethics, but I had trouble putting down in words the reason.

You are not using the same definition of logic EY does. For him logic is everything that is not physics in his physics+logic (or territory+maps, in the previously popular terms) picture of the world. Mathematical logic is a tiny sliver of what he calls "logic". For comparison, in an instrumentalist description there are experiences+models, and EY's logic is roughly equivalent to "models" (maps, in the map-territory dualism), of which mathematics is but one.

With morality though, we have no such method,

Every act of lying is morally prohibited / This act would be a lie // This act is morally prohibited.

So here I have a bit of moral reasoning, the conclusion of which follows from the premises. The argument is valid, so if the premises are true, the conclusion can be considered proven. So given that I can give you valid proofs for moral conclusions, in what way is morality not logical?

doesn't have any of the nice properties of that a well-constructed system of logic would have, for example, consistency, validity, soundness...

The above example of moral reasoning (assume for the sake of simplicity that this is my entire moral system) is consistant, and valid, and (if you accept the premises) sound. Anyone who accepts the premises must accept the conclusion. One might waver on acceptance of the premises (this is true for every subject) but the conclusion follows from them regardless of what one's mood is.

All that said, our moral reasoning is often fraught. But I don't think makes morality peculiar. The mistakes we often make with regard to moral reasoning don't seem to be different in kind from the mistakes we make in, say, economics. Ethics, they say, is not an exact science.

I like this post, and here is some evidence supporting your fear that some people may over-use the morality=logic metaphor, i.e., copy too many anticipations about how logical reasoning works over to their anticipations about how moral reasoning works... The comment is already downvoted to -2, suggesting the community realizes this (please don't downvote it further so as to over-punish the author), but the fact that someone made it is evidence that your point here is valuable one.

http://lesswrong.com/lw/g0e/narrative_selfimage_and_selfcommunication/83ag

The practice of moral philosophy doesn't much resemble the practice of mathematics. Mainly because in moral philosophy we don't know exactly what we're talking about when we talk about morality. In mathematics, particularly since the 20th century, we can eventually precisely specify what we mean by a mathematical object, in terms of sets.

"Morality is logic" means that when we talk about morality we are talking about a mathematical object. The fact that the only place in our mind the reference to this object is stored is our intuition is what makes moral philosophy so difficult and non-logicy. In practice you can't write down a complete syntactic description of morality, so in general¹ neither can you write syntactic proofs of theorems about morality. This is not to say that such descriptions or proofs do not exist!

In practice moral philosophy proceeds by a kind of probabilistic reasoning, which might be analogized to the thinking that leads one to conjecture that P≠NP, except with even less rigor. I'd expect that things like the order of moral arguments mattering come down to framing effects and other biases which are always involved regardless of the subject, but don't show up in mathematics so much because proofs leave little wiggle room.

¹ Of course, you may be able to write proofs that only use simple properties that you can be fairly sure hold of morality without knowing its full description, but such properties are usually either quite boring or not widely agreed upon or don't lead to interesting proofs due to being too specific. eg. "It's wrong to kill someone without their permission when there's nothing to be gained by it."

I think the term you are looking for is "formal" or "an algebra", not "logic".

You're mischaracterizing the quote that your post replies to. EY claims that he is attempting to comprehend morality as a logical, not a physical thing, and he's trying to convince readers to do the same. You're evidently thinking of morality as a physical thing, something essentially derived from the observation of brains. You're restating the position his post responds to, without strengthening it.