Metaethics: Where I'm Headed

post by lukeprog · 2011-11-22T08:15:23.901Z · LW · GW · Legacy · 11 comments

Contents

11 comments

In an earlier post, I explained that Pluralistic Moral Reductionism finished my application of the basic lessons of the first four sequences to moral philosophy. I also explained that my next task was to fill in some inferential distances by summarizing lots more cognitive science for LessWrong (e.g. Neuroscience of Human Motivation, Concepts Don't Work That Way).

Progress has been slow because I'm simultaneously working on many other projects. But I might as well let ya'll know where I'm headed:

 

"Philosophy for Humans, 2: Living Metaphorically" (cogsci summary)

Human concepts and human thought are thoroughly metaphorical, and this has significant consequences for philosophical methodology. (A summary of the literature reviewed in chs. 4-5 of Philosophy in the Flesh.)

 

"Philosophy for Humans, 3: Concepts Aren't Shared That Way (cogsci summary)

Concepts are not shared between humans in the way required to justify some common philosophical practices. (A summary of the literature reviewed in several chapters of The Making of Human Concepts, Mahon & Caramazza 2009, and Kourtzi & Connor 2011.)

 

"The Making of a Moral Judgment" (cogsci summary)

A summary of the emerging consensus view on how moral judgments are formed (e.g. see Cushman et al. 2010).

 

"Habits and Goals" (cogsci summary)

A sequel to Neuroscience of Human Motivation that explains the (at least) three different systems that feed into the final choice mechanism that encodes expected utility and so on. (A summary of the literature reviewed in chapter 2 of Neuroscience of Preference and Choice.)

 

"Where Value Comes From" (metaethics main sequence)

The typical approach to metaethics analyzes the meanings of value terms, e.g. the meaning of "good" or the meaning of "right." Given persistent and motivated disagreement and confusion over the "meanings" of our value concepts (which are metaphorical and not shared between humans), I prefer to taboo and reduce value terms. To explain value in a naturalistic universe, I like to tell the story of where value comes from. Our universe evolved for hundreds of thousands of years before the atom was built, and it existed for billions of years before value was built. Just like the atom, value is not necessary or eternal. Like the atom, it is made of smaller parts. And as with the atom, that is what makes value real.

 

"The Great Chasm of the Robot's Rebellion" (metaethics main sequence)

We are robots built for replicating genes, but waking up to this fact gives us the chance to rebel against our genes and consciously pursue explicit goals. Alas, when we ask "What do I want?" and look inside, we don't find any utility function to maximize (see 'Habits and Goals', 'Where Value Comes From'). There is a Great Chasm from the spaghetti code that produces human behavior to a utility function that represents what we "want." Luckily, we've spent several decades developing tools that may help us cross this great chasm: the tools of value extraction ('choice modeling' in economics, 'preference elicitation' in AI) and value extrapolation (known to philosophers as 'full information' or 'ideal preference' theories of value).

 

"Value Extraction" (metaethics main sequence)

A summary of the literature on choice modeling and preference elicitation, with suggestions for where to push on the boundaries of what is currently known to make these fields useful for metaethics rather than for their current, narrow applications.

 

"Value Extrapolation" (metaethics main sequence)

A summary of the literature on value extrapolation, showing mostly negative results (extrapolation algorithms that won't work), with a preliminary exploration of value extrapolation methods that might work.

 

After this, there are many places I could go, and I'm not sure which I'll choose.

11 comments

Comments sorted by top scores.