Posts
Comments
Right so you're worried about moral hazard generated by insurance (in the case where we have liability in place). For starters, the government arguably generates moral hazard for disasters of a certain size by default: it can't credibly commit ex ante to not bail out a critical economic sector or not provide relief to victims in the event of a major disaster: the government is always implicitly on the hook (see Moss, D. A. When All Else Fails: Government as the Ultimate Risk Manager. See the too-big-to-fail effect for an example). Charging a risk-priced premium for that service can only help.
But you're probably more worried about private insurer's ability to mitigate the moral hazard they generate. Insurers certainly do not always succeed at doing this. However, they sometimes not only succeed, but in fact induce more harm reduction than liability alone probably would have induced on its own (see e.g. the Insurance Institute for Highway Safety rating crashworthiness of vehicles, and the auto-industry's lobbying for airbags in the 80s). For more see:
- Ben-Shahar and Logue, "Outsourcing Regulation: How Insurance Reduces Moral Hazard"
- Abraham and Schwarcz, "The Limits of Regulation by Insurance"
My research finds that, in the specific context of insuring against uncertain heavy-tail risks we can expect private insurers to engage in a mix of:
- causal risk-modeling, because actuarial data will be insufficient for such rare events (cf. causal risk-modeling in nuclear insurance underwriting and premium pricing (Mustafa, 2017)(Gudgel, 2022, ch. 4 sec. VII)).
- monitoring, again due to a lack of actuarial data and the need to reduce information asymmetries (cf. regular inspections by nuclear insurers with specialized engineers (2022, ch. 4 sec. VI.C)).
- safety research and lobbying for stricter regulation, because insurers will almost certainly have to pool their capacity in order to offer coverage, eliminating competition and with it, coordination problems (cf. American Nuclear Insurer’s (ANI) monopoly on third party liability (2022, ch. 4 sec. VII.A)).
- private loss prevention guidance, because it can’t be appropriated or drive away customers here: there will be little competition and the insurance is mandatory (cf. ANI sharing inspection reports and recommendations with policyholders (2022, ch. 4 sec. VII.A.2)).
In other words, if set up correctly, I expect them to do all the things we would want them to do.
The government also needn't mandate specifically commercial insurance: it could also allow/encourage labs to mutualize their risk, entirely eliminating concerns about moral hazard.
You can read more about all of this here.
Ah, I see now! Thanks for the clarifications!
My apologies if this is dumb but if when a model linearly represents features a and b it automatically linearly represents and , then why wouldn't it automatically (i.e. without using up more model capacity) also linearly represent ? After all, is equivalent to , which is equivalent to .
In general, since { , } is truth functionally complete, if a and b are represented linearly won't the model have a linear representation of every expression of first order logic (without quantifiers) that involve only a and b?
(If I'm missing something dumb, feel free to ignore this post).
There is an equivocation going on in the post that bothers me. Mot is at first the deity of lack of technology, where "technology" is characterized with the usual examples of hardware (wheels, skyscrapers, phones) and wetware (vaccines, pesticides). Call this, for lack of a better term, "hard technology". Later however, "technology" is broadened to include what I'll call "social technologies" – LLCs, constitutions, markets etc. One could also put in here voting systems (not voting machines, but e.g. first-past-the-post vs approval), PR campaigns, myths. Social technologies are those that coordinate our behaviour, for ill or good. They can be designed to avoid coordination failures (equilibria no individual wants), or to maximize profits, to maintain an equilibria only a minority wants etc. (The distinction between social and hard tech obviously has blurry boundaries – you'll notice I didn't mention IT because much of it is on the border between the two. But somewhat blurry boundaries doesn't automatically threaten a distinction).
Broadening the definition is fine, but then stick to it. You swing between the two when, e.g. you claim that:
boosting technological progress is far more actionable than increasing coordination
This claim only makes sense if "technology" here excludes social tech. (By the way I'd love to see the numbers on this. I'm pretty skeptical. I'd be convinced of this claim when I see us allocate to e.g. voting reform, the kind of capital we're allocating to e.g. nuclear fusion. But of course capital allocation processes are... coordination processes. More on this next.)
I fully agree that coordination failures can be thought of as a type of technological failure (solving them requires all the technical prowess that other "hard" disciplines require). But they're a pretty distinct class of failure, and a distinctly important one for this reason: coordination failures tend to be upstream of other technological failures. What technology we build depends on how we are coordinate. If we coordinate "well" we get the technologies (hard or social) we all want (+when/what order we want them), and none of the ones we don't want (on some account of what "we" refers to – BIG questions of justice/fairness here, set aside for another day).
I'm also bothered by some normative-descriptive ambiguity in the use of some terms. For example, technological progress is treated as some uni-directional thing here. This is plausibly (though not obviously) true if "progress" here is used normatively. It's definitely false if "progress" here is synonymous with something purely descriptive like "development" or better yet, "change." If technological development were uni-directional, you'd be hard pressed to account for good and bad technological developments. For such normative judgments to make any sense, there arguably had to be alternative developments we could have pursued, or at least developments we could have chosen not to pursue. See Acemoglu and Johnson for better understanding directions of technological development. Their work also provides examples of the additional predictive power gained by including the direction of technological development into one's models. Wheels? Net good. Engagement-maximizing social media? Arguably net bad. Here is an alternative framing (easier for libertarian-leaning folks to swallow): regulation is a coordination technology. Effective regulation directs technological development toward the outcomes the regulator wishes for; good regulation directs development toward good outcomes. The two are obviously not always the same. The social engineer (e.g. mechanism designer) tackles the question of how to create effective regulation; the political philosopher tackles what kind of development should be pursued. (And while the political philosopher is working things out, democracy has been determined as a decent placeholder director).
Another normative-descriptive ambiguity that bothered me: the use of "coordination." In "coordination failure" "coordination" is at least weakly normative: the phrase describes a situation in which agents failed to behave collectively in the manner that would have maximized welfare – they failed to behave the way they should have in some sense. They didn't behave ideally. "Coordination" has a purely descriptive use too though, as meaning something like "patterns in the collective behaviour of agents." I'll italicize the normative use. An instance of coordination failure can also be an instance of coordination. For example, in Scott Alexander's fishermen story (example 3 in Meditations on Moloch), the agents' actions are coordinated by a market that failed to internalize the cost of pollution, and this results in a coordination failure. When you say that:
you could see Vegas as a product of the miraculous coordination technology that is modern capitalism—perhaps an edge case of it, but still an example of its brilliance.
I think there is an equivocation between coordination and coordination going on. Vegas is absolutely a fascinating example of capitalism's coordinating power, much like the peacock's feather is a fascinating example of sexual selection's coordinating power. But are either of these successful coordination? Much harder to say. (Not sure how to even begin answering the normative question in the peacock case). EDIT: Punchier example: the Stanford Prison Experiment is another fascinating example of role-playing's power to coordinate behaviour, but it sure doesn't seem like an example of successful coordination.
For posterity, we discussed in-person, and both (afaict) took the following to be clear predictive disagreements between the (paradigmatic) naturalist realists and anti-realists (condensed for brevity here, to the point of really being more of a mnemonic device):
Realists claim that:
- (No Special Semantics): Our use of "right" and "wrong" are picking up, respectively, on what would be appropriately called the rightness and wrongness features in the world.
- (Non-subjectivism/non-relativism): These features are largely independent of any particular homo sapiens attitudes and very stable over time.
- (Still Learning): We collectively haven't fully learned these features yet – the sparsity of the world does support and can guide further refinement of our collective usage of moral terms should we collectively wish to generalize better at identifying the presence of said features. This is the claim that leads to claims of there being a "moral attractor."
Anti-realists may or may not disagree with (1) depending on how they cash out their semantics, but they almost certainly disagree with something like (2) and (3) (at least in their meta-ethical moments).
I appreciate the comment – kinda gives me closure! I knew my comments on rational basis review were very much a stretch, but thought the Anderson test was closer to strict scrutiny. Admittedly here I was strongly influenced by Derfner and Herbert (Voting Is Speech, 34 Yale L. & Pol’y Rev. 471 (2016)) who obviously want Anderson to be stricter than rational basis. They made it seem as though the original Anderson test was substantially tougher than (and therefore not equivalent to) rational basis, but admitted that in subsequent rulings Anderson seemed to be reinterpreted more and more like simply rational basis. I wonder if there are any more recent rulings to indicate one way or another.
How come they disagree on all those apparently non-spooky questions about relevant patterns in the world?
tl;dr: I take meta-ethics, like psychology and economics ~200 years ago, to be asking questions we don't really have the tools or know-how to answer. And even if we did, there is just a lot of work to be done (e.g. solving meta-semantics, which no doubt involves solving language acquisition. Or e.g. doing some sort of evolutionary anthropology of moral language). And there are few to do the work, with little funding.
Long answer: I take one of philosophy's key contributions to the (more empirical) sciences to be the highlighting of new or ignored questions, conceptual field clearing, the laying out of non-circular pathways in the theoretical landscape, the placing of landmarks at key choice points. But they are not typically the ones with the tools to answer those questions or make the appropriate theoretical choices informed by finer data. Basically, philosophy generates new fields and gets them to a pre-paradigmatic stage: witness e.g. Aristotle on physics, biology, economics etc.; J. S. Mill and Kant on psychology; Yudkowsky and Bostrom on AI safety; and so on. Give me enough time and I can trace just about every scientific field to its origins in what can only be described as philosophical texts. Once developed to that stage, putatively philosophical methods (conceptual analysis, reasoning by analogy, logical argument, postulation and theorizing, sporadic reference to what coarse data is available) won't get things much further – progress slows to a crawl or authors might even start going in circles until the empirical tools, methods, interest and culture are available to take things further.
(That's the simplified, 20-20 hindsight view with a mature philosophy and methodology of science in hand: for much of history, figuring out how to "take things further" was just as contested and confused as anything else, and was only furthered through what was ex ante just more philosophy. Newton was a rival of Descartes and Leibniz: his Principia was a work of philosophy in its time. Only later did we start calling it a work of physics, as pertaining to a field of its own. Likewise with Leibniz and Descartes' contributions to physics.)
Re: meta-ethics, I don't think it's going in circles yet, but do recognize the rate at which it has produced new ideas (found genuinely new choice points) has slowed down. It's still doing much work in collapsing false choice points though (and this seems healthy: it should over-generate and then cut down).
One thing it has completely failed to do is sell the project to the rest of the scientific community (hence why I write). But it's also tough sell. There are various sociological obstacles at work here:
- 20th century ethical disasters: I think after the atrocities committed in the name of science during, during the (especially early) 20th century, scientists rightly want nothing to do with anything that smells normative. In some sense, this is a philosophical success story: awareness of the naturalistic fallacy has increased substantially. The "origins and nature of morality" probably raises a lot of alarm bells for many scientists (though, yes, I'm aware there are evolutionary biologists who explore the topic. I want to see more of this). To be clear, the wariness is warranted: this subject is indeed a normative minefield. But that doesn't mean it can't be crossed and that answers can't be found. (I actually think, in the specific case of meta-ethics, part of philosophy's contribution is to clear or at least flag the normative mines – keep the first and second order claims as distinct as possible).
- Specialization: As academia has specialized, there has been less cross-departmental pollination.
- Philosophy as a dirty word: I think "hard scientists" have come to associate "philosophy" (and maybe especially "ethics") with "subjective" or something, and therefore to be avoided. Like, for many it's just negative association at this point, with little reason attached to it. (I blame Hegel – he's the reason philosophy got such a bad rap starting in the early 20th century).
- Funding: How many governments or private funding institutions in today's post-modern world do you expect prioritize "solving the origins and nature of morality" over other more immediately materially/economically useful or prestigious/constituent-pleasing research directions?
There are also methodological obstacles: the relevant data is just hard to collect; the number of confounding variables, myriad; the dimensionality of the systems involved, incredibly high! Compare, for example, with macroeconomics: natural experiments are extremely few and far between, and even then confounding variables abound; the timescales of the phenomena of interest (e.g. sustained recessions vs sustained growth periods) are very long, and as such we have very little data – there've only been a handful of such periods since record keeping began. We barely understand/can predict macro-econ any better than we did 100 years ago, and it's not for a lack of brilliance, rigor or funding.
Alternately, maybe at least one of them is bad at science :P
In the sense that I take you to be using "science" (forming a narrow hypothesis, carefully collecting pertinent data, making pretty graphs with error bars) neither of them are probably doing it well.[1] But we shouldn't really expect them to? Like, that's not what the discipline is good for.
I'd bet they liberally employ the usual theoretical desiderata (explanatory power, ontological parsimony, theoretical conservatism) to argue for their view, but they probably only make cursory reference to empirical studies. And until they are do refer to more empirical work, they won't converge on an answer (or improve our predictions, if you prefer). But, again, I don't expect them to, since I think most of the pertinent empirical work is yet to be done.
"if morality is as real as tigers" being a cheeky framing
I'm not surprised you find this cheeky, but just FYI I was dead serious: that's pretty much literally what I and many think is possibly the case.
it's not necessary that everyone means precisely the same thing when they talk about tigers, as long as the amount of interpersonal noise doesn't overwhelm the natural sparsity of the world that allows us to have single-world handles for general categories of things. You could still call this an attractor, it's just not a pointlike attractor - there's space for different people to use "tiger" in different ways that are stable under normal dynamics. [...] But it would be a mistake to then say "Therefore the most moral point is the center, we should all go there."
So this is very interesting to me, and I think I agree with you on some points here, but that you're missing others. But first I need to understand what you mean by "natural sparsity" and what your (very very rough) story is of how our words get their referents. I take it you're drawing on ML concepts and explanations, and it sounds like a story some philosophers tell, but I'm not familiar with the lingo and want to understand this better. Please tell me more. Related: would you say that we know more about water than our 1700s counterparts, or would you just say "water" today refers to something different than what it referred to in the 1700s? In which case, what is it we've gained relative to them? More accurate predictions regarding... what?
Maybe the general gist was "if you strip away the supposedly-contingent disagreements like 'is there a morality attractor,'" what are the remaining fundamental disagreements about how to do moral reasoning?
Thanks, yep, I'm not sure. Whether or not there is an attractor (and how that attraction is supposed to work) seems like the major crux – certainly in our case!
- ^
One thing I want to defend and clarify: someone the other day objected that philosophers are overly confident in their proposals, overly married to them. I think I would agree in some sense, since I think their work is often in doing pre-paradigmatic work: they often jump the gun and declare victory, take philosophizing to be enough to settle a matter. Accordingly, I need to correct the following:
Meta-ethicists are like physicists who are interested in understanding what causes the perturbations Uranus' orbit, whatever it turns out to be: they are not married to a specific planet-induced-perturbations hypothesis, dropping all interest once Vulcan was found missing.
I should have said the field as whole is not married to any particular theory. But I'm not sure having individual researchers try so hard to develop and defend particular views is so perverse. Seems pretty normal that in trying to advance theory, individual theorists heavily favor one or another theory – the one they are curious about, want to develop, make robust and take to its limit. One shouldn't necessarily look to one particular frontier physicist to form your best guess about their frontier – instead one should survey the various theories being advanced/developed in the area.
(Re: The Tails Coming Apart As Metaphor For Life. I dunno, if most people, upon reflection, find that the extremes prescribed by all straightforward extrapolations of our moral intuitions look ugly, that sounds like convergence on... not following any extrapolation into the crazy scenarios and just avoiding putting yourself in the crazy scenarios. It might just be wrong for us to have such power over the world as to be directing us into any part of Extremistan. Maybe let's just not go to Extremistan – let's stay in Mediocristan (and rebrand it as Satisficistan). If at first something sounds exciting and way better than where you are now, but on reflection looks repugnant – worse than where you are now – then maybe don't go there. If utilitarianism, Christianism etc yield crazy results in the limit, so much the worse for them. Repugnance keeps hitting your gaze upon tails that have come apart? Maybe that's because what you care about are actually homeostatic property clusters: the good doesn't "boil down" to one simple thing like happiness or a few commands written on a stone tablet. Maybe you care about a balance of things – about following all four Red, Yellow, Blue and Green lines (along with 100 other ones no doubt) – never one thing at the unacceptable expense of another. But this is a topic for another day and I'm only gesturing vaguely at a response.)
(Sorry for delay! Was on vacation. Also, got a little too into digging up my old meta-ethics readings. Can't spend as much time on further responses...)
Although between Boyd and Blackburn, I'd point out that the question of realism falls by the wayside...
I mean fwiw, Boyd will say "goodness exists" while Blackburn is arguably committed to saying "goodness does not exist" since in his total theory of the world, nothing in the domain that his quantifiers range over corresponds to goodness – it's never taken as a value of any of his variables. But I'm pretty sure Blackburn would take issue with this criterion for ontological commitment, and I suspect you're not interested in that debate. I'll just say that we're doing something when we say e.g. "unicorns don't exist" and some stories are better than others regarding what that something is (though of course it's open question as to which story is best).
they both seem to agree we're modeling the world and then pointing at some pattern we've noticed in the world
I think the point of agreement you're noticing here is their shared commitment to naturalism. Neither thinks that morality is somehow tied up with spooky acausal stuff. And yes, to talk very loosely, they are both pointing at patterns in the world and saying "that's what's key to understanding morality." But contra:
If two people agree about how humans form concepts, and one says that certain abstract objects we've formed concepts for are "real," and another says they're "not real," they aren't necessarily disagreeing about anything substantive.
they are having a substantive disagreement, precisely over which patterns are key to understanding morality. They likely agree more or less on the general story of how human concepts form (as I understand you to mean "concept formation"), but they disagree about the characteristics of the concept [goodness] – its history, its function, how we learn more about its referent (if it has any) etc. Blackburn's theory of [goodness] (a theory of meta-ethics) points only to feeling patterns in our heads/bodies (when talking "external" to the moral linguistic framework, i.e. in his meta-ethical moments; "internal" to that framework he points to all sorts of things. I think it's an open question whether he can get away with this internal external dance,[1] but I'll concede it for now). Boyd just straightforwardly points to all sorts of patterns, mostly in people's collective and individual behavior, some in our heads, some in our physiology, some in our environment... And now the question is, who is correct? And how do we adjudicate?
Maybe I can sharpen their disagreement with a comparison. What function does "tiger" serve in our discourse? To borrow terms from Huw Price, is it an e-representation which serves to track or co-vary with a pattern (typically in the environment), or is it an i-representation which serves any number of other "in-game" functions (e.g. signaling a logico-inferential move in the language game, or maybe using/enforcing/renegotiating a semantic rule)? Relevant patterns to determine the answer to such questions: the behaviour of speakers. Also, we will need to get clear on our philosophy of language/linguistic theory: not everyone agrees with Price that this "new bifurcation" is all that important – people will try to subsume one type of role under another.[2] Anyway, suppose we now agree that "tiger" serves to refer, to track certain patterns in the environment. Now we can ask, how did "tiger" come to refer to tigers? Relevant patterns seem to include:
- the evolution of a particular family of species – the transmission and gradual modification of common traits between generations of specimens
- the evolution of the human sensory apparatus, which determines what sorts of bundles of patterns humans tend to track as unified wholes in their world models
- the phonemes uttered by the first humans to encounter said species, and the cultural transmission/evolution of that guttural convention to other humans
- ...and probably much more I'm forgetting/glossing over/ignoring.
We can of course run the same questions for moral terms. And on nearly every point Blackburn and Boyd will disagree. None of these are spooky questions, but they seem relevant to helping us get clear on our collective project to study tigers – what it is and how to go about it. Of course zoologists don't typically need to go to the same lengths ethicists do, but I think its fair to chalk that up to the how controversial moral talk is. It's important to note that neither Blackburn nor Boyd are in the business of revising the function/referents of moral talk: they don't want to merely stipulate the function/referent of "rightness" but instead, take the term as they hear it in the mouths of ordinary speakers and give an account of its associated rules of use, its function, the general shape of its referent (if it has one).
At this point you might object: what's the point? How does this have any bearing on what I really care about, the first-order stuff – e.g. whether stealing is wrong or not? One appeal of meta-ethics, I think, is that it presents a range of non-moral questions that we can hopefully resolve in more straightforward ways (especially if we all agree on naturalism), and that these non-moral questions will allow us to resolve many first-order moral disputes. On the (uncontroversial? in any case, empirically verifiable) assumption that our moralizing (moral talk, reflection, judgment) serves some kind of function or is conducive to some type of outcome, then hopefully if we can get a better handle on what we're are doing when we moralize maybe we can do it better by its own lights.[3]
Assuming of course one wants to moralize better – no one said ethics/meta-ethics would be of much interest to the amoralist. Here is indeed a meta-preference – the usual one appealed to in order to motivate the (meta-)ethicists' entreprise. (Most people aren't anti-moralists, who are only interested in meta-ethics insofar as it helps them do moralizing worse. And few are interested in making accurate predictions about homo sapiens' moralizing for its own sake, without applying it to one's own life). But I don't see this as threatening or differentiating from other scientific endeavours. It's not threatening (i.e. the bootstrapping works) because, as with any inquiry, we begin with already some grasp of our subject matter, the thing we're interested in. We point and say "that's what I want to investigate."As we learn more about it, refining the definition of our subject matter, our interest shifts to track this refinement too (either in accordance with meta-preferences, or through shifts in our preferences in no way responsive to our initial set of preferences). This happens in any inquiry though. Suppose I care about solving a murder, but in the course of my investigation I discover no one killed the alleged victim – they died of an unrelated causes. At that point, I may drop all interest upon realizing no murder occurred, or I might realize what I really wanted to solve was the death of this person.
Might we end up not caring about the results of meta-ethics? I find that highly unlikely, assuming we have the meta-preference of wanting to do this morality thing better, whatever it turns out to be. This meta-preference assumes as little as possible about its subject, in the same way that an interest in solving a death assumes less about its subject than an interest in solving a murder. Meta-ethicists are like physicists who are interested in understanding what causes the perturbations Uranus' orbit, whatever it turns out to be: they are not married to a specific planet-induced-perturbations hypothesis, dropping all interest once Vulcan was found missing.
Hopefully we agree on the first-order claim that one should want to do this morality thing better – whatever "doing morality better" turns out to be! In much same way that a athlete will, upon noting that breathing is key to better athletic performance, want to "do breathing better" whatever breathing turns out to be. The only difference with the athlete is that I take "doing morality better" to be among my terminal goals, insofar as its virtuous to try and make oneself more virtuous. (It's not my only terminal goal of course – something something shard theory/allegory of the chariot).
To make sure things are clear: naturalists all agree there is a process as neutral as any other scientific process for doing meta-ethics – for determining what it is homo sapiens are doing when they engage in moralizing. This is the methodological (and ultimately, metaphysical) point of agreement between e.g. Blackburn and Boyd. We need to e.g. study moral talk, observe whether radical disagreement is a thing, and other behaviour etc. (Also taken as constraints: leaving typical moral discourse/uncontroversial first-order claims intact.) Naturalist realists start to advance a meta-ethical theory when they claim that there is a process as neutral as any other scientific process for determining what is right and what is wrong. On naturalist realist accounts our first-order ethics is (more or less) in the same business as every other science: getting better at predictions in a particular domain (according to LW's philosophy of science). To simplify massively: folk morality is the proto-theory for first-order ethics; moral talk is about resolving whose past predictions about rightness/wrongness were correct, and the making of new predictions. None of this is a given of course – I'm not sure naturalist realist meta-ethics is correct! But I don't see why it's obviously false.
This brings me back to my original point: it's not obvious what homo sapiens are doing when they engage in moralizing! It seems to me we still have a lot to learn! It's not at all obvious to me that our moral terms are not regulated by pretty stable patterns in our environment+behaviour and that together they don't form an attractor.
If we have a crux, I suspect it's in the above, but just in case I'll note some other, more "in the weeds" disagreements between Blackburn and Boyd. (They are substantive, for the broad reasons given above, but you might not feel what's at stake without having engaged in the surrounding theoretical debates.)
Blackburn won't identify goodness with any of the patterns mentioned earlier – arguably he can't strictly (i.e. external to the moral linguistic framework) agree we can determine the truth of any moral claims (where "truth" here comes with theoretical baggage). Ultimately, moral claims to him are just projections of our attitudes, not claims on the world, despite remaining "truth-apt." (He would reject some of this characterization, because he wants to go deflationist about truth, but then his view threatens to collapse into realism – see Taylor paper below). Accordingly, and contra Yudkowsky, he does not take "goodness" to be a two-place predicate with its predication relativized to the eye of the beholder. ("Goodness" is best formalized as an operator, and not a predicate according to Blackburn.) This allows him to refute that what's good depends on the eye of the beholder. You can go with subjectivists (moral statements are reports of attitudes, attitudes are what determine what is good/bad relative to the person with those attitudes), who point to basically the same patterns as Blackburn regarding "what is key to understanding morality," and now you don't have to do this internal external dance. But this comes with other implications: moral disagreement becomes very hard to account for (when I say "I like chocolate" and you say "I like vanilla" are we really disagreeing?), and one is committed to saying things like "what's good depends on the eye of the beholder."
I know it can sound like philosophers are trying to trap you/each other with word games and are actually just tripping on their own linguistic shoelaces. But I think it's actually just really hard to say all the things I think you want to say without contradiction (or to be a person with all the policies you want to have): that's part of what what I'm trying to point out in the previous paragraph. In the same vein, perhaps the most interesting recent development in this space has been to investigate whether views like Blackburn's don't just collapse into "full-blown" realism like that of Boyd (along with all it's implications for moral epistemology). This is the Taylor paper I sent you a few months ago (but see FN 2 below). Similarly, Egan 2007 points out how Blackburn's quasi-realism could (alternatively) collapse into subjectivism.
the actionable points of disagreement are things like "how much should we be willing to let complicated intuitions be overruled by simple intuitions?"
I suspect their disagreement is deeper than you think, but I'm not sure what you mean by this: care to clarify?
- ^
I use Carnap's internal-external distinction but IIRC, Blackburn's view isn't exactly the same since Carnap's internal-external distinction is meant to apply to all linguistic frameworks, where Blackburn seems to be trying to make a special carve out specifically for moral talk. But it's been awhile since I properly read through these papers. I'm pretty sure Blackburn draws on Carnap though.
- ^
I mention Price's theory, because his global expressivism might be the best chance anti-realists like Blackburn have for maintaining their distance from realism while retaining their right to ordinary moral talk. There is still much to investigate!
- ^
"by it's own lights" here is not spooky. We notice certain physical systems that have collections of mechanisms that each support one another in maintaining certain equilibria: each mechanism is said to have a certain function in this system. We can add to/modify mechanisms in the system in order to make it more or less resilient to shocks, more or less reliably reach and maintain those equilibria. We're "helping" the system by its lights when we make it more resilient/robust/reliable; "hindering" it when we make it less resilient/robust/reliable.
Good models of moral language should be able to reproduce the semantics that normal people use every day.
Agreed. So much the worse for classic emotivism and error theory.
But semantics seems secondary to you (along with many meta-ethicists frankly – semantic ascent is often just used as a technique for avoiding talking past one another, allowing e.g. anti-realist views to be voiced without begging the question. I think many are happy grab whatever machinery from symbolic logic they need to make the semantics fit the metaphysical/epistemological views they hold more dearly.) I'd like to get clear just what it is you have strong/weak credence in. How would you distribute your credences over the following (very non-exhaustive and simplified) list?
- Classic Cultural Relativism: moral rules/rightness are to be identified with cultural codes (and for simplicity, say that goodness is derivative). Implication for moral epistemology: like other invented social games, to determine what is morally right (according to the morality game) we just need to probe the rulemakers/keepers (perhaps society at large or a specific moral authority).
- Boyd's view (example of naturalist realism): moral goodness is to be identified with the homeostatic clusters of natural (read regular, empirically observable) properties that govern the (moral) use of the term "good" in basically the same way that tigerness is to be identified with homeostatic clusters of natural properties that govern the (zoological) use of the term "tiger." To score highly on tigerness is to score highly on various traits e.g. having orange fur with black strikes, being quadrupedal, being a carnivore, having retractable claws... We've learned more about tigers (tigerness) as we encountered more examples (and counterexamples) of them and refined our observation methods/tools; the same goes (will continue to go) for goodness and good people. Implication for moral epistemology: "goodness" has a certain causal profile – investigate what regulates that causal profile, the same we investigate anything else in science. No doubt mind-dependent things like your own preferences or cultural codes will figure among the things that regulate the term "good" but these will rarely have the final say in determining what is good or not. Cultural codes and preferences will likely just figure as one homeostatic mechanism among many.
- Blackburn's Projectivism or Gibbard's Norm-Expressivism (sophisticated versions of expressivism, examples of naturalist anti-realism): morality is reduced to attitudes/preferences/plans.
- According to Blackburn we talk as if moral properties are out their to be investigated the way Boyd suggests we can, but strictly speaking this is false: his view is a form of moral fictionalism. He believes there is no general causal profile to moral terms: nothing besides our preferences/attitudes regulates our usage of these terms. The only thing to "discover" is what our deepest preferences/attitudes are (and if we don't care about having coherent preferences/attitudes, we can also note our incoherencies). Implication for moral epistemology: learn about the world while also looking deep inside yourself to see how you are moved by that new knowledge (or something to this effect).
- According to Gibbard normative statements are expressions of plans – "what to do." The logical structure of these expressions helps us express, probe and revise our plans for their consistency within a system of plans, but ultimately, no one/nothing outside of yourself can tell you what system of plans to adopt. Implication for moral epistemology: determine what your ultimate plans are and do moral reasoning with others to work out any inconsistencies in your system of plans.
If I had to guess you're in the vicinity of Blackburn (3.a). Can you confirm? But now, how does your preferred view fit your three bullet points of data better than the others? Your 4th data point, matching normal moral discourse (more like a dataset), is another story. E.g. I think (1) pretty clearly scores worse on this one compared to the others. But the others are debatable, which is part of my point – it's not obvious which theory to prefer. And there is clearly disagreement between these views – we can't hold them all at once without some kind of incoherence: there is a choice to be made. How are you making that choice?
As for this:
In terms of epistemology of morality, the average philosopher has completely dropped the ball. But since, on average, they think that as well, surely I'm only deferring to those who have thought longer on this when I say that.
I'm sorry but I don't follow. Care to elaborate? You're saying philosophers have, on average, failed to develop plausible/practical moral epistemologies? Are you saying this somehow implies you can safely disregard their views on meta-ethics? I don't see how: the more relevant question seems to be what our current best methodology for meta-ethics is and whether you or some demographic (e.g. philosophers) are comparatively better at applying it. Coming up with a plausible/practical moral epistemology is often treated as a goal of meta-ethics. Of course the criteria for success in that endeavor will depend what you think the goals of philosophy or science are.
So the rules of chess are basically just a pattern out in the world that I can go look at. When I say I'm uncertain about the rules of chess, this is epistemic uncertainty that I manage the same as if I'm uncertain about anything else out there in the world.
The "rules of Morality" are not like this.
This and earlier comments are bald rejections of moral realism (including, maybe especially, naturalist realism). Can I get some evidence for this confident rejection?
I'm not sure what linking Yudkowsky's (sketch of a) semantics for moral terms is meant to tell us. Case in point, Cornell Realists adopt a similar relativism in their semantics ("good" like "healthy" can only be judged relative to the type of creature you are), but (some of them anyway) will still argue that we can simply discover what is good through a more or less standard scientific process. In other words, they do believe there is a basin of attraction for human values and there is a neutral process for finding it. (It's only a basin of attraction of course insofar as this process will find it and should we trust that process, we will gravitate toward that basin). To be clear, few if any claim there is one exact lowest point in this basin – there will be many constellations of goods in a life that are equally worthwhile in some sense (first gloss: in the sense that we would be indifferent to a choice between those lives, from behind a veil of ignorance that only assumes we are homo sapiens).
In any case, every major view in meta-ethics has a developed semantics for moral terms: you'll have to say more about why e.g. your favored semantics is a point in favor of your meta-ethical view. You don't need to start from scratch of course: philosophers have been working on this for decades (and continue to). Ayer's classic emotivism (a.k.a. expressivism) ran into the problem of embedding (if moral statement P doesn't have a truth value, how am I supposed to evaluate statements with P embedded in them, like "P → Q"? Our nice truth tables get thrown out the window...). In response several anti-realists have made proposals, e.g. Blackburn's quasi-realism. More recently, those responses have come under fire for struggling to hold onto their distinct semantics (or metaphysics or epistemology) while also holding onto their having a distinct view from realism. There is always Error Theory of course but then you're committed to saying things like "It is false that arbitrary torture is bad. It is also false that arbitrary torture is good."
If none of this discussion on meta-ethics is your thing, that's fine, but then you might want to dampen your certainty? Consider deferring to those who have thought longer on this – update on the distribution of philosophers' views on meta-ethics, modulo whatever selection effects you think are biasing that distribution in a particular direction?
Harder, yes; extremely, I'm much less convinced. In any case, Chevron was already dealt a blow in 2022, so those lobbying Congress to create an AI agency of some sort should be encouraged to explicitly give it a broad mandate (e.g. that it has the authority to settle various major economic or political questions concerning AI.)
Thanks for reading!
conflict theory with a degrowth/social justice perspective
Yea, I find myself interested in the topics LWers are interested in, but I'm disappointed certain perspectives are missing (despite them being prima facie as well-researched as the perspectives typical on LW). I suspect a bubble effect.
this is unfortunately where my willing suspension of disbelief collapsed
Yup, I suspected that last version would be the hardest to believe for LWers! I plan on writing much more in depth on the topic soon. You might be interested in Guive Assadi's recent work on this topic (not saying he makes the story more plausible, but he does tease out some key premises/questions for its plausibility).
My only intention here was to layout the comparison that needs making (assuming you're a consequentialist with very low discount rates etc): what's the EV of this "once and for all" expansionist solution vs the EV of a "passing the torch" solution? And what level of risk aversion should we be evaluating this with? Neither will last forever or will be perfect. I wouldn't so quickly dismiss the potentially ~10^5 or ~10^6 year long "passing the torch" solution over the comparatively OOMs lower certainty "once and for all" solution. Especially once I add back in the other cruxes that I couldn't develop here (though I encourage reading the philosophical literature on it). I want to see a lot more evidence on all sides – and I think others should too.
The mechanism of the compound interest yields utility.
Depends on what you mean by "utility." If "happiness" the evidence is very much unclear: though Life Satisfaction (LS) is correlated with income/GDP when we make cross-sectional measurement, LS is not correlated with income/GDP when we make time-series measurements. This is the Easterlin Paradox. Good overview of a recent paper on it, presented by its author. Full paper here. Good discussion of the paper on the EA forum here (responses from author as well Michael Plant in the comments).
While I completely agree that care should be taken if we try to slow down AI capabilities, I think you might be overreacting in this particular case. In short: I think you're making strawmen of the people you are calling "neo-luddites" (more on that term below). I'm going to heavily cite a video that made the rounds and so I think decently reflects the views of many in the visual artist community. (FWIW, I don't agree with everything this artist says but I do think it's representative). Some details you seem to have missed:
- I haven't heard of visual artists asking for the absolute ban of using copyrighted material in ML training data – they just think it should be opt-in and/or compensated.
- Visual artists draw attention to the unfair double standard for visual vs audio data, that exists because the music industry has historically had tighter/more aggressive copyright law. They want the same treatment that composers/musicians get.
- Furthermore, I ask: is Dance Diffusion "less-aligned" than Stable Diffusion? Not clear to me how we should even evaluate that. But those data restrictions probably made Dance Diffusion more of a hassle to make (I agree with this comment in this respect).
- Though I imagine writers could/might react similarly to visual artists regarding use of their artistic works, I haven't heard any talk of bans on scraping the vast quantities of text data from the internet that aren't artistic works. It's a serious stretch to say the protections that are actually being called for would make text predictor models sound like "a person from over 95 years ago" or something like that.
More generally, as someone you would probably classify as a "neo-luddite," I would like to make one comment on the value of "nearly free, personalized entertainment." For reasons I don't have time to get into, I disagree with you: I don't think it is "truly massive." However, I would hope we can agree that such questions of value should be submitted to the democratic process (in the absence of a better/more fair collective decision-making process): how and to what extent we develop transformative AI (whether agentic AGI or CAIS) involves a choice in what kind of lifestyles we think should be available to people, what kind of society we want to live in/think is best or happiest. That's a political question if ever there was one. If it's not clear to you how art generation AI might deprive some people of a lifestyle they want (e.g. being an artist) see here and here for some context. Look past the persuasive language (I recommend x1.25 or 1.5 playback) and I think you'll find some arguments worth taking seriously.
Finally, see here about the term "luddite." I agree with Zapata that the label can be frustrating and mischaracterizing given its connotation of "irrational and/or blanket technophobia." Personally, I seek to reappropriate the label and embrace it so long as it is used in one of these senses, but I'm almost certainly more radical than the many you seem to be gesturing at as "neo-luddites."
That sounds about right. The person in the second case is less morally ugly than the first. This is spot on:
the important part is the internalized motivation vs reasoning out what to do from ethical principles.
What do you mean by this though?:
(although I notice my intuition has a hard time believing the premise in the 2nd case)
You find it hard to believe someone could internalize the trait of compassion through "loving kindness meditation"? (This last I assume is a placeholder term for whatever works for making oneself more virtuous). Also, any reason you swapped the friend for a stranger? That changes the situation somewhat – in degree at least, but maybe in kind too.
I would note, according to (my simplified) VE, it's the compassion that makes the action of visiting the stranger/friend morally right. How the compassion was got is another question, to be evaluated on different merits.
I'm not sure I understand your confusion, but if you want examples of when it is right to be motivated by careful principled ethical reasoning or rule-worship, here are some examples:
- for a judge, acting in their capacity as judge, it is often appropriate that they be motivated by a love of consistently respecting rules and principles
- for policymakers, acting in their capacity as policymakers (far-removed from "the action"), it is often appropriate for them to devise and implement their policies motivated by impersonal calculations of general welfare
These are just some but I'm sure there are countless others. The broader point though: engaging in this kind of principled ethical reasoning/rule-worship very often, making it a reflex, will likely result in you engaging in it when you shouldn't. When you do so involuntarily, despite you preferring that you wouldn't: that's internal moral disharmony. (Therefore, ethicists of all stripe probably tend to suffer internal moral disharmony more than the average person!)
Sorry, my first reply to your comment wasn't very on point. Yes, you're getting at one of the central claims of my post.
what I remain skeptical of is the idea that moral calculation mostly interferes with fellow-feeling
First, I wouldn't say "mostly." I think in excessive amounts it interferes. Regarding your skepticism: we already know that calculation (a maximizer's mindset) in other contexts interferes with affective attachment and positive evaluations towards the choices made by said calculation (see references to psych litt). Why shouldn't we expect the same thing to occur in moral situations, with the relevant "moral" affects? (In fact, depending on what you count as "moral," the research already provides evidence of this).
If your skepticism is about the sheer possibility of calculation interfering with empathy/fellow-feeling etc, then any anecdotal evidence should do. See e.g. Mill's autobiography. But also, you've never ever been in a situation where you were conflicted between doing two different things with two different people/groups, and too much back and forth made you kinda feel numb to both options in the end, just shrugging and saying "whatever, I don't care anymore, either one"? That would be an example of calculation interfering with fellow-feeling.
Some amount of this is normal and unavoidable. But one can make it worse. Whether the LW/EA community does so or not is the question in need of data – we can agree on that! See my comment below for more details.
In my view, the neural-net type of processing has different strength and weaknesses from the explicit reasoning, and they are often complementary.
Agreed. As I say in the post:
Of course cold calculated reasoning has its place, and many situations call for it. But there are many more in which being calculating is wrong.
I also mention that faking it til you make it (which relies on explicit S2 type processing) is also justified sometimes, but something one ideally dispenses with.
"moral perception" or "virtues" ...is not magic, bit also just a computation running on brains.
Of course. But I want to highlight something you might be have missed: part of the lesson of the "one thought too many" story is that sometimes explicit S2 type processing is intrinsically the wrong sort of processing for that situation: all else being equal you would be better person if you relied on S1 in that situation. Using S2 in that situation counted against your moral standing. Now of course, if your S1 processing is so flawed that it would have resulted in you taking a drastically worse action, then relying on S2 was overall the better thing for you to do in that moment. But, zooming out, the corollary claim here (to frame things another way) is that even if your S2 process was developed to arbitrarily high levels of accuracy in identifying and taking the right action, there would still be value left on the table because you didn't develop your S1 process. There are a few ways to cash out this idea, but the most common is to say this: developing one's character (one's disposition to feel and react a certain way when confronted with a given situation – your S1 process) in a certain way (gaining the virtues) is constitutive of human flourishing – a life without such character development is lacking. Developing one's moral reasoning (your S2 process) is also important (maybe even necessary), but not sufficient for human flourishing.
Regarding explanatory fundamentality:
I don't think your analogy is very good. When you describe mechanical phenomena using the different frameworks you mention, there is no disagreement between them about the facts. Different moral theories disagree. They posit different assumptions and get different results. There is certainly much confusion about the moral facts, but saying theorists are confused about whether they disagree with each other is to make a caricature of them. Sure, they occasionally realize they were talking past each other, but that's the exception not the rule.
We're not going to resolve those disagreements soon, and you may not care about them, which is fine – but don't think that they don't exist. A closer analogy might be different interpretations of QM: just like most moral theorists agree on ~90% of all common sense moral judgments, QM theorists agree on the facts we can currently verify but disagree about more esoteric claims that we can't yet verify (e.g. existence of other possible worlds). I feel like I need to remind EA people (which you may or may not be) that the EA movement is unorthodox, it is radical (in some ways – not all). That sprinkle of radicalism is a consequence of unwaveringly following very specific philosophical positions to their logical limits. I'm not saying here that being unorthodox automatically means you're bad. I'm just saying: tread carefully and be prepared to course-correct.
... consequentialism judges the act of visiting a friend in hospital to be (almost certainly) good since the outcome is (almost certainly) better than not doing it. That's it. No other considerations need apply. [...] whether there exist other possible acts that were also good are irrelevant.
I don't know of any consequentialist theory that looks like that. What is the general consequentialist principle you are deploying here? Your reasoning seems very one off. Which is fine! That's exactly what I'm advocating for! But I think we're talking past each other then. I'm criticizing Consequentialism not just any old moral reasoning that happens to reference the consequences of one's actions (see my response to npostavs)
If our motivation is just to make our friend feel better is that okay?
Absolutely. Generally being mindful of the consequences of one's actions is not the issue: ethicists of every stripe regularly reference consequences when judging an action. Consequentialism differentiates itself by taking the evaluation of consequences to be explanatorily fundamental – that which forms the underlying principle for their unifying account of all/a broad range of normative judgments. The point that Stocker is trying to make there is (roughly) that being motivated purely by intensely principled ethical reasoning (for lack of a better description) is ugly. Ethical principles are so general, so far removed, that they misplace our affect. Here is how Stocker describes the situation (NB: his target is both DE and Consequentialism):
But now, suppose you are in a hospital, recovering from a long illness. You are very bored and restless and at loose ends when Smith comes in once again. [...] You are so effusive with your praise and thanks that he protests that he always tries to do what he thinks is his duty, what he thinks will be best. You at first think he is engaging in a polite form of self-deprecation [...]. But the more you two speak, the more clear it becomes that he was telling the literal truth: that it is not essentially because of you that he came to see you, not because you are friends, but because he thought it his duty, perhaps as a fellow Christian or Communist or whatever, or simply because he knows of no one more in need of cheering up and no one easier to cheer up.
I should make clear (as I hope I did in the post): this is not an insurmountable problem. It leads to varying degrees of self-effacement. I think some theorists handle it better than others, and I think VE handles it most coherently, but it's certainly not a fatal blow for Consequentialism or DE. It does however present a pitfall (internal moral disharmony) for casual readers/followers of Consequentialism. Raising awareness of that pitfall was the principle aim of my post.
Orthogonal point:
The problem is certainly not just that the sick friend feels bad. As I mention:
Pretending to care (answering your friend "because I was worried!" when in fact your motivation was to maximize the general good) is just as ugly and will exacerbate the self-harm.
But many consequentialists can account for this. They just need a theory of value that accounts for harms done that aren't known to the one harmed. Eudaimonic Consequentialism (EC) could do this easily: the friend is harmed in that they are tricked into thinking they have a true, caring friend when they don't. Having true, caring friends is a good they are being deprived of. Hedonistic Consequentialism (HC) on the other hand will have a much harder time accounting for this harm. See footnote 2.
I say this is orthogonal because both EC and HC need a way to handle internal moral disharmony – a misalignment between the reasons/justifications for an action being right and the appropriate motivation for taking that action. Prima facie HC bites the bullet, doesn't self-efface, but recommends we become walking utility calculators/rule-worshipers. EC seems to self-efface: it judges that visiting the friend is right because it maximizes general human flourishing, but warns that this justification is the wrong motivation for visiting the friend (because having such a motivation would fail to maximize general human flourishing). In other words, it tells you to stop consulting EC – forget about it for a moment – and it hopes that you have developed the right motivation prior to this situation and will draw on that instead.
Here is my prediction:
I claim that one's level of engagement with the LW/EA rationalist community can weakly predict the degree to which one adopts a maximizer's mindset when confronted with moral/normative scenarios in life, the degree to which one suffers cognitive dissonance in such scenarios, and the degree to which one expresses positive affective attachment to one's decision (or the object at the center of their decision) in such scenarios.
More specifically I predict that, above a certain threshold of engagement with the community, increased engagement with the LW/EA community correlates with an increase in the maximizer's mindset, increase in cognitive dissonance, and decrease in positive affective attachment in the aforementioned scenarios.
The hypothesis for why that correlation will be there is mostly in this section and at the end of this section.
On net, I have no doubt the LW/EA community is having a positive impact on people's moral character. That does not mean there can't exist harmful side-effects the LW/EA community produces, identifiable as weak trends among community goers that are not present among other groups. Where such side-effects exist shouldn't they be curbed?
It’s better when we have our heart in it, and my point is that moral reasoning can help us do that.
My bad, I should have been clearer. I meant to say "isn't it better when we have our heart in it, and we can dispense with the reasoning or the rule consulting?"
I should note, you would be in good company if you answered "no." Kant believed that an action has no moral worth if it was not motivated by duty, a motivation that results from correctly reasoning about one's moral imperatives. He really did seem to think we should be reasoning about our duties all the time. I think he was mistaken.
Regarding moral deference:
I agree that moral deference as it currently stands is highly unreliable. But even if it were, I actually don't think a world in which agents did a lot of moral deference would be ideal. The virtuous agent doesn't tell their friend "I deferred to the moral experts and they told me I should come see you."
I do emphasize the importance of having good moral authorities/exemplars help shape your character, especially when we're young and impressionable. That's not something we have much control over – when we're older, we can somewhat control who we hang around and who we look up to, but that's about it. This does emphasize the importance of being a good role model for those around us who are impressionable though!
I'm not sure if you would call it deference, but I also emphasize (following Martha Nussbaum and Susan Feagin) that engaging with good books, plays, movies, etc. is critical for practicing moral perception, with all the appropriate affect, in a safe environment. And indeed, it was a book (Marmontel's Mimoires) that helped J.S. Mill get out of his internal moral disharmony. If there are any experts here, it's the creators of these works. And if they have claim to moral expertise it is an appropriately humble folk expertise which, imho, is just about as good as our current state-of-the-art ethicists' expertise. Where creators successfully minimize any implicit or explicit judgment of their characters/situations, they don't even offer moral folk expertise so much as give us complex detailed scenarios to grapple with and test our intuitions (I would hold up Lolita as an example of this). That exercise in grappling with the moral details is itself healthy (something no toy "thought experiment" can replace).
Moral reasoning can of course be helpful when trying to become a better person. But it is not the only tool we have, and over-relying on it has harmful side-effects.
Regarding my critique of consequentialism:
Something I seem to be failing to do is make clear when I'm talking about theorists who develop and defend a form of Consequentialism and people who have, directly or indirectly, been convinced to operate on consequentialist principles by those theorists. Call the first "consequentialist theorists" and the latter "consequentialist followers." I'm not saying followers dance around the problem of self-effacement – I don't even expect many to know what that is. It's a problem for the theorists. It's not something that's going to get resolved in a forum comment thread. I only mentioned it to explain why I was singling out Consequentialism in my post: because I happen to know consequentialist theorists struggle with this more than VE theorists. (As far as I know DE theorists struggle with it to, and I tried to make that clear throughout the post, but I assume most of my readers are consequentialist followers and so don't really care). I also mentioned it because I think it's important for people to remember their "camp" is far from theoretically airtight.
Ultimately I encourage all of us to be pluralists about ethics – I am extremely skeptical that any one theorist has gotten it all correct. And even if they did, we wouldn't be able to tell with any certainty they did. At the moment, all we can do is try and heed the various lessons from the various camps/theorists. All I was just trying to do was pass on a lesson one hears quite loudly in the VE camp and that I suspect many in the Consequentialism camp haven't heard very often or paid much attention to.
Regarding feelings about disease far away:
I'm glad you have become concerned about these topics! I'm not sure virtue ethicists couldn't also motivate those concerns though. Random side-note: I absolutely think consequentialism is the way to go when judging public/corporate/non-profit policy. It makes no sense to judge the policy of those entities the same way we judge the actions of individual humans. The world would be a much better place if state departments, when determining where to send foreign aid, used consequentialist reasoning.
Regarding feelings toward your friend:
I'm glad to hear that moral reasoning has helped you there too! There is certainly nothing wrong with using moral reasoning to cultivate or maintain one's care for another. And some days, we just don't have the energy to muster an emotional response and the best we can do is just follow the rules/do what you know is expected of you to do even if you have no heart in it. But isn't it better when we do have our heart in it? When we can dispense with the reasoning, or the rule consulting?
I agree that, among ethicists, being of one school or another probably isn't predictive of engaging more or less in "one thought too many." Ethicists are generally not moral paragons in that department. Overthinking ethical stuff is kind of their job though – maybe be thankful you don't have to do it?
That said, I do find that (at least in writing) virtue ethicists do a better job of highlighting this as something to avoid: they are better moral guides in this respect. I also think that they tend to muster a more coherent theoretical response to the problem of self-effacement: they more or less embrace it, while consequentialists try to dance around it.
Great question! Since I'm not a professional ethicist, I can't say: I don't follow this stuff closely enough. But if you want a concrete falsifiable claim from me, I proposed this to a commenter on the EA forum:
I claim that one's level of engagement with the LW/EA rationalist community can weakly predict the degree to which one adopts a maximizer's mindset when confronted with moral/normative scenarios in life, the degree to which one suffers cognitive dissonance in such scenarios, and the degree to which one expresses positive affective attachment to one's decision (or the object at the center of their decision) in such scenarios.
More specifically I predict that, above a certain threshold of engagement with the community, increased engagement with the LW/EA community correlates with an increase in the maximizer's mindset, increase in cognitive dissonance, and decrease in positive affective attachment in the aforementioned scenarios.
The hypothesis for why I think this correlation exists is mostly here and here.
I agree with both of you that the question for consequentialists is to determine when and where an act-consequentialist decision procedure (reasoning about consequences), a deontological decision procedure (reasoning about standing duties/rules), or the decision procedure of the virtuous agent (guided by both emotions and reasoning) are better outcome producers.
But you're missing part of the overall point here: according to many philosophers (including sophisticated consequentialists) there is something wrong/ugly/harmful about relying too much on reasoning (whether about rules or consequences). Someone who needs to reason their way to the conclusion that they should visit their sick friend in order to motivate themselves to go, is not as good a friend as the person who just feels worried and goes to visit their friend.
I am certainly not an exemplar of virtue: I regularly struggle with overthinking things. But this is something one can work on. See the last section of my post.
lol. Fixed, thanks!
Agreed. Roberts, Kavanaugh, and Barrett are generally considered center-right.
In addition, Chief Justice Roberts made it clear on multiple occasions he is concerned with public confidence in the court. This would give them a chance to prove they are non-partisan, on an issue that literally pits the people against incumbent major parties. And as I point out in my case, allowing greater freedom of expression on the ballot that translates into more representative elected officials should help with public trust in general.
I think that's a bit reductionist. There are a number of ideologies/theories regarding how law should be interpreted and what role courts are meant to play etc. Parties certainly pick justices who have legal ideologies that favor the outcomes parties want, regarding current political issues. But I think those legal ideologies are more stable in the justices than their tendency to rule the way desired by the party which appointed them.
I am painfully aware of this. I've been doubting myself throughout, and for awhile just left the idea in the drawer precisely out of fear of its naïvety.
Ultimately I did write it up and post, for three reasons: (1) to avoid getting instantly dismissed, to get my idea properly assessed by a legal expert in the first place, I needed to lay things out clearly; (2) I think it's at least possible that our voting system has largely become invisible, and that many high-powered legal experts are focused on other things (of course there are die-hard voting reform activists, but how much do those two groups overlap?); (3) I really do think the evidence has changed, slowly mounting against plurality voting. E.g. before the modern rediscovery of approval voting in the 70s and its subsequent study in the following decades, their would not have been enough evidence to support the case. Much of my argument turns on just how easy it would be for states to adopt approval voting.
So like... maybe now is the time, and I just happened to stumble on the idea? It's not exactly clever. And it's not like it somehow overturns some long-time legal precedent – as far as I know, voting methods have just never even been tried in court. I'm really just asking: why not?
(OK, there is one wild section about Rational Basis review that would significantly alter a long-time precedent, and that's the weakest section of the post. But the Anderson test shouldn't devolve into RB review anyway.)
Trying to! Any guidance would be welcome. So far I've only sent it to the First Amendment Lawyers Association because it seemed like they would be receptive to it. Should I try the ACLU? Was also thinking of the Institute for Free for Free Speech, though they seem to lean conservative which might make them less receptive. I wonder if there is a high power libertarian leaning firm that specializes in constitutional law... ideally we're looking for lawyers who are receptive to the case, but who also would not be looked upon by the Court as judicial activists (who the conservative judges will be biased against).
Or should I not be thinking about political leanings? Sadly, I fear I must.
Agreed, but that doesn't make for a legal case today. The Originalism many on today's Court subscribe to does not take into consideration the intent of lawmakers (in this case the framers), but instead simply asks: what would reasonable persons living at the time of its adoption have understood the ordinary meaning of the text to be? This is original meaning theory, in contrast with original intent theory.
Very true! I should get feedback from legal experts though before I sink any more time into this.
Yes, such sentences are a thing. Kendall Walton calls them "principles of generation" because, according to his analysis, they generate fictional truths (see his Mimesis as Make-Believe). Pointing at the sand and shouting "There is lava there!" we have said something fictionally true, in virtue of the game rule pronounced earlier. "Narrative syncing" sounds like a broader set of practices that generate and sustain such truths – I like it! (I must say "principles of generation" is a bit clunky anyway – but it's also more specific. Maybe "rule decreeing utterances" would be better?).
Anyway, I could imagine naturally extending this analysis beyond make-believe games to etiquette. The principles of generation here would be generating truths of etiquette.
And then, if you like, you could follow a (small) minority of philosophers in claiming that morality is constructed of such etiquette truths! See e.g.:
Foot, P., 1972. Morality as a system of Hypothetical Imperatives
Joyce, R., 2001. The Myth of Morality
Kalderon, M. E., 2005. Moral Fictionalism
I don't follow the reasoning. How do you get from "most people's moral behaviour is explainable in terms of them 'playing' a status game" to "solving (some versions of) the alignment problem probably won't be enough to ensure a future that's free from astronomical waste or astronomical suffering"?
More details:
Regarding the quote from The Status Game: I have not read the book, so I'm not sure what the intended message is but this sounds like some sort of unwarranted pessimism about ppl's moral standing (something like a claim like "the vast majority of ppl are morally ugly in this way"). There is all the difference between being able to explain most ppl's behaviour in terms of "playing a status game" (in some sense of the word "playing"), and claiming that most ppl's conscious motivation to act morally is to win at the status game. The latter claim could plausibly warrant the pessimism; the former, not. But I don't see an argument for the latter. Why is the former claim not evidence for ugliness? For the same reason that the claim "a mother's love for their child is genetically 'hard-wired'" is not evidence that a mother's love for their child is ugly (or fake, not genuine etc). Explaining the underlying causal mechanisms of a given moral behaviour is not (in general) enough to warrant a given moral judgment. (If instead the book is arguing for some sort of meta-ethical anti-realism, well, then the discussion needs to be much longer...)
Regarding your fear about morality: is the worry that if we just aggregated everyone's values we would get a lock-in to some sort of "ugly" status game? Again, we need more details on its proposed implementation before we can judge whether its ugly (something to be scared of).
But also, why are we assuming some sort of aggregation of first-order human value preferences (no matter the method of aggregation)? Assuming we're talking about AGI (and not CAIS), I always thought it strange to think we need to make sure it shares our own idealized preferences, as opposed to merely the preferences we would hope for in, say, a benevolent god or something. I don't see any a priori reason to believe that the preferences/goals of a benevolent shepherd are likely to be shared with/or strongly aligned with those of the shepherd's flock (no matter how you aggregate the preferences/goals of the flock). (I suppose it depends on the nature of the species we're talking about, but whether it's sheep or humans, I maintain my skepticism). In any case, I agree with you that a lot more meta-ethics needs to be discussed in the alignment research community.
I'm not down or upvoting, but I will say, I hope you're not taking this exercise too seriously...
Are we really going to analyze one person's fiction (even if rationalist, it's still fiction), in an attempt to gain insight into this one person's attempt to model an entire society and its market predictions – and all of this in order to try and better judge the probability of certain futures under a number of counterfactual assumptions? Could be fun, but I wouldn't give its results much credence.
Don't forget Yudkowsky's own advice about not generalizing from fictional evidence and being wary of anchoring. If I had to guess, some of his use of fiction is just an attempt to provide alternative framings and anchors to those thrust on us by popular media (more mainstream TV shows, movies etc). That doesn't mean we should hang on his every word though.