Safetywashing

This is why we introduced X-Risk Sheets, a questionnaire that researchers should include in their paper if they're claiming that their paper reduces AI x-risk. This way researchers need to explain their thinking and collect evidence that they're not just advancing capabilities.

We now include these x-risk sheets in our papers. For example, here is an example x-risk sheet included in an arXiv paper we put up yesterday.

Another issue with greenwashing and safetywashing is that it gives people who earnestly care a false impression that they are meaningfully contributing.

Despite thousands of green initiatives, we're likely to blow way past the 1.5c mark because the far majority of those initiatives failed to address the core causes of climate change. Each plastic-straw ban and reusable diaper gives people an incorrect impression that they are doing something meaningful to improve the climate.

Similarly I worry that many people will convince themselves that they are doing something meaningful to improve AI Safety, but because they failed to address the core issues they end up contributing nothing. I am not saying this as a pure hypothetical, I think this is already happening to a large extent.

I quit a well paying job to become a policy trainee working with AI in the European Parliament because I was optimizing for "do something which looks like contributing to AI safety", with a strenuous at best model of how my work would actually lead to a world which creates safe AI. What horrified me during this was that a majority of people I spoke to in the field of AI policy seemed to be making similar errors as I was.

Many of us justify our work this by pointing out the second-order benefits such as "policy work is field-building", "This policy will help create better norms" or "I'm skilling up / getting myself to a place of influence", and while these second order effects are real and important, we should be very sceptical of interventions whose first-order effects aren't promising.

I apologize that this became a bit of a rant about AI Policy, but I have been annoyed with myself for making such basic errors and this post helped me put a word on what I was doing.

This is why we need to say "existential safety" or "x-safety" more often.  "Long-term risk" is no longer an appropriate synonym for existential risk, though "large-scale risk" is still fitting.

I've used the term "safetwashing" at least once every week or two in the last year. I don't know whether I've picked it up from this post, but it still seems good to have an explanation of a term that is this useful and this common that people are exposed to.

I like this. It adds a perception lens and an easy way to name it (and thus cognitively and socially notice it).

I imagine the following is obvious to you and to many readers, but for those for whom it's not, I think the link is important: Both greenwashing and safetywashing are downstream of Goodhart's Law.

(Occasionally I wonder how much of the Art of Rationality amounts to noticing & adjusting for Goodhart drift. It seems like a stunningly enormous proportion each time I check.)

A tongue-in-cheek suggestion for noticing this phenomena: when you encounter professions of concern about alignment, ask yourself whether it seems like the person making those claims is hoping you’ll react like the marine mammals in this DuPont advertisement, dancing to Beethoven’s “Ode to Joy” about the release of double-hulled oil tankers.

I think of greenwashing as something that works on people who are either not paying much attention, not very smart, or incentivized to accept the falsehoods, or some combination of these. Similarly, safetywashing looks to me like something that will present an obstacle to any attempts to use politicians or the general public to exert pressure, and that will help some AI capabilities researchers manage their cognitive dissonance. Looking at, eg, the transformers-to-APIs example, I have a hard time imagining a smart person being fooled on the object level.

But it looks different at simulacrum level 3 [? · GW]. On that level, safetywashing is "affiliating with AI safety", and the absurdity of the claim doesn't matter unless there's actual backlash, which there aren't many people who have time to critique the strategies of second- and third-tier AI companies.

"Safewashing" would be more directly parallel to "greenwashing" and sounds less awkward to my ears than "safetywashing", but on the other hand the relevant ideas are more often called "AI safety" than "safe AI", so I'm not sure if it's a better or worse term.

Safetywashing describes a phenomenon that is real, inevitable, and profoundly unsurprising (I am still surprised whenever I see it, but that's my fault for knowing something is probable and being surprised anyway). Things like this are fundamental to human systems; people who read the Sequences [? · GW] know this.

This post doesn't prepare people, at all, for the complexity of how this would play out in reality. It's possible that most posts would fail to prepare people, because these posts change goalposts; and in the mundane process of following their incentives, both adversaries and wishful thinkers (and everything in between) automatically adapt around the cultural expectations set. However, it is a critical first step and vastly superior to nothing at all. 

Anticipating ways for Goodhart's law [? · GW] to play out in reality isn't a nerdy hobby, it isn't even a way of life, it's being an adult/agent in the real world.