The case for stopping AI safety research

This seems more an argument against evals, interpretability, trojans, jailbreak protection, adversarial robustness, control, etc right? Other (less funded & staffed) approaches don’t have the problems you mention.

I feel like you're saying "safety research" when the examples of what corporations centrally want is "reliable control over their slaves"... that is to say, they want "alignment" and "corrigibility" research.

This has been my central beef for a long time.

Eliezer's old Friendliness proposals were at least AIMED at the right thing (a morally praiseworthy vision of humanistic flourishing) and CEV is more explicitly trying for something like this, again, in a way that mostly just tweaks the specification (because Eliezer stopped believing that his earliest plans would "do what they said on the tin they were aimed at" and started over). 

If an academic is working on AI, and they aren't working on Friendliness, and aren't working on CEV, and it isn't "alignment to benevolence " or making "corrigibly seeking humanistic flourishing for all"... I don't understand why it deserves applause lights.

(EDITED TO ADD: exploring the links more, I see "benevolent game theory, algorithmic foundations of human rights" as topics you raise. This stuff seems good! Maybe this is the stuff you're trying to sneak into getting more eyeballs via some rhetorical strategy that makes sense in your target audience?)

"The alignment problem" (without extra qualifications) is an academic framing that could easily fit in a grant proposal by an academic researcher to get funding from a slave company to make better slaves. "Alignment IS capabilities research".

Similarly, there's a very easy way to be "safe" from skynet: don't built skynet!

I wouldn't call a gymnastics curriculum that focused on doing flips while you pick up pennies in front of a bulldozer "learning to be safe". Similarly, here, it seems like there's some insane culture somewhere that you're speaking to whose words are just systematically confused (or intentionally confusing).

Can you explain why you're even bothering to use the euphemism of "Safety" Research? How does it ever get off the ground of "the words being used denote what naive people would think those words mean" in any way that ever gets past "research on how to put an end to all AI capabilities research in general, by all state actors, and all corporations, and everyone (until such time as non-safety research, aimed at actually good outcomes (instead of just marginally less bad outcomes from current AI) has clearly succeeding as a more important and better and more funding worthy target)"? What does "Safety Research" even mean if it isn't inclusive of safety from the largest potential risks?

The burden of proof is on you that current safety research is not incremental progress towards safety research that matters on superintelligent AI. Generally the way that people solve hard problems is to solve related easy problems first, and this is true even if the technology in question gets much more powerful. Imagine if we had to land rockets on barges before anyone had invented PID controllers and observed their failure modes.

Also, the directions suggested in section 5 of the paper you linked seem to fall well within the bounds of normal AI safety research.

Edit: Two people reacted to taboo "burden of proof". I mean that the claim is contrary to reference classes I can think of, and to argue for it there needs to be some argument why it is true in this case. It is also possible that the safety effect is significant but outweighted by the speedup effect, but that should also be clearly stated if it is what OP believes.

This conflates research that is well enough aimed to prevent the end of everything good, with the common safety research that is not well aimed and mostly is picking easy, vaguely-useful-sounding things; yup, agreed that most of that research is just bad. It builds capability that could, in principle, be used to ensure everything good survives, but it is by no means the default and nobody should assume publishing their research is automatically a good idea. It very well might be! but if your plan is "advance a specific capability, which is relevant to ensuring good outcomes", consider the possibility that it's at least worth not publishing.

Not doing the research entirely is a somewhat different matter, but also one to consider.

OK but what is your plan for a positive Singularity? Just putting AGI/ASI off by say 1 year doesn't necessarily give a better outcome at all.

Can you say more about what types of AI safety research you are referring to? Interpretability, evals, and steering for deep nets, I assume, but not work that's attempting to look forward and apply to AGI and ASI?

AI safety research is speeding up capabilities. I hope this is somewhat obvious to most.

This contradicts the Bitter Lesson, though. Current AI safety research doesn't contribute to increased scaling, either through hardware advances or through algorithmic increases in efficiency. To the extent that it increases the usability of AI for mundane tasks, current safety research does so in a way that doesn't involve making models larger. Fears of capabilities externalities from alignment research are unfounded as long as the scaling hypothesis continues to hold.

1. AI safety research is still solving easy problems. [...]
2. Capability development is getting AI safety research for free. [...]
3. AI safety research is speeding up capabilities. [...]

Even if (2) and (3) are true and (1) is mostly true (e.g. most safety research is worthless), I still think it can easily be worthwhile to indiscriminately increase the supply of safety research^[1].

The core thing is a quantitative argument: there are far more people working on capabities than x-safety and if no one works on safety, nothing will happen at all.

Copying a version of this argument from a prior comment I made [LW(p) · GW(p)]:

There currently seems to be >10x as many people directly trying to build AGI/improve capabilities as trying to improve safety.

Suppose that the safety people have as good ideas and research ability as the capabilities people. (As a simplifying assumption.)

Then, if all the safety people switched to working full time on maximally advancing capabilities, this would only advance capabilites by less than 10%.

If, on the other hand, they stopped publically publishing safety work and this resulted in a 50% slow down, all safety work would slow down by 50%.

Naively, it seems very hard for publishing less to make sense if the number of safety researchers is much smaller than the number of capabilities researchers and safety researchers aren't much better at capabilities than capabilities researchers.

AI control is useful to corporations even if it doesn't result in more capabilities. This is why so much money is invested in it. Customers want predictable and reliable AI. There is a great post here about AI's aligning to Do What I want and Double Checking in the short term. There's your motive.  

Also in a world where we stop safety research, it's not obvious to me why capabilities research will be stopped or even slowed down. I can imagine them being slightly less economically valuable but not much less capable. If anything, without reliability, devs might be pushed to extract value out of these models by making them more capable. 

Fixing them will push the failure modes beyond our ability to understand and anticipate, let alone fix.

So that's why this point isn't very obvious to me. It seems like we can just have both failures we can understand and can't understand. They aren't mutually exclusive.^[1]

1. ^{^}

   Also if we can't understand why something is bad, even given a long amount of time, is it really bad?

I think this is an important discussion to have but I suspect this post might not convince people who don't already share similar beliefs.

1. I think the title is going to throw people off. 

I think what you're actually saying "stop the current strain of research focused on improving and understanding contemporary systems which has become synonymous with the term AI safety" but many readers might interpret this as if you're saying "stop research that is aimed at reducing existential risks from AI". It might be best to reword it as "stopping prosaic AI safety research". 

In fairness, the first, narrower definition of AI Safety certainly describes a majority of work under the banner of AI Safety. It certainly seems to be where most of the funding is going and describes the work done at industrial labs. It is certainly what educational resources (like the AI Safety Fundamentals course) focus on. 
 

2. I've had a limited number of experiences informally having discussions with researchers on similar ideas (not necessarily arguing for stopping AI safety research entirely though). My experience is that people either agree immediately or do not really appreciate the significance of concerns about AI safety research largely being on the wrong track. Convincing people in the second category seems to be rather difficult.

To summarize what I'm trying to convey:
I think this is a crucial discussion to have and it would be beneficial to the community to write this up into a longer post if you have the time. 
 

I agree with the premises (except "this is somewhat obvious to most" 🤷).

On the other hand, stopping AI safety research sounds like a proposal to go from option 1 to option 2:

1. many people develop capabilities, some of them care about safety
2. many people develop capabilities, none of them care about safety

Could you expand upon your points in the second-to-last paragraph? I feel there are a lot of interesting thoughts leading to these conclusions, but it's not immediately clear to me what they are.