Dan Braun's Shortform

This argument isn't control specific, it applies to any safety scheme with some operating tax or implementation difficulty.^[1][2]

I agree that if there is insufficient security, paying any sizable safety tax (including slowing down) will be difficult. It would either require substantial coordination or much more agressive USG action than would otherwise be needed.

Thus, I agree that security is a huge factor in how AI goes, plausibly more important than technical safety work done in advance, and I agree that the technical AI safety community probably understimates the importance of security. (Security is probably notably less tractible than technical AI safety measures for a variety of reasons, but perhaps of greater importance than doing technical AI safety in advance.)

That said, I can certainly imagine worlds where AI control saves us despite the model being stolen by multiple adversaries. My sense is that the situation is much worse if a powerful AI is stolen, especially if it is stolen by multiple actors, but it isn't obviously fatal.

Therefore, AI control will only be useful in the coming years under one (or more) of these conditions

I would note that we don't just aim to prevent the outcome of the model escaping. Indeed, even if a (misaligned) model was literally made publicly available (on a torrent or whatever), we might still care about safely extracting work out of instances of the model we control. We no longer care about preventing escape, but we do care about getting good non-sabotaged work (on e.g. security or technical AI safety).

We also aren't just worried about direct sabotage, we want to prevent the AI from setting itself up in a position where it has much more power over the AI company. E.g., we really want to prevent it from being able to run itself without monitoring in a position where it has full permissions to do things within the AI company.

Even if the model has literally already escaped into the outside world, I think control is useful. (Though this is obviously a dismal situation to be in.)

As an example scenario:

• China steals the model and is using it heavily.
• We still want to be able to use our model without it causing huge problems, though escaping to the outside world isn't as big of a problem (it still might be better to reduce further proliferation).
• We coordinate with China to slow down and to take sufficient precautions that the model doesn't escape from either of us. This allows for paying the control tax at least for a while. We don't necesarily need China to do a good job with control other than avoiding their model escaping. Just delaying the point at which it escapes is also pretty helpful.
• During the time we have, we use huge amounts of AI labor to figure out a cheaper and more robust alignment strategy or are able to build the case for a stronger level of coordination.

Edit: One further note is that if the tax from control is sufficiently small and takeoff is sufficiently slow by default, then control can still let us get out the key work done on time (prior to adversaries being able to train much more powerful models such that we can't be competitive anymore).

I think one of the priorities of safety-conscious AI developers should be to develop some realistic and downright terrifying demos to show privately to NatSec folks in US/UK/EU. Gotta get that military-grade security in place before we get too much further.

I'd say a combo of 6B and 6C is my story for AI control working, where AI labor is used to make computer security much better, and importantly security becomes good enough such that even nation-states can't defeat the security, by a combination of both general security practices becoming better, and AIs becoming good enough at mathematics and coding such that they can generate sound formal proofs without humans being the loop, which is used to defend the most critical infrastructure for AI labs.

This post is probably the best story on how we could get to a state such that AI control is useful:

https://www.lesswrong.com/posts/2wxufQWK8rXcDGbyL/access-to-powerful-ai-might-make-computer-security-radically [LW · GW]

I agree safety-by-control kinda requires good security. But safety-by-alignment kinda requires good security too.

I think that even after your edit, your argument still applies more broadly than you're giving it credit for: if computer security is going to go poorly, then we're facing pretty serious AI risk even if the safety techniques require trivial effort during deployment.

If your AI is stolen, you face substantial risk even if you had been able to align it (e.g. because you might immediately get into an AI enabled war, and you might be forced to proceed with building more powerful and less-likely-to-be-aligned models because of the competitive pressure).

So I think your argument also pushes against working on alignment techniques.

In general, the hacking capabilities of state actors and the likely involvement of national security when we get closer to AGI feel like significant blind spots of Lesswrong discourse.

(The Hacker and The State by Ben Buchanan is a great book to learn about the former)

I think it's important to consider hacking in any safety efforts. These hacks would probably include stealing and using any safety methods for control or alignment, for the same reasons the originating org was using them - they don't want to lose control of their AGI. Better make those techniques and their code public, and publicly advertise why you're using them!

Of course, we'd worry that some actors (North Korea, Russia, individuals who are skilled hackers) are highly misaligned with the remainder of humanity, andd might bring about existential catastrophes through some combination of foolishness and selfishness.

The other concern is mere proliferation of aligned/controlled systems, which leads to existential danger as soon as those systems approach the capability for autonomous recursive self-improvement: If we solve alignment, do we die anyway? [LW · GW]

This might be a reason to try to design AI's to fail-safe and break without controlling units.  E.g. before fine-tuning language models to be useful, fine-tune them to not generate useful content without approval tokens generated by a supervisory model.