Posts
Comments
This is really cool research! I look forward to seeing what you do in future. I think you should consider running human baselines, if that becomes possible in the future. Those help me reason about and communicate timelines and takeoff a lot.
Great post! Glad to see more discussion of the implications of short timelines on impactful work prioritization on LW.
These last two categories—influencing policy discussions and introducing research agendas—rely on social diffusion of ideas, and this takes time. With shorter timelines in mind, this only make sense if your work can actually shape what other researchers do before AI capabilities advance significantly.
Arguably this is not just true of those two avenues for impactful work, but rather all avenues. If your work doesn't cause someone in a position of power to make a better decision than they otherwise would (e.g., implement this AI control solution on a production model, appoint a better-informed person to lead such-and-such an AI project, care about AI safety because they saw a scary demo, etc.), it's unlikely to matter. Since timelines are short and governments are likely to get involved soon, only a highly concentrated range of actors have final sway over decisions that matter.
I'm fairly confident that this would be better than the current situation, and primarily because of something that others haven't touched on here.
The reason is that, regardless of who develops them, the first (militarily and economically) transformative AIs will cause extreme geopolitical tension and instability that is challenging to resolve safely. Resolving such a situation safely requires a well-planned off-ramp, which must route through extremely major national- or international-level decisions. Only governments are equipped to make decisions like these; private AGI companies certainly are not.
Therefore, unless development is at some point centralized in a USG project, there is no way to avoid the many paths to catastrophe that threaten the world during the period of extreme tension coinciding with AGI/ASI development.
Akash, your comment raises the good point that a short-timelines plan that doesn't realize governments are a really important lever here is missing a lot of opportunities for safety. Another piece of the puzzle that comes out when you consider what governance measures we'd want to include in the short timelines plan is the "off-ramps problem" that's sort of touched on in this post.
Basically, our short timelines plan needs to also include measures (mostly governance/policy, though also technical) that get us to a desirable off-ramp from geopolitical tensions brought about by the economic and military transformation resulting from AGI/ASI.
I don't think there are good off-ramps that do not route through governments. This is one reason to include more government-focused outreach/measures in our plans.
I think it is much less clear that pluralism is good than you portray. I would not, for example, want other weapons of mass destruction to be pluralized.