Summary of Situational Awareness - The Decade Ahead

post by Oscar (Oscar Delaney) · 2024-06-10T08:44:41.675Z · LW · GW · 2 comments

This is a link post for https://forum.effectivealtruism.org/posts/zmRTWsYZ4ifQKrX26/summary-of-situational-awareness-the-decade-ahead

Contents

  Short Summary
None
2 comments

Original by Leopold Aschenbrenner, this summary is not commissioned or endorsed by him.

Short Summary

• Extrapolating existing trends in compute, spending, algorithmic progress, and energy needs implies AGI (remote jobs being completely automatable) by ~2027.
• AGI will greatly accelerate AI research itself, leading to vastly superhuman intelligences being created ~1 year after AGI.
• Superintelligence will confer a decisive strategic advantage militarily by massively accelerating all spheres of science and technology.
• Electricity use will be a bigger bottleneck on scaling datacentres than investment, but is still doable domestically in the US by using natural gas.
• AI safety efforts in the US will be mostly irrelevant if other actors steal the model weights of an AGI. US AGI research must employ vastly better cybersecurity, to protect both model weights and algorithmic secrets.
• Aligning superhuman AI systems is a difficult technical challenge, but probably doable, and we must devote lots of resources towards this.
• China is still competitive in the AGI race, and China being first to superintelligence would be very bad because it may enable a stable totalitarian world regime. So the US must win to preserve a liberal world order.
• Within a few years both the CCP and USG will likely ‘wake up’ to the enormous potential and nearness of superintelligence, and devote massive resources to ‘winning’.
• USG will nationalise AGI R&D to improve security and avoid secrets being stolen, and to prevent unconstrained private actors from becoming the most powerful players in the world.
• This means much of existing AI governance work focused on AI company regulations is missing the point, as AGI will soon be nationalised.
• This is just one story of how things could play out, but a very plausible and scarily soon and dangerous one.

To read my longer summary, see the EAF version.

2 comments

Comments sorted by top scores.