[Linkpost] Hawkish nationalism vs international AI power and benefit sharing

In the scope of this article, we will consider Transformative Artificial Intelligence (TAI) according to a common (albeit vague) definition where TAI is ‘an AI that precipitates a transition comparable to (or more significant than) the agricultural or industrial revolution’. We will not consider the impact of ASI which achieves performance at a level far above humans. The enormous power of ASI means that social interactions, economic mechanisms and geopolitical dynamics are likely to be transformed in ways so significant that current forecasting is marred by uncertainty. In particular, if one actor obtains a ‘decisive strategic advantage’, the decisions made by that actor plausibly make it very hard to predict the outcome. We believe that the relevance of this article increases with longer TAI timelines and longer transition periods between TAI and ASI. This is because slower development speeds increase the probability of multipolar outcomes and give more time for governance and collaboration to come into effect. While in a fast takeoff scenario the systemic and misuse risks will certainly be severe, the potential misalignment risks stemming from a hastily developed ASI might dwarf them.

I like your article, it has some pleasant forecasts and nice recommendations for what to do if our near future turns out to be serendipitously comfortable. Makes me feel good to read it, and imagine such a future in store for us.

I also like that you are explicit about the fact that the recommendations in this article apply only to a particular subset of possible futures.

Where we get to transformative AI, but not further. Either humanity comes together on a global pause, or the technology does not allow for rapid advancement. The TAI in this possible future does not unlock the ability of defectors-from-the-pause to defeat the entire rest of the world combined.

Also, the possible weirdness is kept in check. Autonomous seafloor or subterranean robots don't kickstart a new industrial base. Local space isn't colonized by autonomous robotic probes. Humanity has immense world-changing power at its fingertips, and chooses to proceed slowly and carefully, in peaceful cooperation.

I'm working on a similar article, which aims to address a wider set of future possibilities. I described this path that you outlined here "The Gentle Path". I'll cite your article as an example of this hopeful option.

I think someone more pessimistic than me might describe this article with an analogy along the lines of: "How we recommend arranging our new garden furniture after we win the lottery."

That would be a bit unfair. I'd give this space of future possibilities better than a 1:1000 chance. Still less than 1%, but worth keeping in mind.