Forging A New AGI Social Contract

post by Deric Cheng (deric-cheng) · 2025-04-10T13:41:11.817Z · LW · GW · 3 comments

This is a link post for https://agisocialcontract.substack.com/p/forging-a-new-agi-social-contract

Contents

  Revisiting Our Social Contract
  The Path Forward
  Launching an Anthology on Governance Solutions
  A Research Agenda: Key Questions For a New Social Contract
  Final Thoughts
None
3 comments

This is the introductory piece for a series of essays written by AI economists, policy researchers, and political thinkers on the topic of a new AGI Social Contract. 

Current contributors include: Anton Korinek, Deger Turan (CEO of Metaculus), Steve Omohundro, Iason Gabriel (DeepMind), Julian Jacobs (DeepMind), Sam Manning (GovAI), Dylan Hadfield (MIT), Seth Lazar (ANU), Peter Salib (UH), Colleen McKenzie (AOI), Philip Tomei (AOI), Dean Ball (Hyperdimensional), Justin Bullock (ARI), Anna Yelizarova (Windfall Trust). 

If you're interested in getting updates on this anthology, leave us your email on our Substack at this link here.

“Digital technologies will bring the world into an era of more wealth and abundance and less drudgery and toil. But there’s no guarantee that everyone will share in the bounty. The outcome—shared prosperity or increasing inequality—will be determined not by technologies but by the choices we make as individuals, organizations, and societies.”

Erik Brynjolfsson - The Great Decoupling, 2015

Society has long operated on a simple bargain: we contribute our labor and, in return, gain income, security, and a stake in the economy. Governments tax our wages to fund public services; corporations rely on human workers to create value; and in return, workers expect their efforts to be rewarded with opportunity and security. This centuries-old implicit bargain will soon come under strain.

There is significant evidence to suggest that artificial general intelligence (AGI) systems capable of directly replacing much of human labor are right around the corner. Leading AI researchers are not just predicting that these systems will be better at performing tasks than today’s AI - they are suggesting that human intelligence will no longer be unique. Some suggest that we are on the threshold of a major inflection point akin to the Agricultural or Industrial Revolutions.

If these claims turn out to be true, they will force us to completely reevaluate how we organize labor throughout the economy. Historically, technology has mainly increased the productivity of workers, automating repetitive tasks and increasing the efficiency of human effort. In contrast, AGI represents something fundamentally different – a technology capable of exceeding human performance across virtually all cognitive domains. Such a system could potentially substitute for workers entirely across countless professions, performing nearly any task with equal or superior skill to the average human.

Inarguably, these types of systems could supercharge the overall economy. We could see growth rates and productivity soar, as the costs to scale digital organizations and services could plummet. We could see incredible technological advances and biomedical research conducted in years, instead of decades^[1]. 

However, such AI systems could simultaneously undermine the fundamental economic bargain that has structured our society, causing pervasive negative outcomes for workers: 

• Instead of augmenting and benefiting labor, these systems could simultaneously replace skilled workers across many sectors of the service economy, leading to massive labor displacement and nowhere for workers to go.^[2] Studies from Oxford and the IMF estimate that between 30% - 47% of jobs could be directly replaceable by AI systems.
• The wealth generated from such systems could flow directly to a small set of capital owners, bypassing human workers and leading to skyrocketing wealth inequality and economic power concentration.
  • Already, the richest 1% already own more wealth than the bottom 95% of the world’s population. An acceleration could see individuals or corporations with greater wealth than the GDP of major countries.
  • Superstar firms with 10x fewer human workers could dominate the new technological frontier and capture a sizable portion of growth.
• Labor-replacing technological change could lead to another Engels' Pause, where GDP per capita rapidly expands despite wages for the working class stagnating or falling. Falling wages or a reduced share of labor income could lead to a collapse of the tax base: for example, income & payroll taxes make up 84% of US federal revenue, whereas corporate taxes account for a mere 9%.
• We could see an increase of downward social mobility for workers, as traditionally “high-skilled” service jobs become cheaply performed by AI, but manual labor remains difficult to automate due to the marginal costs of deploying robotics. These economic pressures could reduce the bargaining power of workers, potentially forcing more people towards gig economy roles or less desirable (e.g. physically demanding) jobs.
• Simultaneously, upward social mobility could decline dramatically as ambitious individuals from lower-class backgrounds may lose their competitive intellectual advantages to AGI systems. Fewer entry-level skilled jobs and the reduced comparative value of intelligence could reduce pathways to success – accelerating the hollowing out of the middle class.

If AGI systems are developed, evidence across the board points towards the conclusion that the majority of workers could likely lose out from this coming economic transformation. A core bargain of our society – if you work hard, you can get ahead – may become tenuous if opportunities for advancement and economic security dry up. 

We risk moving towards a world where wages could fall below living standards as human labor supply outstrips demand; where prospects for the lower-class become scarce; where the wealth generated by AI flows almost entirely to those who own the means of production.

This may not happen all at once, as some AI researchers are suggesting. These massive shifts will take years to propagate through the economy, and longer to impact developing countries and manual labor.^[3] But due to increasing digital interconnectedness, it will also happen much faster than previous economic revolutions – measured in years, not decades. 

Crucially - we do not have to accept these negative externalities of progress. These downsides are not inherent to AI, or technological progress^[4] – they arise as a result of the societal and economic frameworks that we have collectively chosen to operate within. These challenges will be problems of our own making.

Revisiting Our Social Contract

In order to ensure the well-being of most humans through this technological revolution, we will need to rethink our assumptions on the roles of governments, corporations, and individuals. We will need to revisit our underlying social contract.

Historically, the social contract describes the set of agreements concerning the legitimacy of government and its role in governing citizens. This concept, developed by philosophers like Hobbes, Locke, and Rousseau, posits that individuals surrender certain natural freedoms and contribute a portion of their wealth to governments in exchange for protection, order, and social stability.

Over time, this foundational concept evolved beyond the basic relationship between citizen and state. The industrial age expanded the social contract to encompass economic relationships between workers, employers, and broader society. Citizens came to expect not just security, but also economic opportunity. Governments increasingly took on responsibilities for education, infrastructure, and basic welfare as part of their obligation under this implicit agreement.

This evolution produced the modern social contract we recognize today: citizens contribute their labor and a portion of their earnings through taxation; in return, they receive not just protection but also economic security and the promise that hard work would be rewarded with prosperity. 

The existence of AGI systems will fundamentally challenge the underlying principles of this existing social contract. If the economic impacts are nearly as transformative as AI researchers currently suggest, then our interventions will have to be equally ambitious:

• We will need to overhaul tax policies to better capture and utilize the value generated by AI technologies, so that this wealth doesn’t fully bypass governments and the majority of society.
• We may need to reconceptualize our existing systems of social safety nets to provide more security for millions of humans who will be displaced from their current jobs.
• We may need to redefine the relationship between private tech corporations and public governments on matters of national security, intellectual property, and economic redistribution.
• We will have to grapple with our existing concepts of economic justice if we enter a society where the people with access to capital become orders of magnitude more advantaged than the rest of the labor force.

In short- we need to collaboratively develop a new, coherent vision to guide economic and societal governance during the upcoming era of transformation. We have the opportunity to design the future that we want, not the future handed to us by our inherited paradigms.

The Path Forward

Catalyzing these foundational changes to our societal structures will not be easy. First, society needs more innovative & concrete policy proposals from motivated economists and governance researchers. Many of the practical ideas that will shape the next era have yet to be articulated and explored in detail. Rather than just theorizing about the possibilities, we need researchers to describe exactly what interventions should look like and how to go about deploying these policy proposals.

Second – and much more challenging – societies will need to develop a collective consensus around the best methods to rewrite our social contract. Major economic impacts – shocks to the economy – will likely need to be felt in order to catalyze such policy discussion. Powerfully entrenched capitalistic forces will resist the major restructuring that will be necessary, because these interventions will threaten their economic interests. Millions, perhaps billions of voters will need to agree that these major changes are necessary in order to create a more equitable society. 

In order to kickstart this process of consensus, we need stronger ideas for policymakers and the public to latch onto. Let us begin to envision together: what do we want our society to look like after the flood? 

Launching an Anthology on Governance Solutions

Moving forward – we are gathering a consortium of leading economists, policy researchers, and political thinkers around these ideas to build capacity and develop an expert consensus around the strongest proposals.

In the next few months, we’ll be publishing a series of essays from our contributors proposing novel strategies and interventions towards improving societal outcomes after an AI economic transformation. New essays will be published roughly every two weeks.

Essays will largely cover two topics:

1. New principles that should guide economic governance in a world where AI systems become substantial drivers of productivity. These pieces will address fundamental questions about value creation, distribution, and economic organization.
2. Specific and creative economic policies that may target certain negative outcomes (e.g. income inequality, tax base distortion, societal unrest) associated with an AI economic transformation. Articles will discuss a diverse range of innovative policy approaches.

Some examples of topics our contributors plan to write about include:

• Revising the OECD BEPS 2.0 global corporate minimum tax for an economy with extremely profitable multinational AI enterprises, particularly if such corporations are replacing local labor with AI capital.
• New policy systems to manage AI liability, such as contract or insurance-based approaches to allocate responsibility for agentic AI systems.
• Developing an Automation Adjustment Assistance plan similar to the US Trade Adjustment Assistance (TAA) plan to provide unemployment benefits, reskilling opportunities, and health insurance to workers displaced by AI systems.
• A revision of the Windfall Clause in the context of the modern era of AI systems, accounting for the commoditization of frontier AI systems and geopolitical tensions.
• The case in favor of limited economic legal rights for agentic AI systems, akin to legal personhood for corporations.

If you're interested in getting updates on these essays, leave us your email on our Substack at this link here.

A Research Agenda: Key Questions For a New Social Contract

In the rest of this treatise, we’ll present a lightweight research agenda for the outstanding questions we see in the direction of a new AGI social contract. We’ll share a few dozen outstanding theoretical and practical questions, point to existing work in this direction, and identify gaps that we see that should be filled. 

For the full research agenda, see this link here. 

 

Final Thoughts

Humanity's highest calling should be to create systems that ensure security and financial stability for all, enabling every person to achieve their full potential. AGI offers us that opportunity—the prospect of extraordinary abundance through incredible productivity gains.

Yet our existing frameworks may funnel these revolutionary benefits to the few rather than the many. The question before us isn't whether AGI will generate more prosperity, but how we choose to allow it to permeate into society. This upcoming transformation will offer us not just new tools, but a rare moment to reimagine our social contracts. Let us choose to invest in a vision for society that truly works for everyone.

1. ^{^}

   We don’t intend to downplay the massive societal benefits that AGI systems will bring in this article. Instead, we are emphasizing areas that need attention to preserve good outcomes for human workers. 

2. ^{^}

   Research shows that job mobility is segmented into “clusters” of related roles: a construction worker will be unlikely to transition to be a computer programmer, for example. If many jobs in the same cluster are simultaneously automated, labor displacement could lead to persistent unemployment. It may be structurally challenging for workers to transition to a new cluster of the workforce. 

3. ^{^}

   While AGI systems capable of cognitive labor replacement can be diffused globally relatively rapidly and cheaply (see the launches of new LLM models), manual labor replacement will be restricted by the marginal costs of per-unit deployment, accounting for the costs to build, deploy, and maintain robotics infrastructure. In particular, manual labor automation will not occur until the cost per unit falls below the cost of hiring a human worker, which could take a very long time in developing countries with low wages and a large supply of human labor.

4. ^{^}

    See: The Direction of Technological Change in Learning From Ricardo and Thompson: Machinery and Labor in the Early Industrial Revolution and in the Age of Artificial Intelligence.

3 comments

Comments sorted by top scores.