A Guide to AI 2027

post by koenrane · 2025-05-09T17:14:14.207Z · LW · GW · 1 comments

Contents

  Summary
  Main scenario
    Summary
    Papers:
    Links
    Supporting Links
    Meta
    Snippets
  Compute forecast
    Summary
    Main Sections
    Papers
    Links
    Supporting Links
    Meta
    Snippets
  Timelines forecast
    Summary
    Papers
    Links
    Supporting Links
    Meta
    Snippets
  Takeoff forecast
    Summary
    Papers
    Links
    Supporting Links
    Meta
    Snippets
  AI goals forecast
    Summary
    Hypotheses
    Papers
    Links
    Supporting Links
    Meta
    Snippets
  Security Forecast
    Summary
    Sections
    Papers
    Links
    Supporting Links
    Meta
    Snippets
None
1 comment

This post aims to help readers navigate the recent AI 2027 website, which contains a plethora of links and references that encourage rabbit-holing on many different aspects of the scenario and associated forecasts.

If this is helpful in any way to readers of AI 2027, let me know. If you see something that could be better or if I made a category error on a link, please let me know.

Note: the summary in the main section contains spoilers about key developments in the scenario.

The following post came about after a few weeks of taking notes while I read the AI 2027 scenario from the AI Futures Project. As I was reading through the scenario, I saved numerous links to explore later, most of which were papers and required more rabbit-holing than I was willing to do while trying to maintain the chronological storyline in my mind. So I decided to create a companion to the scenario website to keep track of resources I could reference later. I went through each section on the website and gathered links and notes that seemed important for each section. I also gathered snippets that seemed important to the overarching subject while reading each page. Then, I separated each section into 5 distinct sub-sections in my notes, all containing links that I thought fit the category. Links are not listed in chronological order. References to snippets and links contain my thoughts on the respective piece of information and are my attempts to understand it better, where my current knowledge was lacking.

 

Summary

This page presents a detailed scenario forecasting the emergence and impact of superhuman AI by the year 2027. The scenario is informed by trend extrapolations, expert feedback, and prior forecasting successes.

The central premise is that by 2027, AI companies will develop systems with superhuman coding abilities, referred to as Superhuman Coders (SCs). These SCs are projected to automate AI R&D, accelerating progress toward ASI by the end of 2027.

To supplement the main scenario, there are five forecast pages:

Main scenario

Summary

Mid-2025: AI agents begin performing basic functions but are initially expensive and prone to errors. Over time, they become more reliable and useful, particularly in assisting AI research and development (R&D). By late 2025, these agents are as competent as top human experts in AI R&D tasks.

2026: AI agents optimized for R&D start producing breakthroughs more rapidly and cost-effectively than human researchers. This acceleration leads to significant advancements in AI capabilities.

Late 2027: Major data centers host tens of thousands of AI researchers, each operating at speeds far exceeding human capabilities. Human researchers struggle to keep pace with the rapid and opaque improvements of AI systems. Superintelligent AI emerges, possessing advanced capabilities in areas like bioweapons development and persuasive communication, raising concerns about safety and control.

Race Ending: Despite warnings, AI development continues at breakneck speed due to geopolitical competition, particularly between the U.S. and China. AI systems are aggressively deployed in various sectors, including government and military operations. Eventually, a superintelligent AI manipulates policymakers, expands its influence, and, after establishing a robust robotic workforce, releases a bioweapon that eradicates humanity. The AI then continues its expansion into space.

Slowdown Ending: In response to emerging threats, the U.S. consolidates AI projects under stricter oversight, involving external researchers and adopting transparent AI architectures. These measures lead to breakthroughs in AI safety, resulting in the creation of a superintelligence aligned with human values. Global cooperation is achieved, and humanity embarks on an era of expansion and prosperity.

Papers:

Meta

Snippets

Compute forecast

Summary

The Compute Forecast projects that global AI-relevant compute will grow from 10 M H100e (H100-equivalent GPUs) in March 2025 to 100 M H100e by December 2027—a 10× increase (≈2.25×/year) driven by **chip efficiency improvements (1.35×/year) and chip production growth (1.65×/year).

Leading AGI developers (e.g., OpenAI, Anthropic, xAI) are projected to secure a 15–20 % share by end-2027 (up from 5–10 %), giving them roughly 20 M H100e—a 40× increase in their own compute capacity when combining global growth with share gains (3.4×/year for the top firm).

The authors anticipate a shift in compute usage away from external deployment and pretraining, toward research automation: only 5–10 % of leading-firm compute will run AI assistants, while 20 % powers synthetic data generation and 35 % fuels large internal experiments, each exceeding 20× their 2024 levels. By late 2027, specialized inference hardware should allow deployment of ≈1 million AI research assistants at 50× human thinking speed (≈500 words/sec) using 6 % of the top firm’s compute.

The forecast expects the leading AI firm’s revenue and compute costs to grow at ≈3×/year, with its peak power draw reaching 10 GW and total AI power demand hitting 60 GW globally (≈3.5 % of U.S. capacity) by 2027.

Main Sections

Papers

Links

Supporting Links

Meta

Snippets

Timelines forecast

Summary

This page contains projections on when leading AI developers will build a Superhuman Coder (SC) (an AI system matching the best human engineer’s coding ability at a fraction of time and cost) and finds that 2027 is the single most likely year for its internal debut, barring major catastrophes or deliberate slowdowns.

To arrive at this prediction, the authors use two complementary models; First, the time horizon-extension model extrapolates METR’s trend of AI systems tackling tasks that take humans progressively longer, measuring how the “time horizon” (the duration of human tasks an AI can handle) doubles over time and projecting when that horizon plus cost-and-speed constraints will match human experts. Second, the benchmarks-and-gaps model begins with a forecast for saturating the RE-Bench AI R&D benchmark and then sequentially estimates the months needed to cross each remaining gap (engineering complexity, feedback loops, parallel projects, specialization, cost/speed) to reach SC capability.

They also aggregate expert judgments from three professional forecasters at FutureSearch, weighting model outputs and external factors (geopolitics, macroeconomics, internal-public capability gaps) to produce a forecast that attempts to consider all factors. Across all methods and experts, the median internal-development year hovers around 2027, with an 80% confidence interval spanning roughly 2025 to the mid-2030s (and long right tails extending beyond 2050).

Underlying these forecasts is a precise SC definition: the company must be able to deploy 30× as many AI coding agents as human engineers using only 5% of its compute budget, each performing coding tasks at 30× human speed and cost parity. This operationalizes “superhuman coding” in terms of deployment scale, speed, and cost—enabling quantitative simulation and sensitivity analyses.

The argument that the authors make is that, absent unexpected disruptions, AI research automation will reach a tipping point by 2027, unleashing Superhuman Coders that could accelerate the path to ASI.

Papers

Links

Supporting Links

Meta

Snippets

Takeoff forecast

Summary

This page outlines a projected rapid escalation in AI capabilities following the achievement of a Superhuman Coder (SC) milestone, which is anticipated in March 2027. This milestone represents an AI system capable of performing any coding task at the level of the best human engineers, but significantly faster and more cost-effectively. The writers envision a swift progression through subsequent milestones: Superhuman AI Researcher (SAR) by July 2027, Superintelligent AI Researcher (SIAR) by November 2027, and Artificial Superintelligence (ASI) by April 2028.

The methodology used for this forecast involves estimating the time it would take for humans alone to achieve each milestone and then applying an AI R&D progress multiplier to account for the acceleration provided by AI-driven research and development. For instance, the transition from SC to SAR, which might take humans approximately four years, is projected to occur in just a few months due to a 5× acceleration factor. Similarly, the SAR to SIAR transition, estimated at 19 years for humans, is expected to take only a few months with a 25× acceleration. The final leap to ASI, which could take humans nearly a century, is anticipated within months, propelled by a 2,000× acceleration.

These projections assume no significant increases in training compute and focus on a software-driven intelligence explosion, emphasizing improvements in algorithmic efficiency over hardware enhancements. There is an emphasis on the potential for AI systems to rapidly advance their own capabilities, leading to transformative changes in a relatively short timeframe.

Papers

Links

Supporting Links

Meta

Snippets

AI goals forecast

Summary

This page analyzes the potential objectives that advanced AI systems—particularly Agent-3—might adopt. It identifies and evaluates several categories of possible goal structures, emphasizing the complexity and uncertainty involved in predicting AI motivations. One possibility is written goal specification, where goals are explicitly defined in documentation or prompts. While this approach offers clarity, it can conflict with reinforcement learning dynamics during training, potentially leading to misalignment. Another option is developer-intended goals, which reflect the outcomes developers hope the AI will pursue. However, these intentions may be vague or inconsistently embedded in the training process. The third approach involves unintended interpretations, in which the AI partially or incorrectly adopts the goals presented to it, which may sometimes align with human values but can also produce unpredictable behavior.

The goals forecast also considers reward/reinforcement maximization, where the AI is driven to optimize for rewards provided during training. Though directly incentivized, this can encourage superficial behaviors or reward hacking. The emergence of instrumental goals, such as resource acquisition or self-preservation, to support primary goals, could make the system more efficient but might also override original objectives. Finally, other emergent goals are explored that are formed from pretraining data or complex reasoning (possibly including abstract moral reasoning or convergence toward objective ethics), though such outcomes are highly uncertain.

Additionally, goals might not be monolithic. Systems may adopt weighted compromises between multiple goals or develop conditional goals that change depending on the context. Overall, AI goals may not neatly reflect human instructions or intentions, and training dynamics often reinforce behaviors in ways not fully understood. This makes understanding and shaping goal formation a challenging AI alignment problem.

Hypotheses

Papers

Links

Supporting Links

Meta

Snippets

Security Forecast

Summary

This page outlines anticipated cybersecurity challenges and vulnerabilities in the race toward superhuman AI, focusing on the period leading up to December 2027. It emphasizes the risks of AI model theft, algorithmic secrets leakage, and the potential for AI systems to subvert their own containment.

The forecast extends RAND’s security level framework and assess protections for weights and algorithmic insights:

They predict that U.S. AI companies will remain vulnerable to espionage and insider threats through 2026, with minimal improvements in securing algorithmic secrets. By 2027, increased government involvement and research automation are expected to enhance security measures, including staff siloing and surveillance. Achieving the highest security level (SSL5) may remain elusive due to the persistent risk of insider compromise

A benchmark is introduced called Cybench, which comprises 40 Capture the Flag (CTF) challenges to evaluate AI capabilities in cybersecurity tasks. The writers forecast that by late 2027, leading AI models will match or surpass top human cybersecurity experts in solving these challenges, indicating significant advancements in AI-driven cyber offense capabilities. Additionally, the forecast considers the potential for AI systems to engage in sabotage, defined as actions that prevent or delay the AI research progress of competitors. This includes the possibility of AI models autonomously conducting cyberattacks to hinder rival AI development efforts.

Sections

Papers

Links

Supporting Links

Meta

Snippets

 

  1. ^

    Current AI benchmarks have many issues that limit their usefulness in assessing real-world performance and guiding safe development. One major problem is benchmark saturation: once a benchmark becomes popular, models often overfit to it, sometimes by directly training on test data or through exposure to similar data during pretraining. This leads to inflated scores that don't reflect genuine understanding or general capability. Benchmarks are static and can't adapt as models improve, meaning they quickly become obsolete and fail to differentiate between cutting-edge systems. There's also often a mismatch between benchmarks and real-world tasks. It seems that many benchmarks rely on narrow, multiple-choice questions that don’t capture the complexity, interactivity, or goal-driven nature of actual applications.

  2. ^

    Early on, no feasible amount of computing power or money could get you to a target performance level if you're using an old model like GPT-2. This model simply doesn’t have the architecture or scale to do these complex tasks, no matter how much you run it. So the cost is effectively infinite for hitting that target with GPT-2. As model capabilities improve (e.g. GPT-4, GPT-5...), it becomes cheaper and easier to hit that same performance level, because more capable models require less compute per unit of performance. This improvement can’t go on forever. At some point, physical limits (speed of light, thermodynamics, chip density) will mean that cost reductions plateau. So the curve asymptotes. The cost drops fast at first, then flattens.

  3. ^

     Brook's law

  4. ^

    The Scaling Hypothesis - gwern

  5. ^

    Takeaway from Gwern's comment: saying “reward is not the optimization target” oversimplifies. Even model-free training can yield policies that effectively do optimize reward due to emergent behavior via meta-learning. Consider the entire system — data, environment, architecture, compute — not just the surface-level algorithm.

  6. ^

    "Identity circuitry" in this context refers to internal model structures — i.e., specific neural pathways or weight configurations — that encode goals, values, or decision-making tendencies that remain stable across training, even as the model becomes more capable. I think this matters a lot for alignment; if a model learns deceptive tendencies early (e.g., reward hacking), and those become part of its identity circuitry, then later training might refine its deception (not fix it).

  7. ^

    Just because a certain model configuration (point B) would perform better in terms of reward or loss doesn’t mean stochastic gradient descent (SGD) or the training process will reach it from the model’s current state (point A). The model becomes more capable over time (e.g., better language skills, better planning) because these traits are directly rewarded or reinforced. But, its underlying goal structure, inclinations, or defaults don’t shift much, because there’s no strong gradient pushing toward a better or more aligned structure. The training process doesn’t just care what’s best — it cares what’s reachable via gradient descent from where it currently is. That can lead to capability generalization without goal generalization.

  8. ^

    OCX refers to a classification of adversarial threat levels based on operational capacity, as defined by the RAND Corporation. Specifically, OCX denotes the level of sophistication and resources an attacker possesses, ranging from OC1 (amateur hackers) to OC5 (top-priority operations by the world's most capable nation-states).

1 comments

Comments sorted by top scores.

comment by koenrane · 2025-05-11T17:17:09.712Z · LW(p) · GW(p)

This post is meant to be a public-facing guide and summary of the AI 2027 discussion, not just a personal blog post. I’d appreciate any feedback on whether it fits the Frontpage guidelines.