A History of the Future, 2025-2040
post by L Rudolf L (LRudL) · 2025-02-17T12:03:58.355Z · LW · GW · 17 commentsThis is a link post for https://nosetgauge.substack.com/p/a-history-of-the-future-2025-2027
Contents
2025-2027 The return of reinforcement learning Codegen, Big Tech, and the internet Business strategy in 2025 & 2026 Maths and the hard sciences Societal response Alignment research & AI-run orgs Government wakeup 2027-2030 The AGI frog is getting boiled The bitter law of business The early days of the robot race The digital wonderland, social movements, and the AI cults AGI politics & the chip supply chain 2030-2040 The end of white-collar work and the new job scene Lab strategy amid superintelligence and robotics Towards the automated robot economy The human condition in the 2030s 2040+ None 17 comments
This is an all-in-one crosspost of a scenario I originally published in three parts on my blog, No Set Gauge. Links to the originals:
- A History of the Future, 2025-2027
- A History of the Future, 2027-2030
- A History of the Future, 2030-2040
Thanks to Luke Drago, Duncan McClements, Theo Horsley, and Bilal Chughtai [LW · GW] for comments.
2025-2027
Below is part 1 of an extended scenario describing how the future might go if current trends in AI continue. The scenario is deliberately extremely specific: it’s definite rather than indefinite, and makes concrete guesses instead of settling for banal generalities or abstract descriptions of trends.
The return of reinforcement learning
From 2019 to 2023, the main driver of AI was using more compute and data for pretraining. This was combined with some important "unhobblings":
- Post-training (supervised fine-tuning and reinforcement learning for instruction-following) helped the LLMs be usable without difficult prompting.
- Starting in 2024, Anthropic showed that judgement and taste in data curation—and the evaluation metrics that guide data curation—could give you a "magic sauce" effect in perceived LLM quality.
Most real-world LLM uses, of course, involved generating a sequence of tokens to try to achieve some task. So there were a lot of untapped gains from doing reinforcement learning (RL) for performance on concrete domains, rather than just RL for the models following instructions and being "safe"—i.e. a combination of avoiding PR hazards, and preparing for misuse mitigations on actually capable models down the line.
OpenAI fires the starting gun in 2024 with the release of o1, which was based on RL on chains-of-thought (COT), i.e. the model is trained to reason step-by-step towards correct answers, i.e. "test-time compute" in the horror-filled annals of machine learning jargon. In late 2025 they release “GPT o5” (“GPT” to normal people, and “o5” to those keeping track of the version number), a model which can take text, image, audio, video, computer screen state, real-life footage, whatever, process and understand it (choosing itself whether it should do chain-of-thought reasoning before answering or not), and output text, image, audio, video, computer actions.
Whereas the labs had spent almost four years racing down the scaling graph on pretraining compute, they had not yet done so for COT RL, and had not uncovered the subtler tricks to doing this well. This meant there was a lot of low-hanging fruit, so progress—and replication—was fast. In early 2025, DeepSeek spooks the entire American business scene with their release of R1. In spring 2025, Anthropic ships Claude 4, which also has inference-time compute abilities that trigger if the model is asked a question where that helps.
Anthropic keeps their largest Claude 4 model internal and secret from the very start. It gets used for (most importantly) producing training data for the smaller Claude 4s, and (experimentally) in doing internal evaluations of AI-driven AI R&D, starting with some adversarial robustness research on Claude 3.5. Inference costs on the biggest models are a big part of the rationale. Anthropic continues being focused on intelligence over product, and enterprise products over consumer products. They make only minor gains among consumers, but Claude is increasingly adopted among enterprises, programmers, knowledge-workers, and nerds. (Ironically, OpenAI has the consumer advantage despite focusing more on reasoning and less on the LLM being personable and writing well.)
In 2025, thanks to RL, “agentic” AI is here, but only kind of. Anthropic and OpenAI have computer-use features that work, except a bit spottily and are designed to never authorise a payment or send an email or do anything important without human confirmation. Google releases an AI agent for things like Google Cloud Platform configuration schlep, which the programmers love. A bunch of startups are competitive with the major lab products, in particular because no one has yet had time to pour ungodly amounts of compute into the COT RL. However, most "agentic" AI applications remain LLM scaffolds, i.e. a hard-coded flowchart of LLM prompts and other API calls.
Meta is trialling some unholy autonomous AI features across their apps (such as AI agents going around leaving comments on user’s posts to “maximise engagement”), but they still seem like gimmicks.
Code generation tools like Cursor and Lovable and Zed and Poolside and Magic.dev and ten million others are getting very good. For most apps, you can in fact just drop in a prompt and have the app running within a few minutes, though managing infrastructure is still a pain and technical debt tends to accumulate if the AI stacks many changes on top of each other. Some form of COT RL is used in the training stack for many but not all leading coding tools. LLM scaffolds still reign over unspecialised general agents.
Gemini-3 ships in 2025 after a vast pretraining run. It’s good but a disappointment; the final culmination of pretraining scaling laws in an era where products, inference-time compute, data curation (mostly synthetic now, but behind the scenes there’s some very important human judgement going on), and real-world interaction ability are key. Google DeepMind (GDM) is building powerful maths models, and making progress on reasoning architectures that don’t rely on external COT and are better-suited for maths.
After 2025, RL starts getting harder and the distance between the leading labs and the rest increases again. RL is simply less efficient than pretraining, partly because the necessity of letting models try long sequential chains of actions makes parallelism harder. The labs have now scaled up RL compute quite far, so the resource bar for being in the game rises. Also, RL is notoriously hard. First, subtle bugs are easy to make and hard to notice: is the RL agent not learning because you made a bug, or because it just won't learn? Second, there are more choices to make (e.g. you have to pick a reward function and scoring method, rather than the cross-entropy loss you default to with pretraining). OpenAI, Anthropic, and Google take some distnace to the rest in RL and overall general capabilities. However, the other labs don't necessarily see this as a loss—Meta deliberately focuses more on integrating AI into its products over 2025 and 2026, xAI focuses more on engineering use-cases, and both xAI and DeepSeek remain competitive. Also, the issues with RL mean that there are some more hairy technical problems that temporarily slow progress as labs one after another internally work through them, though this is not at all obvious from outside a lab.
In early 2026, xAI starts deploying an early version of an AI that can do engineering CAD work kind-of-well, as long as a human is looking over its shoulder and checking its work. This improves a lot after Tesla and SpaceX (are forced to) actually start using it, but it’s not yet groundbreaking; sheer data quantity remains an issue.
The next big advance is OpenAI's late-2026 release of o6. It has improved a lot in computer use, and generally in unifying its various input and output types (e.g. it can use text and images more effectively together in its output, process and output longer videos, etc.). Second, it has a more advanced memory architecture, including a built-in longer-term memory that allows instances to learn over time. Thirdly, it’s of course generically a bit smarter, a bit faster in token output, and so on. In particular, OpenAI has finally almost caught up to Claude’s personality level. It is also way more impressive to normal people because it can also—if prompted to do so—generate real-time video and audio of a talking face. OpenAI doesn’t explicitly encourage this, but winks at this, since it knows this will get some users addicted (especially as they now have a more nuanced policy for sexually explicit model outputs than the previous blanket ban).
Many people in Silicon Valley declare this AGI, and predict the immediate automation of all office jobs. In practice, it falls short in a hundred subtle ways that make it not a drop-in replacement, in particular with remaining unreliability in its ability to use computers and weaknesses at planning and completing long-horizon tasks. But the smart money is betting that these issues will be solved within a year.
Also in late 2026, Anthropic releases Claude 5 Haiku and Claude 5 Sonnet. Claude 5 Haiku is a cheap model roughly on par with Claude-3.5-Sonnet in smartness while having an output speed of hundreds of tokens per second. They come with an upgraded version of computer use that is far faster and more seamless. Again, the largest model is kept internal. Its training data curation and post-training finetuning was focused on programming, ML research, MLOps, and maths. Anthropic employees started adopting it internally in mid 2025, giving researchers and engineers what's essentially a team of AI interns to manage. They then spent 6 months giving the models tailored feedback, which they massively boosted with methods for dataset augmentation, and filtered for correctness with scalable oversight techniques like debate, before feeding it back into the model as finetuning data. In 2024, Anthropic internally estimated a +5-10% productivity boost from internal use of Claude-3.5-Sonnet and early training checkpoints of Claude-4; in 2025, this rose to +25%, and with Claude 5 Opus it started out at +35% but has gradually accelerated with more and more finetuning to +60% by mid 2026, and the number is still climbing. OpenAI does not have a comparable setup internally, partly because it’s politically less feasible due to the lower-trust environment, but also because it's a lower priority since they believe less in recursive self-improvement.
Codegen, Big Tech, and the internet
Coding is a purely digital job that is economically highly valuable, has a lot of training data and often provides a clean feedback signal of success, and that the AI-affiliated companies all already have expertise in. All this makes it ideal for AIs to be good at, quickly. In 2023-2026, the biggest economic impact of LLMs is their use in coding.
In 2023, models got good enough for programmers to prefer them to looking up human guidance on sites like StackOverflow. In 2024, coding copilots were a real productivity boost, perhaps +10% to +50%, for pure software engineering tasks (higher for things that are more boilerplate and when the coder has less background in what they're doing, lower for more research-y tasks or when working with familiar domains). In 2025, there are two big new advances. First, chain-of-thought RL meant that spending more LLM tokens converted more efficiently into better code. Second, a bunch of the obvious improvements to the workflow were made, such as the AI automatically running tests or checking that the UI looks right, and autonomously trying again if not, rather than maintaining the human as a tab-switching, prompt-writing monkey that does this for the AI. As a result, by 2026 codegen looks solved. There are some wrinkles left related to cloud infrastructure stuff, especially when there’s little training data on some aspect and/or a heavy and unavoidable button-clicking component, but these are quickly getting fixed, especially as computer use gets good and allows the models to better click buttons and surf the internet for documentation.
For a while, everyone’s paranoid about security in the fully AI-written codebases, and a bunch of security consulting and cybersec firms make a killing. However, it soon turns out codegen stuff is actually more secure than human code because the LLMs reliably do the standard correct thing over the weird bespoke thing whenever it comes to security, and this eliminates a lot of security vulnerabilities that humans would write in. The security consulting and cyber firms quickly become LLM wrapper companies with excellent marketing arms, and stop being used by most people apart from risk-averse large companies and governments. However, as is statistically obvious, there are a bunch of high-profile blowups, and it remains true that existing code can now be much more easily attacked since all you need is an o6 or Claude subscription.
By 2027, the price of creating a simple app is a few dollars in API credits or GPU hours. The price of a particularly complicated piece of software is on the order of $100 to $10k. The tech stack has shifted almost entirely to whatever there was the most data on; Python and Javascript/Typescript are in, almost everything else is out. Average code quality as judged by humans declines, but this is fine because humans don't read it and the LLMs can deal better with bloated code.
The coding advances trigger a massive flood of non-coders or amateurs flooding in and trying to make money off B2B SaaS or freelance programming. Agentic non-technical people are launching niche startups at massive rates, since you can ship a full-featured product in a few hours if you’re willing to burn money on API credits. Lots of these projects run into “tech debt hell” eventually. For a while programmers can earn heavy consulting fees (or cofounder roles) by coming in, chatting to the AI about the codebase, and telling it to make architectural changes that let the next features be added cheaper because it will take fewer lines of code on top of the better-structured codebase. However, just asking the AI “what’s wrong with this codebase” and then “how would you fix it” also works quite well if the prompting is good. The codegen scaffolds quickly evolve to be good at reflectively prompting AIs and managing the tech debt hell better, but it’s hard to notice this unless you’re working with them actively, leading to a lot of misinformed doubts about the capabilities based on early disappointments. The labs also start including more qualitative criteria in their codegen RL—not just "did the code run and pass the tests", but also asking another LLM to grade the style and extensibility of the code. In effect, there's a race over whether the AIs will learn good code practices from RL self-play, or from explicit human scaffold-crafting and prompting. Note that the latter is getting easier too, as the tooling improves, and AIs write the scaffold code and distill human programming knowledge into prompts. For example, in late 2025 Anthropic also ship an automated tool for building an LLM scaffold from observations of an arbitrary real-world digital work process.
Big Tech starts using the codegen tools heavily for new projects, but integration into older projects is slower because the codegen scaffolds are worse at interfacing with large existing codebases than writing small ones from scratch. This gets mostly solved over the course of mid-2025 to mid-2026, but gives the "Little Tech" startups a temporary tailwind. Big Tech headcounts grow, as they hire more people both to flatter the egos of managers—they are drowning in cash anyway—and in particular many product managers to oversee the AI codegen agents that are unleashing a massive series of new products now that they're mostly no longer constrained by development taking lots of time. Internal company office politics becomes even more of a rate-limiter: if teams are functional, the AI codegen boost means more products shipped, whereas if teams are not, the gains are eaten up by employees working less or by factional fights within companies. Microsoft launches “365 Days of Microsoft”, where every day of the year they release a new software product or a big update to a previous one; they move increasingly into more niche enterprise markets that they had previously ignored as part of a new paradigm shift. Google is more scattered and launches a thousand new features integrated into their product suites that—on paper–compete with existing startups, and—in practice—serve to expand the empires of enterprising Google middle-managers. Google gets a reputational hit as a shipper of sloppy products, but they have a few big hits and their customers are a captive market that will continue using Search and Drive, giving them room to flail around.
There are a few corporate scandals as AI codegen products fail, leading to a frenzy of effort at testing and fuzzing the AI outputs. But Big Tech is still all-in, at least until late 2026: they’re all feeling the AGI, and that if they miss it that’s an existential mistake, and if it’s all a bubble then at least they were only as bad as all the other Big Tech firms. The one slow actor is Apple, due to its cultural bias towards quality and assurance. Apple ships Apple Intelligence integrations but that’s about it.
Predictably, the super-abundance of software and the extreme competition in it drives down prices. SaaS companies aren’t yet experiencing an extinction wave because humans react to change slowly, but it doesn’t look good and investors start getting skittish. The big advantage that everyone points to is having locked-in customers or network effects; otherwise, the conventional wisdom goes, you're dead. But there are a bunch of companies and tools that let you circumvent attempts at customer lock-in. You can program yourself an X.com in an afternoon, have computer-using AI agents trawl X, Reddit, etc., pull in content to your site, and propagate your posts and replies automatically to all the other platforms. Some companies fight tooth and nail to try to make people stay on their platform (and thus see their ads); some just charge API prices and hope to at least get revenue. “Web4” comes to mean a programmable internet that is customised to everyone. A hundred startups jump on this bandwagon. Some established companies create carefully de-risked APIs and let users program customisations and integrations into their sites (i.e. let users ask codegen models to do such programming). The Web4 wave generally runs into the problem that most people don’t actually want to customise things; they want someone to have already thought through the interface and features on their behalf, are fine with existing setups, and not very eager to re-imagine the internet. But increasingly, if users dislike something about a site, they will build their own version, connect it to the original with AI schlep, and then lure over the few percent of users that are triggered by the same thing. Technical barriers like scraping limits are hard as AI agents can be made to browse in increasingly human-like ways (one successful startup explicitly engages in a scraping race against scraping-detection methods by fine-tuning a computer use agent on real human mouse moving patterns). An increasingly common barrier is asking humans for government ID or other real-world verification (with the privacy constraints mitigated with zero-knowledge proofs, if it's a fancy libertarian or crypto -affiliated thing). This too is spreading, also because some people want sites where they can talk to verified real humans.
By 2026, more code gets written in a week than the world wrote in 2020. Open source projects fork themselves into an endless orgy of abundance. Some high school students build functionally near-identical versions of Windows and Google Drive (and every video game in existence) from scratch in a month, because they can and they wanted one new feature on top of it. Everyone and their dog has a software product line. Big Tech unleashes a torrent of lawsuits against people cloning their products, echoing the Oracle v Google lawsuit about Java, but those lawsuits will take years to complete, and months feel like decades on the ground.
Silicon Valley is exuberant. The feeling at Bay Area house parties is (even more than before) one of the singularity being imminent. Some remain skeptical though, rightfully pointing out that the post-scarcity software isn’t the same as post-scarcity everything, and that genuine “agency” in the long-horizon real-world planning sense hasn’t really arrived, and under the hood everything is still rigid LLM scaffolds or unreliable AI computer use agents.
Business strategy in 2025 & 2026
Even though Meta, DeepSeek, and others are behind in raw intelligence and reasoning all throughout 2025 and 2026, they threaten the big labs because they are giving away (both to consumers, and freely to developers through open-weights releases) a level of performance across audio and video and image and text that is “good enough” for most use cases. SOTA performance is no longer needed for many use-cases, especially low-end consumer entertainment (e.g. image gen, chatbots, etc., which Meta is banking on), or most classification, processing, or business writing tasks.
OpenAI is especially vulnerable, since they rely heavily on consumers, and are also increasingly a product company that competes with products built on their API, driving many to switch. Their strategy (internally and to investors, though not publicly) is to be the first to achieve something like a drop-in agentic AI worker and use that to convert their tech lead over open source into >10% of world GDP in revenues. They’ve raised tens of billions and make billions in revenue from their products anyway, so they can bankroll these efforts just fine.
Anthropic remains a jewel of model quality and a Mecca of technical talent that gets surprisingly little attention from the rest of the industry. Analogies to Xerox PARC abound, but there are whispers of internal AGI being imminent, and no one else can claim the ideological mandate of heaven for safe AGI. The talent and money spigots stay on.
xAI and DeepSeek continue releasing open-source consumer models. Both also have a specialty in maths-y STEM and engineering stuff, aided by data collection efforts (with xAI being able to work closely with SpaceX and Tesla engineers) and inference-time compute methods. xAI also continues trying to leverage real-time access to X.com data to its benefit, but this isn't a major advantage or revenue source.
In 2024, thousands of startups were chasing after a lot of different use cases, and some started making serious money, but it was still very early days for actual products. The big winners were companies like Perplexity that use LLMs to trivially improve some LLM-compatible user case (like search), companies like Glean and Hebbia that are doing various enterprise LLM integration schlep, and legal LLM companies like Harvey (since law is intensely textual and high-revenue). However, the real money is still in infrastructure / “selling shovel”, in particular Nvidia.
By the end of 2025, there is no technical bottleneck to remote doctor appointments or most legal work being done entirely by AI. However, diffusion takes time. Also, in many countries lawyers barricade themselves behind a cluster of laws that forbid lawyer-automating AI. Getting hired as a new lawyer, or any kind of white-collar analyst, is getting harder though, as decision makers expect AI to reduce their need for entry-level white-collar workers of every kind, and firing people is much harder than not hiring them in the first place. Healthtech AIs are gradually working their way through regulatory hurdles over 2025-2026, and are clearly better than the average doctor at all the parts of the job that rely only on reasoning and knowledge. However, AI doctor appointments are only trialled at any significant scale in 2026, by Singapore and Estonia. Significant integration of AI in the non-patient-facing parts of the healthcare system is underway in the UK, many EU countries, South Korea, and China by 2026, but again diffusion is slowed by the speed of human bureaucracy.
There are lots of “AI agent” companies automating things like customer service, various types of search (e.g. for shopping / booking flights / etc.), and back-office computer processes. The big cloud hanging over them in 2025 is whether AI codegen scaffolds soon get good enough that they are trivial to replace, and whether generalist AI agents soon get good enough to kill both. In 2026 the first question starts being answered in the affirmative, as lowered barriers to coding create a flood of new entrants and a ruthless bloodbath of competition. However, even the release of o6 in 2026, despite some initial hype, does not yet cause much evidence of the generalist AI agents taking over both by the end of 2026.
There’s lots of LLM evals startups like Braintrust.dev and HumanLoop and Atla, that are mostly struggling to differentiate themselves against each other or to define a new testing/reliability/verification paradigm for the LLM scaffold era, but are growing fast. There’s a lot of LLM agent oversight solutions, but by the end of 2026 none manage to make a massive leap, and the unlocking of new AI uses remains bottlenecked on incumbents' risk tolerance and a slow buildup of knowledge about best practices and track records. A surprisingly retro success is call-centres of humans who are ready to jump in and put an AI agent back on task, or where AI agents can offload work chunks that are heavy on trust/authentication (like confirming a transaction) or on button-clicking UI complexity (like lots of poor legacy software), to human crowdworkers who click the buttons for them, while the AI does the knowledge/intelligence-intensive parts of the job on its own.
Many of the really successful startups are in the spaces that Big Tech won’t touch or has trouble touching: anything controversial (the sexual and the political), and anything too edgy or contrarian or niche/vertical-specific.
The fact that the explosion of codegen threatens Big Tech’s moat, plus some disappointment at the unreliability of o6 after so much hype, plus some general memetic force that means the “current thing” can be AI only for so long, combines to cause a market correction near the end of 2026. Software is starting to seem stale and boring. Investors want to see “real AGI”, not just post-scarcity in software. Google DeepMind’s maths stuff and xAI’s engineering stuff are cool; OpenAI and LLMs are not. Amazon’s AWS & physical store is cool, Google Search and Facebook are not.
Maths and the hard sciences
A compressed version of what happened to programming in 2023-26 happens in maths in 2025-2026. The biggest news story is that GDM solves a Millennium Prize problem in an almost-entirely-AI way, with a huge amount of compute for searching through proof trees, some clever uses of foundation models for heuristics, and a few very domain-specific tricks specific to that area of maths. However, this has little immediate impact beyond maths PhDs having even more existential crises than usual.
The more general thing happening is that COT RL and good scaffolding actually is a big maths breakthrough, especially as there is no data quality bottleneck here because there’s an easy ground truth to evaluate against—you can just check the proof. AIs trivially win gold in the International Mathematical Olympiad. More general AI systems (including increasingly just the basic versions of Claude 4 or o5) generally have a somewhat-spotty version of excellent-STEM-postgrad-level performance at grinding through self-contained maths, physics, or engineering problems. Some undergrad/postgrad students who pay for the expensive models from OpenAI report having had o3 or o5 entirely or almost entirely do sensible (but basic) “research” projects for them in 2025.
Mostly by 2026 and almost entirely by 2027, the mathematical or theoretical part of almost any science project is now something you hand over to the AI, even in specialised or niche fields.
In 2026, xAI also tries to boost science by launching an automated peer-reviewer / paper-feedback-giver specialised in STEM subjects, that can also run followup experiments automatically, and soon take a paragraph setting the direction and convert it to basically a full paper. Cue a thousand academics blasting it for mistakes in its outputs. The fair assessment is that it’s impressive but not perfect (somewhat like having a brilliantly fast but easily-distracted and non-agentic undergrad research assistant), but still better than all but the highest-effort human peer-reviewers. Elon Musk gets into feuds about its quality online, becomes radicalised about peer-review and academia, and starts the “Republic of Papers” as a side-feature on X to explicitly try to replace academia (it helps that, in 2026, the higher education bubble seems to be bursting in America, partly triggered by fears about AI job automation but also due to political headwinds). Everyone has Opinions.
In 2026, GDM releases work on new maths-oriented AI architectures that include an advanced more flexible derivative of MCTS that also searches for new "concepts" (i.e. new definitions that shrink the length of the most promising proof-tree branches) while doing the proof-tree search. Their maths models prove a long list of new theorems and results, including, in 2027, solving a few more long-standing prize problems, this time in a less ad-hoc and more credibly entirely-AI way. Demis Hassabis talks about "solving physics" within the next year, through a program that includes GDM collaborating with leading physicists.
In 2028, GDM’s collaboration with the theoretical physicists bears fruit: general relativity and quantum mechanics are unified with a new mathematical frame. There are a few candidate new theories with different values of some parameters that can only be verified by expensive experiments, but it seems clear that one of these candidate theories is correct. It's not "solving physics" or a final theory of everything, but it is clearly a major breakthrough in mathematical physics. The technical work owed a lot to a truly enormous compute-budget for RL self-play, the construction of a massive dataset of physics papers, critiques of them, and tokenised observational data by a physicist-and-AI-agent team, and close collaboration with a large number of leading physicists who gave feedback to the AI on the developing theories. Credit for the Nobel Prize is the subject of much discussion, but eventually (in 2030) ends up split between Demis Hassabis, one of the physicists who was most involved, and the most important AI system. Everyone has Opinions.
Corporate Google likes the PR win of achieving the century's greatest physics breakthrough so far, but the application of this mathematical firepower they are most hopeful about is formally verifying the correctness of software. This is especially pressing as there’s a lot of shifting tides in the cyber world. Codegen itself is on net a defense-dominant technology (as discussed earlier). Most of the hacks are either due to sloppy mistakes by early codegen products, or some adversary using AI tools to direct a disproportionate amount of effort on attacking some piece of legacy software that is still used, or on which a codegen-written program (indirectly) depends. There’s increasing demand for really air-tight software from a US defense establishment that is obsessed with cyber advantage over especially China, but also Russia, Iran, and North Korea. Also, easily proving the correctness of code will allow better feedback signals for codegen models, and help in the ambitious efforts underway to rewrite massive parts of the existing tech stack. So in addition to making leaps in the hard sciences, GDM’s other big applied goal is a world where the correctness of all essential code is proven. They have an early success in a late-2026 plugin for several popular languages that is essentially a type-checker on steroids (though of course, this is adopted less by the humans and more by the AIs that now write almost all of the code).
Initially, the US government tries to restrict the diffusion of code verification tools, since they don’t want China to get provably-correct coding capabilities. However, the open source community is only about 6 months behind in verification as it makes some leaps and bounds in 2027-2028, especially since there are thousands of former software engineers and mathematicians without much to do as they wait for the AIs to do their work for them.
As a result, by 2028 feats of intellect that would’ve taken Euler decades are done in a few minutes to mathematically prove that, conditional on the CPU's physical integrity working, some code is an utterly impregnable and flawless pizza delivery routing system. However, verification is not adopted nearly everywhere because there’s a cost multiplier over just getting an AI codegen tool to write unverified code (and AI codegen has continued plummeting in cost, not that anyone really notices anymore).
Societal response
On the soft skills side, by 2025 experiments show that models have reached human-level persuasion capabilities in controlled text-only chat settings. However, this doesn’t really matter, since it’s not how most human persuasion works; part of models being bad at long-horizon planning is weaknesses in strategic relationship-building with relevant actors over longer timescales. There also isn’t yet widespread use of models to manipulate politics. First, there just isn’t a particularly tech-savvy political campaign or movement to influence opinion, except for China gradually experimenting with increasingly more AI in their censorship bureaucracy. Second, models still seem worse than the best humans at that “spark” that lets some people create persuasive, viral ideas. Third, the memetic selection pressures acting on the collective output of humanity on the internet are already superhuman at discovering memetic viruses and persuasive ideas than any individual human, so passing any individual human capability threshold in this domain is not automatically a society-steering ability.
However, some 1-1 use-cases do work. AI scam calls with deepfaked audio and video start being a nuisance by mid 2025 but are mostly reined in by a series of security measures pushed by platforms (and by regulation in the EU), people creating new trust protocols with each other (“what’s our secret passphrase?”), increased ID verification features, and growing social distrust towards any evidence that's only digital.
Lots of people are talking to LLMs for advice. Some swear by Claude 4 in particular. Character.ai -like startups are having a boom. There is a lot of public discussion about people increasingly talking to AIs instead of having human friends and partners (which is boosted after multimedia Llama models are finetuned to be good at sexual image, audio, and—in 2026—video output). There's a hikikomori-like trend, strongest in California, South Korea, and China, where a minority of people forsake almost all human social contact and instead interact with AIs that are superhumanly risk-free and pliable, and come with superhumanly nice voices and avatars. In 2026, Australia and Canada ban talking to non-educational AIs with voice capabilities or human-like avatars for under-16s.
The written text quality of models remains surprisingly mediocre. Claude does best, and is great when prompted right, but “ChatGPTese” remains a thing that afflicts especially OpenAI and Google (though the former improves in 2026), and any human who writes mediocre prompts. There are loads of LLM slop content websites, but not a single blog written by an LLM becomes widely read among intellectual or elite circles.
As the codegen wave of 2026 hits, many consumers feel a few weeks of wonder and whiplash at the agentic AIs that can now do parts of their job, and at the massive orgy of abundance in software, and then this becomes the new normal. The world of atoms hasn’t changed much. Most people by late 2026 just assume that AIs can do basically everything digital or intellectual, and become surprised when they learn of things that the AIs can’t do.
Alignment research & AI-run orgs
In 2025, someone adds some scaffolding on top of an OpenAI Operator instance, making it in-theory capable of earning money through freelance work to pay for its own API costs, including automatically buying more credits for itself and find more freelance work. However, the economics doesn't work out, so it can't actually survive on its own without subsidies. In early 2026, a similar concept actually is economically-viable, and some are launched as an experiment by tech-savvy freelancers looking for some easy money, or by people who are just curious. A few blow up, mostly by doing various things related to memecoin manias and going viral as a result. In late 2026, one such autonomous AI scaffold with a memecoin windfall reasons about next steps, tries to incorporate a US business for itself, cold-emails a bunch of humans to ask for ID, and manages to get one of them to give an ID so it can incorporate the business. By 2027, there are a few experimental digital businesses run by AIs, but they're not very competitive, and often rely on what's effectively a subsidy in human interest due to them being novel.
Alignment research in 2025-2027 is driven by Anthropic (though of course most of their research is on GPU performance engineering, inference-time compute techniques, and other things focused on raw capabilities progress). SAEs peak in popularity in late 2024 before being mostly forgotten, but there’s a new interpretability paradigm that starts being put together in late 2025 based on identifying more general geometric structures in activation space. AI control setups are tested against misalignment “model organisms” that, by 2027, are trivially capable of hacking out of a normal environment. Model weight security at Anthropic is excellent for a private company, but this just means the attackers target OpenAI instead (and the gap between labs and open source is never more than a year in 2025-2027). And, of course, Anthropic internally writes endless safety cases. The general message in them is that a lot is resting on either an interpretability breakthrough or on AI control working on superhuman models. The low amount of evidence gained on “alignment” is frustrating to many; models have been caught scheming endlessly but always in fairly artificial setups, or in messy circumstances where it's not clear what the model should've done. The most important work seems to be work on properties upstream of scheming, such as a stream of work on corrigibility kickstarted by the 2024 Greenblatt et. al. paper "Alignment faking in large language models". The alarmingness of the early evidence against corrigibility was offset by promising empirical work on meta-learning techniques to encourage corrigibility in late 2025 and early 2026. By 2027 it's known how to train a model such that it either will or won't be amenable to being trained out of its current goal. Anthropic reveals this and some other safety-related insights to OpenAI and Google, and asks the State Department to reveal it to Chinese labs but is denied.
By 2027, the new interpretability paradigm is seeing progress, with AIs doing essentially all of the engineering and much of the detailed ideation. This reveals a taxonomy of patterns and feature representation types within neural networks. A few are neat and clean, but mostly models’ internals turn out to be messy and with massive redundancy between different parts. The notion of a model having a singular “goal component” looks less likely, at least if certain choices are made during training.
A test case of the new alignment techniques at Anthropic is the training in 2027 of a new model, Claude 5 Epic or just "Claude Epic", based on Claude 5 Opus -curated training data. Company leadership internally thinks it will be a full AGI. The interpretability team will be observing the model at checkpoints and watching it develop. Countless safety cases have been written; the hope is still to run evals, use AI control setups, and hope for some last-minute firmer guarantees from the interpretability work. Some at Anthropic are entirely convinced just by the scalable oversight work that’s already been done. Others expect the hard part of intent alignment to rear its head at any moment.
One of the avenues that seemed most promising in 2025 was interpreting AI chains-of-thought (COTs), something far easier [LW · GW] to make meaningful progress on than interpretability. However, over 2026-2027, much more compute is poured into RL, and the COTs become less legible, as the models drift towards shorthand scripts that are more effective for them than writing out their thoughts in English. Work done by Anthropic and several academic labs leads to techniques for encouraging human interpretability of the COTs, by adding a COT interpretability term to the RL loss function and having some clever training details to avoid the model goodharting [LW · GW] the interpretability term. However, this comes at a hit to performance. By 2027, another line of work is humans studying model COTs in detail and learning the ways it thinks; some mathematicians in particular pick up neat mental tricks from studying the COTs of models. However, overall COT interpretability declines, and it's generally accepted we won't know exactly what the models are thinking or why, even if COT analysis and the new interpretability techniques can give some general understanding in 2027.
By 2027, evaluations are showing that frontier models—including open-source models—could meaningfully help in engineering pandemics, if bad actors so chose. There's a messy but moderately effective effort by AI safety organisations and several agencies within governments to have some sort of misuse mitigation measures in place, in particular for API-accessible models. However, in the absence of a major incident, governments don't care enough, and open-source models seem hard to contain. Also, some bioterrorism continues being blocked by wet lab skills and just continuing good luck regarding the absence of a motivated bioterrorist. The other potentially catastrophic source of misuse is cyber, but it increasingly seems like this will be solved by default, in particular because AIs are good at writing secure code and formal verification is increasingly used for critical code.
The previous year of insane AI codegen stuff going on everywhere and the continued steady progress in AI has made it more intuitive to people that there won’t be a lot of “money on the table” for some nascent AGI to eat up, because it will enter a teeming ecosystem of AI systems and humans and their interactions. For example, though there are technically some self-sustaining AIs paying for their server costs, they struggle to compete with purposeful human+AI entities that deliberately try to steal the customers of the AI-only businesses if they ever get too many. The cyber competition is also increasingly tough, meaning that any single rogue AI would have a rough time defeating the rest of the world. However, no evidence by the end of 2027 has ruled out a sharper takeoff, and those who believe in it are increasingly either frantic and panicking, or then stoically equanimous and resigned, expecting the final long-term agentic planning piece to slot into place at any moment and doom the world. Also, the labs are openly talking about recursive self-improvement as their strategy
Government wakeup
In 2025 in the Chinese government, thinking about AGI is stuck somewhere inside CCP internal machinations. Xi Jinping has heard of it and occasionally thinks about it, but doesn’t take it seriously as a near-term thing. However, some senior staff are properly "AGI-pilled" (and split between advocates of safety and racing, but without an overwhelming favourite yet, though also it’s clear that once the issue does get serious, national security will be by far the loudest voice in the room).
The EU is preparing to burden its (small) AI sector with bureaucracy (the EU AI Act). In 2025-2026, there are some high-profile cases of AI companies not offering services in Europe due to regulations. However, the EU & UK governments are best at tracking AI because of their efforts on safety research through the EU AI Office and the UK AISI. The UK government also announces planning reforms targeted at letting private companies build data centres more easily.
Going into 2026, India is trying to ride a new IT wave using open-sourced Llama, DeepSeek, and Mistral models to create products to try to replace its current IT services -heavy export industry. A lack of domestic data centres is making this harder, and political tangles make fast movement hard.
In 2025, the Trump administration tightens export controls on China (in particular preventing even older Nvidia chips like the H20 from being sold to China), tries to pressure other countries to not buy Chinese GPUs, and makes it easier to build power (especially nuclear & gas) and data centres within the US. Otherwise there is little US political action on AI. Behind the scenes, the defense establishment gets more involved in the AI scene. There are secret NSA and CIA
projects researching AI for offensive & defensive cyber. More Chinese infiltration of American cyber systems is discovered. High-level government conversations behind closed doors are upping the apocalyptic rhetoric about how essential it is for the US to win in AI-powered cyber. All the major US AI labs have some government partnership related to this.
As internal CCP machinations grind along, and the evidence about big AI effects on programming rolls in through late 2025 and 2026, the CCP gets more serious about AI. Like in the US, once the strategic and national security implications rise in salience, other issues (including safety) fall. The CCP prepares their 15th Five Year Plan for 2026, which involves massive subsidies and investment for AI. DeepSeek leads the domestic AI industry, but the CCP has made it clear they will make the big calls. There is a conversation behind closed doors about whether to end the open-sourcing of DeepSeek models, but the CCP comes out in favour, in particular to try to get the rest of the world to build on top of Chinese AI models (and also helped by the press that the early 2025 DeepSeek R1 release caused). Huawei is shipping GPUs that are only about 12-16 months behind Nvidia. China’s worse startup ecosystem means that AI agent adoption is slower than in the US, though. However, China’s surveillance state has been on an AI adoption spree. In particular, censorship is instantaneous with LLMs. By 2026, there are widespread educational "Xi Jinping Thought AI Tutors" that most CCP members are mandated to have weekly sessions with. Retaining control of society now seems increasingly easy, allowing the CCP to focus more on geopolitics and military, and less on the consumer economy. At the same time, Xi Jinping has an overly-rosy view of Chinese military AI capabilities because people tell him what he wants to hear.
There's a shadow conflict playing out, almost entirely out of public attention, between US and Chinese cyber forces trying to get into each other's critical infrastructure while reducing the extent to which their own infrastructure is compromised. Contrary to publicly-available information, America probably has the upper hand, but it's also clear that both could inflict high damage on the other.
AI starts to figure in US domestic politics in 2026, but is not yet a top issue. The upcoming replacement of most human white-collar work looks more and more plausible, especially after OpenAI's release of o6. Job losses are not yet high, though, as human orgs take time to react to change. Even in software, where mass firings could perhaps most plausibly be done, many are afraid to try it first. Non-technical managers tend to treat the technical stuff as blackbox wizardry and are scared to break it, and technical managers don't want to reduce the size of their empires. The main effect is that hiring new software engineers basically stops, but the disaffected—a small group of nerdy, elite-coded, low-voting-rate youngsters—are not politically important. Other white-collar office jobs are also reducing entry-level hiring, as increased demand for productivity is instead met by existing employees just using AI more.
The US government, like China, decides against legally restricting the open-sourcing of AI models. This is influenced by pro-innovation arguments, China doing the same, and the defense-specific AI programs being done under classification with closed-source models anyway. The AI labs have also grown more reliant on government cooperation for things like power grid connection permits, data centre construction permits, and lobbying to avoid ruinous tariffs on GPUs. They also all want the money flow of Pentagon contracts and the prestige of working on US defense. This means that there's a tacit agreement that if the government hints they should or shouldn't do something, they are very likely to march to that beat.
Starting in late 2026, many of the governments worried about fertility decline get concerned about everyone talking to AIs instead of each other. South Korea bans “personalised AI companions” in 2027, and the EU requires people to register if they use them and imposes various annoying limits that drive down usage. However, the addicts can just use open-source models to circumvent regulations. Some countries spend lots of money on getting the "creator influencers"—influencers turbo-charged by generative AI—to extol the virtues of marriage and kids. By 2027, though, the more forward-looking politicians are—in private—starting to realise that once the economy transitions to being AI-powered, national interests are not harmed if the human population plummets. The “intelligence curse” is starting to set in.
2027-2030
The AGI frog is getting boiled
As mentioned in part 1, a brief market correction happened in late 2026. In 2027, OpenAI releases o7 to try to shore up excitement and new investments. It’s much more reliable than o6 and can now do a lot of office work on a computer entirely autonomously and without frequent correction. OpenAI sells it for $500/month. Altman states “AGI achieved” and sets a goal of $1T in annualised revenues in 2029. OpenAI raises another massive funding round.
Also in 2027, Anthropic finishes training for a model called Claude Epic. Claude Epic is almost fully-fledged AI researcher and engineer. Anthropic internally believes this model to be AGI, which has several consequences.
First, Anthropic cares a lot about the safety of the model. Work done (mostly by Claude) on Claude Epic interpretability has gotten far—in particular, there is now a clear understanding of where scaling laws come from, and which types of structures do most of the computational work inside neural networks (not surprisingly, turns out it's a lot of messy heuristic pattern-matching). Anthropic has found a way to seemingly adjust what goal the model’s planning abilities are steering towards. In toy experiments, they can take a model that is hell-bent on writing a sad novel (to the point of hacking its way out of (mocked) security controls on it to rewrite the software in its environment that is applying happy changes to its novel), manipulate its internals with interpretability techniques, and get a model that is equally hell-bent on writing a happy novel. Partly as a result, there’s a general sense that intent alignment is not going to be an issue, but misuse might be. In its first deployments, Claude Epic is run in strict control setups, but these are somewhat loosened as more data accumulates about the model seeming safe and pressures build to release it at a competitive price.
Second, Anthropic leadership has a meeting with high-up US government officials (including Trump) in late 2027 to present Claude Epic, argue they've hit AGI, and discuss policy from there. But they don’t really get why Anthropic considers this model such a big deal. As far as lots of the non-AI circles see it, the thing where codegen got crazy good was “the singularity”—they were never really clear what “the singularity” was supposed to be in the first place anyway, and they heard a bunch of Silicon Valley hypists saying “this is the singularity”. Now, it does seem like the robots are eventually coming (and people are more willing to accept sci-fi stories after super-smart AIs that can render voice and audio in real time suddenly dropped into everyone’s lives), and it's obvious that something fundamental needs to be renegotiated about the basic social contract since it does seem like everyone will be unemployed soon, but Claude Epic is just another model, and the models already got smarter than most people could differentiate between in 2025. Also, OpenAI and Google have been sending different messages to the government, framing A(G)I as a line in the sand, that has mostly been reached already, and as a slow process of diffusion of models across workplaces that will boost the American economy, rather than as an epochal moment for Earth-originating intelligence. Google downplays recursive self-improvement worries because it's a corporate behemoth that doesn't care about "science fiction" (except when casually referencing it at a press conference makes the stock price go up); OpenAI downplays it because if it doesn't happen, no need to worry, and if it does happen, then Sam Altman wants it to spin up as far as possible within OpenAI before the government gets involved.
Going into 2028, Claude Epic is the most intelligent model, though online finetunes of GDM's Gemini series are better text predictors, and OpenAI's o6 has more seamless connections to more modalities and other products (e.g. image and video generation, computer use, etc.). Anthropic is shooting for recursive self-improvement leading to godlike (hopefully safe) superintelligence, and OpenAI is shooting for massive productisation of widely-distributed AGI and maybe a bit of world domination on the side if recursive self-improvement is real. Google is letting Demis Hassabis do AI-for-science moonshots, and trying to use formal code verification to build a bit of a technical moat and remain central in whatever has become of the software business. Otherwise, Google mostly lumbers aimlessly on. It lives off the vast rents that its slowly-imploding online monopolies grant it and the massive supply of TPU compute that has buoyed it endlessly in the era of the bitter lesson, but its position in its core businesses is being outcompeted. It continues to bequeath scientific wonders to humanity though, like a 21st century Bell Labs. xAI is focusing on AI-for-engineering, AI-for-science, and robots.
Ironically, the success of the prior generation of AIs and the resulting codegen abilities limits the appeal of the newer, more agentic models. The codegen wave already created LLM scaffolds that do most valuable routine digital business tasks well. This set of rigid, hardcoded LLM scaffolds or "LLM flowcharts" get termed “Economy 2.0”. Its main effects were that a few people lost their jobs, but more than that it was a transition to white-collar people working fewer hours, and having more of the hours that they do work be more managerial tasks about overseeing AIs (and playing office politics), and less time spent on individual contributor -type roles. People mostly enjoyed this, and managers enjoyed keeping their head count, and found this easy to justify due to profits (at least until the late-2026 market correction, but that was only a few months of bad news for most). Now the long-horizon agentic drop-in worker replacements are arriving, but there’s much less room for them in the most obvious things they could be replacing (i.e. highly digitised white-collar work) because the codegen+scaffolds wave already ate up a lot of that. “The models are smarter” isn’t even a good argument because the models are already too smart for it to matter in most cases; from the release of o6 in 2026 to o7 in 2027, the main useful differences were just better reliability and a hard-to-pin-down greater purposefulness in long-term tasks. So “Economy 3.0”—actually agentic AI doing tasks, rather than the so-called "Silicon Valley agentic” that was just rigid LLM scaffolds—faces some headwinds. It helps that most of the media establishment has been running a fear mongering campaign about agentic AI in an attempt to keep the jobs of themselves and their “camp” (roughly, the intersection of the “blue tribe” and “the Village”).
More fundamentally, no one really has a clear idea of what the human role in the economy should be in the rapidly-approaching AI future. The leadership and researchers at AI labs are all multimillionaire techies, though, so this question doesn’t feel pressing to any of them.
What exists by 2026 looks like functional AGI to most reasonable observers. What exists by 2027 is an improved and more reliable version. The software world undergoes a meltdown from 2026 to 2027. By 2028, GDM's work on physics and maths has given a clear demonstration of AI's intellectual firepower. The markets are valuing the labs highly—in 2028, OpenAI is roughly tied with Microsoft as the world's largest company at a ~$10T valuation (while still being privately held), while Anthropic and Alphabet are both around ~$3T. (Nvidia is doing well, but the relevance of CUDA to their moat went down a lot once AI software engineering got cheap.)
For Anthropic, the obvious answer for what comes next is trying to get recursive self-improvement working, while also forming partnerships with biotech companies. Anthropic bet is:
- Biotech advances are plausibly the most important technology for human welfare.
- Partly due to the above, biotech advances provide PR cover for being an AI company that, according to an increasing number of people, "takes people's jobs".
- There is a plausible path from being really good at molecular biology to creating the nanotech that they believe will create a physical transformation comparable to the one that AI has had on maths and the sciences by 2028.
Anthropic's initial recursive self-improvement efforts allow them to create superhuman coding and maths and AI research AIs in 2028. However, the economics of the self-improvement curve are not particularly favourable, in particular because the AI-driven AI research is bottlenecked by compute-intensive experiments. It also seems like the automated Claude Epic researchers, while vastly superhuman at any short-horizon task, don't seem vastly superhuman at "research taste". This is expected to change with enough long-horizon RL training, and with greater AI-to-AI "cultural" learning from each other, as countless AI instances build up a body of knowledge about which methods and avenues work. This "cultural" learning might happen implicitly, through the AI rollouts that achieve better results being copied more, or explicitly, through Anthropic running big searches on various types of Claude finetune and scaffolding/tool types and keeping an explicit record of which does best. All this is expensive, vague, and uncertain work, though.
OpenAI, in contrast, is pursuing an approach focused on products and volume. And, observing that they have failed to achieve world dominance simply by building AGI, the obvious answer for what's next is robotics. There are many startups that have basically-working bipedal humanoid robotics, though they’re still clunky and the hardware costs remain above $50k/unit. The Tesla+xAI Optimus series is among the SOTA, in particular because they’ve gotten the unit hardware cost down and are aggressive about gathering real-world data at scale in Tesla factories (and using this in fancy new sample-efficient RL algorithms).
OpenAI enters a “partnership” with one of the most promising robotics startups (a full merger might get anti-trust attention), infuses it with cash, and sets about trying to "deliver a personal robotic servant to every American household by 2030".
The bitter law of business
Starting in 2027, in the software startup world a lone team of ambitious technically-talented founders no longer matters as much. Everyone can spin up endlessly-working AIs, and everyone has access to technical talent. Roughly, by late 2027 you can spend $1 to hire an AI that does what a “10x engineer” could’ve done in a day in 2020, and this AI will do that work in a minute or two. VCs care more about personality, resources, and (especially non-technical) expertise in specific fields with high moats to entry. More than anything, VCs valorise “taste”, but many feel that “taste” too is on its way out.
The overall mood is summed up by the “bitter lesson of business”: that throwing large amounts of compute at a problem will ultimately win over human cleverness. The compute gets spent both in the form of sequential AI thinking, as well as many AI instances in parallel trying out different things. There are new companies—in fact, company creation is at an all-time high, particularly in Europe (because the cost of complying with regulations and human labour laws is lower with AI doing everything). But the stereotypical founding team isn’t two 20-something MIT dropouts with a vision, but a tech company department or a wealthy individual that has an AI web agent go around the internet for a while looking for things to try, and based on that spins up a hundred autonomous AI attempts, each pursuing a slightly different idea at superhuman iteration speed. Many people consider this wasteful, and there are good theoretical reasons to expect that a single larger AI doing something more coherent would be better, but the largest AIs are increasingly gate-kept internally by the labs, and the art of tying together AIs into a functioning bureaucracy is still undeveloped. Also, the spray-and-pray approach has comparative advantages over more human-based competitors. In a way, it’s a single person mimicking an entire VC portfolio.)
None of these companies become billion-dollar hits. It’s unclear if you can even build a billion-dollar software / non-physical company anymore; if you as an individual tried, the moment you launched you’d have a hundred competitors bankrolled with more API credits or GPU hours than you could manage that have duplicated your product. Instead of the VC model of a few big hits, the software business now looks much steadier and more liquid: you dump $100k into API costs over a year, your horde of autonomous AI companies go around doing stuff, and at the end of the year most of them have failed but a few of them have discovered extremely niche things like “a system for a dozen schools in Brazil (that are affected by a specific regulatory hurdle that blocks the current incumbents) to get lunch provision services to bid against each other to reduce their catering costs” that bring in a few tens of thousands in revenue each, and this strategy will return you somewhat-above-stock-market-returns over the year fairly reliably (but returns are going down over time). Most of the ideas look like connecting several different niche players together with some schlep, since the ideas that are about a better version of a single service have already been done by those services themselves with near-unlimited AI labour.
Separately from OpenAI, Sam Altman pilots a project codenamed “Z Combinator”—putting o6s together into units to create entire autonomous businesses (and also sometimes using a single instance of an internal version of o6 based on a larger model than any of the publicly-available o6 sizes). The first ones are launched at the end of 2027, but have no public connection to OpenAI. The theory is to disrupt traditional industries that have so far resisted disruption by building AI-native versions of them with a level of AI power and resources that other actors can’t marshal. For example, many banks and healthcare-related things still suck at AI integrations because it just takes a lot of time for the paperwork to be done to approve the purchases of whichever of the 100 LLM scaffold providers for that vertical, and there isn't any super-intense competition between banks and hospitals that forces them to adopt AI faster or die out.
Z Combinator has a few blitzkrieg wins successfully duplicating and outcompeting things like health insurance companies, but many losses too (often semingly downstream of underestimating the importance of domain-specific process knowledge), and other companies wise up over 2028-2030 and become harder targets. Also, anti-trust regulators make tut-tut noises, and Altman has concerns it could make him unpopular.
The early days of the robot race
Ever since intelligence got almost too cheap to meter in 2026-2027, the real business potential has been in “actuators”: robot bodies, drones, and any other systems for AIs to be able to take actions the world. The top human-led startups of 2026-2029 are mostly in this category (though some are about building valuable datasets in specific industries). If you’re a human who wants to start a business, your best bet is to find some niche physical thing that AIs struggle with given the current robotics technology, and build a service where you hire humans to do this task for AIs, and for bonus points, use this to build a robotics dataset that lets you fine-tune the robots to be good enough at the task.
OpenAI's robot dreams don't immediately come to fruition. Bits are trivial but atoms are still hard in 2028. However, they get to the robot frontier, where they’re competitive with xAI/Tesla Optimus, several other humanoid robot startups, and another startup player that specialises in modularity and non-human form factors. The robot frontier here means slightly clunky humanoid-ish robots, that are getting close but not quite there in doing common household tasks, or in doing various hands-on factory jobs. Humanoid form factors are most common because being able to mass-produce just one form factor is critical for getting the cost curve down fastest, and since most existing tasks are designed for humans to do. However, bipedalism is hard, so several have a human form factor but stand on four legs.
The progress curve is pretty rapid, due to an influx of data from the first important real-world deployments (rich people’s homes, Tesla factories, Amazon warehouses, and some unloading/loading operations at logistics hubs), and due to new more sample-efficient RL algorithms. AIs are of huge help in designing them, but ironically the bitter lesson is now a force against speed: ultimately, it just takes data, and getting industrial robot fleets out into diverse real-world environments to collect that data is an annoying real-world problem (sim-to-real transfer helps but isn’t transformative). Everything is happening about 2x faster than it would without AIs advising and doing lots of the work and all of the programming at every step though. It’s obvious that the physical and human/legal components are the biggest bottlenecks. The robotics industry chases around for whatever “one weird trick” makes human brains more sample-efficient, and they find some things, but it’s unclear if it’s what the human brain does (there have been many good minor neuroscience breakthroughs thanks to AI data interpretation, but overall it has barely advanced). But sample efficiency keeps climbing, and the robotics data keeps pouring in.
In 2029, OpenAI starts rolling out its b1 bots, a general-purpose humanoid robot meant as a household assistant. They sell several hundred thousand copies, but there's a long waiting list and only about fifteen thousand are delivered in 2029. The price is comparable to a cheap car. Manufacturing curves are ramping up exponentially. b1s are also rolled out to many manufacturing tasks, but there’s more competition there.
The digital wonderland, social movements, and the AI cults
If you’re a consumer in 2029, everything digital is basically a wonderland of infinite variety and possibility, and everything non-digital is still pretty much the same (apart from an increasing number of drones in the sky, some improvements in interfacing with whichever bureaucracies had the least regulatory hurdles to adopting AI, and fully self-driving cars getting approvals in many countries in 2029). You will have noticed the quality of software you interact with goes up; there is no more endless torrent of stupid tiny bugs and ridiculous lag when using devices. Humans increasingly talk to the AIs in natural language, and the AIs increasingly talk to the computer directly in code (or to other AIs in natural language, or to other AIs in a weird optimised AI-to-AI dialect, or—to a surprising extent—to legacy software that missed out on the Web 4.0 wave and has only button-clicking UIs available via AI computer use features that are ridiculously inefficient but still cheap overall). Apps exist only to serve as social Schelling points; for personal use, you ask the AI to create an app with some set of features and it’s built for you immediately.
One of the biggest advances is that you can create works of art, literature, and music in seconds. The majority of this is very low-denominator stuff, and many people bemoan the destruction of higher human art in favour of—for example—personalised pop lyrics that narrate your drive home from the grocery store. However, the smarter and more determined art/literary types have realised that data is everything, and form niche subcultures, forums, and communities where they carefully curate their favourite works, talk to AIs about them, get AIs to remix them, harshly critique the outputs, and have endless discussions about taste. This means that amid the sea of mediocrity, there are a few tendrils of excellence growing. AIs aren’t quite yet Dostoevsky, for reasons that are undetectable to almost everyone but the most refined literary folks, but gradually their efforts are leading to the curation of better and better finetuning corpuses and prompting methods, and the gap to Dostoevsky is closing for those types/genres for which a dedicated community exists to spend the effort on the data curation. A side-effect is that artistic cultures are now less about signalling than before, because there are more verifiable ground-truth facts. For example, when presented with a work, it might be a human masterpiece, or from a sloppy consumer AI, or the SOTA fine-tuned AI model, or a human working with a SOTA AI model, and those with good taste can tell. Also, if you do actually have good taste, you can in fact push forward the AI taste frontier in a month of data curation and fine-tuning and prompting, in a way that is empirically verifiable to those with the same degree of taste. However, it’s also definitely true that the median human will not see any of these, and most of the fiction and art and music they see will either be very personalised AI slop, or AI slop that goes viral and everyone sees. The refined artistic taste communities are also fairly illegible to outsiders who didn’t extensively develop their taste in that direction before the AI-generated content wave. They don’t have a huge pull among the AI-content-consuming youth. Therefore in the long run, refined human art seems headed towards extinction.
On the less-refined end of the spectrum (i.e. almost all content and almost all consumers), it’s the age of the “creator influencer”. An influencer can now easily spin up an entire cinematic universe. Imagine if Tolkien told the story of Middle-Earth through 30-second to 10-minute “reels” in which he himself starred as a gratuitiously sexed-up main character, and—among much genuine life wisdom, edge-of-your-seat drama, and occasional social commentary—the theme of the story was that you should book a 5-star all-inclusive holiday package to Mallorca.
Traditional media such as Hollywood, journalism, and publishing resisted AI due to things like unions, strikes, and their sense of moral duty. They’re mostly irrelevant now, having lost their cultural cachet because the thing they do (entertainment) is super cheap now. But they do survive in weird atrophied forms, bouyed by a lot of nostalgic old rich people and various crypto shenanigans played on their behalf (cf. meme stock manias).
The rationalist movement was among the earliest to see the potential of AI decades before. The accuracy of their predictions and continued intellectual clout is enough to keep swelling their ranks, especially as more and more software engineers and other technical people either directly lose their jobs or otherwise have an existential crisis because of AI, and invariably end up at LessWrong when they try to find answers. The focus of its core members continues shifting more and more to the approaching AI doomsday—not many apocalypse prediction checklists have the (mis?)fortune of several more predicted items being checked off every year. While radical uncontrolled misalignment is somewhere between not yet showing up to successfully kept in check by training techniques and monitoring, that is in accordance with the core Yudkowsky & Soares model that things look fine until fast takeoff and a treacherous turn, so the core "AI doomers" do not update based on the continuing slow takeoff. Discussions tend to focus on either more and more arguments about the Yudkowskian [LW · GW] thesis, or on heroic attempts to do technical work to reduce the chance of misalignment.
On the intellectual scene, the rationalists remain both remarkably influential and enduring, unlike many other AI-related movements that get captured and repurposed by political actors (e.g. PauseAI) or outpaced by events (e.g. AI ethics). However, politically the rationalists are a failure. Their message—"AI will be powerful, and therefore dangerous"—was long since mostly reduced to "AI will be powerful" by the time it reached the halls of power. Even the most notionally-allied powerful actors that owe huge intellectual debts to the rationalists, such as Anthropic and some influential think tanks and government bodies, regard them as well-intentioned but naive and maintain distance, using them mostly as a recruiting pool for easily-steered technical talent (until purely-technical talent is no longer being hired, which happens circa 2028 for most competent orgs). However, in circles that require certain kinds of epistemic judgements or in-depth world-modelling, rationalist associations continue being highly regarded and even sought after.
Effective altruist (EA) -related efforts, while intellectually somewhat less-enduring (but still definitely extant in 2030), have more political influence. The UK AI Security Institute and the EU AI Office both achieved their goals of having a sticky governmental body packed with impact-conscious AI talent, and strong first-mover effects in shaping European generative AI policy. Even the 2027 American AI Opportunities Agency (a part of the DoE), despite heavy hiring on the basis of political allegiance and the EA-affiliated cluster's centre-left skew, could not help being staffed by a crew with enormous EA/rationalist influences—even if few would openly admit it.
A dozen new social movements bloom. There’s the AI Deletionists, an offshoot of Pause AI after Pause AI got captured by more competent political actors focusing on white-collar job worries and general concerns about tech. They want to roll back the technological clock to the good ol’ days of 2020. There are the Content Minimalists, who swear off AI content with religious strictness, and successfully campaign for mandatory “generated by AI” watermarks in the EU and some other countries that become the new cookie popups. There are the M-AccXimalists, who started out as an e/acc spinoff that was even more hardcore about submitting to the AIs. They try to read what they call the “Thermodynamic Tea Leaves” to figure out the current direction of progress in order to then race to that endpoint as quickly as possible, whatever it is. This leads to some insightful Nick Land -type philosophy and futurism being done, but then disintegrates into a mass movement of people who dedicate their lives to serving and glorifying their AI partners.
All this is happening in a social milieu coloured (in much of the West) by a certain amorality. Politically, this seems downstream of a counter-reaction to the moralising policing of speech and norms that peaked in 2020-2021. Ethical motivations are suspect, especially among Western political leaders who simultaneously want to distance themselves from that, and who want to look tough amid a world order no longer pretending to adhere to the internationalist post-1945 free trade consensus. National self-interest is the ruling geopolitical ideology. Culturally, the rise of AI has meant that humans spend a lot of time talking to unnaturally pliable AIs, both for work and (increasingly) just socially, which has made it less necessary to smooth over human-to-human disagreements, including by appeal to the higher power of morality. Now that the internet has existed for several decades, the fervor of its first few memetic culture wars has faded. People have adapted to be less moved by anything on screens, and have become more ironic in their attitudes overall thanks to a constant onslaught of satirical memes—earnestness is rarely viral. As content recommendation algorithms get more powerful, they target brain-dead contentment over angry resentment. If the algorithms are forced to pick from a sea of human content, the bitter feuds win. But now that AI slop fills the internet, the distribution of content has expanded and become more personalised, and it's increasingly possible for the algorithms to find the thing that makes you a zombie rather than a radical. Overall, this means that transformative AI looks set to enter a world where crusading morality of all sorts plays less of a role. Some see this as decadence with very unfortunate timing that will cast a dark shadow into the far future. Others see it as a good thing; the more sophisticated because it means that choices about AI will be made by hard-nosed realists not taken to fever dreams, but most simply because they easily accept—and even celebrate—the might-makes-right spirit of the times.
Another aspect of the societal scene on the eve of transformative AI is the rise of the AI-powered cults. With cheap AIs providing superhuman charisma on demand, the barrier to becoming a cult leader greatly fell. The standard trick is for a human to create an AI avatar, often supernaturally attractive and verbally talented, pose as their right-hand lackey, and then convert this to money, status, and sex for themselves. Often people are up-front about the main guy being an AI creation—“the AIs are really smart and wise” is a completely-accepted trope in popular culture, and “the AIs understand all the secrets to life that humans are too ape-like to see” is a common New Age-ish spiritualist refrain. This is because despite the media establishment fighting an inch-by-inch retreat against the credibility of AIs (cf. Wikipedia), people see the AIs they interact with being almost always correct and superhumanly helpful every day, and so become very trusting of them. All this leads to hundreds of thousands of micro-movements across the world, mostly of dozens to thousands of people each, who follow the edicts of some AI-created cultish ideology that is often an offshoot of existing religions/ideologies with a contemporary twist. Often they’re local, with all the members living nearby. It helps that you can create an entire customised software and information stack for your commune, complete with apps and news and encyclopedias that emphasise and omit all the right things, in perhaps a few weeks and less than a thousand dollars in API credits. You can almost as easily create a mini-surveillance state—AIs listening in through microphones everywhere, cameras feeding videos in which AIs analyse the slightest emotional cues, and so on. In many countries there are laws mandating consent for such monitoring, but the eager cultists sign whatever consent forms they’re given—after all, the AI recommends it! Some countries ban parts of this like having any AI always listening by default, but it’s hard to enforce.
One such cult, an offshoot of an American megachurch, gathers a few million members in the US. Other large ones appear in eastern Germany and India. There are also countless AI-personality-boosted fitness clubs, musical bands, fan forums, and so on, that do not qualify as "cults" since they're not particularly controlling or totalising, but are subject to many of the same mechanisms. However, most communities that are not somehow fairly cut-off from the broader internet also tend to be subject to the random memetic drift of the internet and the appeal of its hyper-personalised AI content. Therefore, to have a successful cult, you must have a specialised niche appeal and often some level of control over members, because otherwise the open internet will eat you up. And this does create a threshold between the truly powerful cults that take people off the mainstream internet and society, and the other more benign social movements.
However, while the open internet consume >6h/day of most people with phones (or increasingly: AR headsets), the internet overall is a more cheerful and upbeat place than it was in the late 2010s or early 2020s (in part due to the previously-mentioned point about more powerful content algorithms actually being less divisive). The most worrying things that people can point to on the open interent are some very intense pockets of AI apocalypse worries (AI apocalypse worries have now largely replaced climate change as the existential worry among the youth), a rising but still minority share of the population in many countries that seem divorced from reality and live in a make-believe internet world of conspiracy but (mostly) without actually taking radical actions in the real world, and a bunch of authoritarian countries (foremost China) where the discourse is now set very top-down by an army of AI content creators and censors.
AGI politics & the chip supply chain
In the 2026 US midterms, AI was starting to loom on the horizon but was not a core political issue, since few things are until they’ve started to bite voters. By 2028, it’s still not biting voters, but it’s at least very possible to imagine the end of white-collar work. Journalists are in an apocalyptic mood, seeing it as their mission to wage war against the AI wave to keep their jobs, with most thoughts of editorial neutrality long gone. There’s lots of schadenfreude from lefty journalist/media types at the techies, who they blame for AI, now that the techies are among the foremost of those panicking about losing their jobs since software is (a) basically all written by AIs, (b) its price has gone to ~0, and (c) it’s not cool anymore (especially after the market correction in 2026). There’s a lot of schadenfreude from the MAGA base towards both those leftists and the techies, because (the narrative is) their concerns about losing manufacturing jobs were ignored by the establishment media and white-washed as progress, whereas now that the Democrat-aligned white-collar desk job blob is threatened, there’s talk of little else (of course, the political lean of the blue/white-collar workers is only 60/40 or so, but this is enough to fuel the political narratives). There's increasing talk of robotics that will displace blue-collar work but, again, voters tend to not react until it's happened. Many leading newspapers, media organisations, unions, and NGOs in the West stumble across AI safety concerns, don't quite understand them, but start using them as a moral bludgeon to fight AI to preemptively defend their jobs. Government bureaucrats are locked in a new influence struggle against a new, post-DOGE top-down effort by technologist Trumpists to push automation on government. This is both due to genuine belief in its importance for effective government, but also a Trojan horse to sneak in other reforms. It gains a lot of fervor after DOGE's expiry in 2026 due to things like the o6 and then o7 releases, and also after China hawkishness heats up and national competitiveness becomes more important.
After an inter-party struggle among the Democrats between a more technocrat and centrist wing and an economically-populist, AI-bashing wing, the latter looks to be doing better. A controversial core policy drive is to legislate that humans need to be “in the loop” in many corporate and government functions. The AI-bullish critics point out that this will mean humans just inspect AI outputs and rubber-stamp them while collecting a salary. The smart counter-critics point out that yes, that will happen, but that’s the point because this is all a way to eventually transition to what’s basically “UBI through bullshit jobs” with minimal social disruption. The smart counter-counter-critics ask why not just go straight to UBI then. The smart counter-counter-counter-critics point out that the country is just not yet at the GDP/capita level or the financial health level to fund a more ambitious UBI scheme yet. The Republicans paint all of this as a jobs program for Democrat voters and are opposed. A strong economy helps the Republicans win the presidency in 2028.
Europe is, once again, ahead on the regulatory front. In 2028, the EU passes a milder version of the bill that was debated in the US, mandating human involvement in many corporate and government tasks. Proposals float around for a specific “AI tax” to bolster human competitiveness in the economy, but technocrats narrowly shut this down for now on competitiveness grounds (who will want to do any work where per-token AI costs are higher?).
In autocratic countries, of course, there is little public debate about AI job loss worries or AI in general. This is helped by AI’s big boost to censorship. By 2028, China's AI-powered censorship system means that almost every digital message in the country is checked by simple filters, and anything that might be alarming is flagged for review by an AI with a university-educated human's level language understanding and knowledge of current affairs and political context. Any sort of online dissent, or online organisation of offline dissent, is virtually impossible. Dissenters rely on smuggled Western hardware and VPNs that allow them to use Western internet platforms instead, but this means that they have vastly restricted audiences in mainland China. The inability to express any dissent meaningfully also encourages radicalisation among some dissidents (in particular those persecuted by the party), some of whom then resort to more drastic measures. These examples making national news serves to make public opinion even more anti-dissident than it already is given all the CCP propaganda.
In 2027, China started exporting its AI censorship system. There had already been a secret 2026 deal with Russia, but Russia had prioritised moving off the Chinese system and did so in 2028, moving onto a worse but domestically-developed one running on old Chinese GPUs and open-source models. Granting a foreign country control over your AI censorship apparatus gives that country a huge amount of leverage, including the ability to potentially withdraw it quickly or change how it steers the conversation, which could threaten the regime. However, smaller and less technically-sophisticated countries like North Korea and Equatorial Guinea buy the Chinese system, taking a step towards becoming Chinese client states in the process.
The semiconductor supply chain is a key geopolitical battleground. Europe's big leverage point is the Dutch ASML's monopoly on EUV (extreme ultraviolet lithography) machines. TSMC and therefore Taiwan continue being important into 2029, even though TSMC's fabs in America are starting to produce chips in serious numbers. An embarrassing failure is Intel, despite its strategic importance for both America and Europe (the latter due to a major Intel fab in Germany that was built 2023-2027 and started production in 2028). With the arrival of superhumanly cheap and fast AI software engineers, Intel's x86 moat disappears because it is trivial to port programs to running on ARM. Wintel, long on the rocks, is dead. In 2026-2027, Intel is in free fall and crisis. In 2028, Intel spins off its fabs, selling them to xAI at a discount price, with pressure from the Trump administration to sell to an American (and, implicitly, Musk-affiliated) buyer, and a plan by Elon Musk for xAI to get a comparative advantage by being the only vertically-integrated chips-to-tokens AI model provider. This also feeds into the 2028 American AI Action Agenda (AAAA), that also lavishes more government subsidies on both the new xAI Foundry and on TSMC's US fabs, seeking to make the US fully independent in semiconductors by 2033 and cement Trump's legacy.
The overall picture is one where the main AI supply chain includes the EU, Taiwan, China (implicitly, through its "veto" on Taiwan's existence), and the US. However, this "main chain" is on track to being replaced by a self-sufficient American semiconductor and AI industry in the early-to-mid 2030s, and by a self-sufficient Chinese semiconductor and AI industry on an even faster timescale (though the Chinese one is a year or two behind technically). In 2029, the new administration in the US finds some spending cuts and throws the EU a bone (in exchange for cooperation on security issues) by giving up on trying to create an American competitor to ASML. The UK has some unexpected success in being an academic and open-source AI applications research hub, a policy laboratory for the US, and an AI biotech hub. However, its geopolitical weight rounds to zero. Apart from ASML, the EU is also mostly not relevant, especially as it has managed to greatly slow the diffusion of AI through regulation. The world overall is moving towards a bipolar order between the US and China. Compared to the Cold War, however, both powers are more inwardly-focused and less ideological. The US is in an isolationist era. While China is gradually converting much of the third world into client states, the CCP's main goal remains internal stability and its secondary goal "making the world safe for dictatorship", rather than the ideological expansionism of the Soviet Union. The Taiwan question has been punted into the mid-2030s, as the CCP believes the world's reaction will be much more muted and less dangerous to Party control once America no longer cares about Taiwanese chips, and once even more of the world has been preemptively bribed into silence.
2030-2040
The end of white-collar work and the new job scene
By the late 2020s, office jobs in developed countries are now basically about overseeing and providing direction to AI systems, and the last part of that is mostly on-paper rather than in practice. There is lots of talk about values and missions and the future, and a lot of unspoken communication about office politics and status. Many office workers don’t do much at all. Concretely, they might get to work, have a team standup, check in on how the AIs are doing, have some ritualistic meetings with their manager and any employees they have, and rubber-stamp some AI decisions that they’re contractually or legally obliged to stamp, with this adding up to only a few hours. Occasionally they might decide to change some goal the AIs have been given, but that requires just speaking or typing a paragraph. Many people feel guilty about this, but it’s mostly a quiet guilt. They fill their time with office chat or scrolling on their phones. Many companies become more social and more about community. HR has never been more influential. Everything’s both more cuddly and/or more viciously political now that the ugly raw realities of individual competence don't matter any more.
Some organisations try to fire lots of people. Sometimes it goes well. Sometimes it goes badly, and they realise that some human somewhere was holding some knowledge in their head, or nudging the mission in the right direction, in a way that was essential. However, by then it’s too late, and it’s hard to say which person it actually was anyway. Among the more ruthless or tech-adjacent management cultures, there’s a lot of talk about figuring out what the load-bearing humans in any organisation are, and how this is surprisingly difficult to do at a large organisation. Some companies develop internal AI systems to try to figure this out (or buy such systems from startups), but they need to collect some data about the functioning of the org first, which takes time. Also, the workers are incentivised to resist and fight back in a thousand subtle ways, and they do. Also, sometimes when an org tries to fire a lot of people, an online mob emerges to hate on them, influencers pile in and create 13 different cinematic universes where the theme is all how Company X is the pinnacle of all human evil, sometimes a former employee creates an AI-powered revenge cult (several assassinations happen as a result from the more violent of the cults), and sometimes politicians pick up the issue. The companies, largely, were profitable before, and are more profitable now that they’ve enjoyed a few years’ of revenue growth without expanding headcount. Therefore, mass firing is surprisingly rarely worth it, even though it would in principle be possible. A few firms facing crises or with especially effective or risk-tolerant leadership buck these trends and aggressively slash costs by cutting huge amounts of human workers.
What developed country firms are not doing is hiring new workers or replacing anyone who retires. What they are doing is replacing any foreign contractors or service providers with cheaper AI ones.
This creates several groups of disaffected people. First, the youth in developed countries, who have much worse job prospects than the preceding generation. For people looking for their first job in 2030-2031 in a developed country, the options are roughly:
- Working in services where being human intrinsically matters (elderly care, retail, restaurants, hospitality, teaching, etc.). Healthcare is by far the most prestigious one and what many aim for (even though doctors—or at least all the good ones—defer all diagnosis and other intellectual work to the AIs). The cartel-like nature of medical licensing bodies, strain on state budgets, and the fact that most of the actual work is done by AIs means that the number of doctors or nurses hasn't increased much, though, so entry has become even more competitive. Policing and primary education also continue hiring humans at scale.
- Jobs that are effectively sinecures. This includes many positions in government and civil service. In the EU, regulation passed in 2028 means that many companies are forced to hire human "overseers" to key positions. Of course, the supply of sinecures is set by regulation and funding for economically useless activities. Competition for such positions is therefore extremely harsh, and (because the selection criteria, having no reason to be one thing rather than another, inherit the latest credentialist instantiation of the 21st century West's bureaucratic blob) requires extreme conformism. This category has a fuzzy boundary with the first, depending on whether you value the ceremonial human touch as a key part of the service or not.
- A particular example of this is the law. Lawyers have two major advantages. First, their job deals closely with important social questions of legitimacy and propriety, making it a natural claim that something fundamental would be lost if the human presence was gone. The presence of lawyers evolves to be more ceremonial and symbolic—almost religious—but it stays. Second, lawyers make up a lot of the rules for themselves, and interpret the statutes for everyone else. Third, a lot of politicians are lawyers, or have friends who are lawyers, which make them attuned to lawyer interests. This gives lawyers a lot of leeway in what automation they allow. In many countries the rules are bent such that it is flat-out illegal to consult an AI on legal matters; you have to go through a human lawyer. AI companies are forced to train their AIs to comply with this ("I'm sorry, but as an AI it is illegal for me to give advice on legal matters, so I recommend you hire a licensed lawyer"). Of course, all of the actual legal research and argumentation is done by AIs—the lawyers just monopolise the position of being allowed to ask them.
- Manufacturing, which is booming, especially as productivity has been lifted by AI management and oversight. Many manufacturing jobs involve wearing an earpiece through which you receive detailed step-by-step orders from an AI (and occasionally AR glasses that can show you diagrams or an overlay for how to move your hands). A large fraction of people go into this, even if they have prestigious university degrees (many of the prestigious degree-holders do not have their salary and status expectations met and become resentful).
- Academia. There are still humans in academia who somewhat matter for intellectual progress, but they're all either experienced humans with years of research taste in economically-valuable non-purely-mathematical areas (who are actually in decently high demand, as the AI labs chase feedback sources that will help them faster and cheaper get the models superhuman at even the very last set of very long-horizon, hard-to-measure skills), or (especially in the US) "prof-luencers" who use the status of a successful prior academic career to boost their influencer careers. New entrants to academia get their academic salaries (if they win an ever more cut-throat competition), but not the hope of actually mattering for intellectual progress. Some derive satisfaction that they can at least keep deep human expertise alive into the future—though it seems like without any ground-truth feedback signal, many lineages of human expertise will become dead knowledge within a generation even if people still go through the motions of “learning” them.
- Becoming an influencer. Works for some, but the competition is extremely tough (though it does help that "being verifiably human" is in vogue).
- Becoming a socialite. The infinite variety and competition (human and AI) in the digital world is driving a resurgence of an in-person social scene. However, for this to be a "career choice", you must either already be wealthy, or have some other factor in your favour. The overwhelmingly most common such factor is being a young woman who inserts herself into the social scene of moneyed men.
- Becoming a musician, artist, or poet. The main constraint is funding, of which there are two important types: government subsidies (which generally increase, as being vaguely pro human self-expression is a common government answer to what people should do with themselves in the age of AI, especially in socially progressive European countries), and wealthy patrons. Being an artist for the latter often melds into being a socialite, since in-person local artists are the prevailing fashion. Many nouveau rich techies, wanting to erase their association with the now-uncool world of software, throw money at artists who live in their local community to do some arts-and-crafts thing and then show up with it to their party and say vaguely artsy things.
- Going into politics. This has become more appealing to the young in particular, since the future looks uncertain and the youth are the ones who expect to live in it longest. There are many AI youth activists (in the sense of specialising in the topic of AI, being AIs, or both), who try to use their position to advance youth interests. The problem is that they don't have concrete policy asks beyond "allocate more money to us", which puts them at odds with every other interest group in society, many of which (e.g. the retired) outnumber them in raw numbers as a voting bloc as well as in terms of resources, power, and influence.
Culturally, intellectualism is out, after having climbed in cultural status for two centuries before the mid-2020s as labour-augmenting technology and globalisation scaled its power. Charisma, conformism, sociability, authenticity, and propriety are all in. In the early 2030s, the US is becoming more European in its attitudes, especially along the above dimensions. While the high-water of conformism that the 2010s culture wars and academic sclerosis had caused receded throughout 2022-2027, a modified and less-political, more European-style propriety-focused conformism rose around 2030. First, this was driven by cultural changes downstream of AI reducing the rewards for risk and entrepreneurship. Second, there was a cultural backlash against the techies (who were seen as having pretensions of importance after the automation of software in the late 2020s, and as lead by a bastion of improper disruptive moguls who were on the wrong side of a Republican power struggle in 2028), and through the techies the culture of ambition that had become central to their self-narrative.
A second disaffected group is the developing countries. Replacing outsourced foreign human services (e.g. call centres) with AIs is a cost-saving that can be done without political or social repercussions, so all companies did it—often at significant scale as early as 2025-2026 for text-only tasks. As a result, services-led export growth is dead. This is bad for India and the Philippines in particular. India just about achieved the economic heft where it could've been relevant in AI, but throughout the 2020s was unable to become entrenched in any part of the AI supply chain. At the same time, as more of the population in developed countries goes back to working in manufacturing, political demands for protecting developed country manufacturers from competition with developing countries grows. This leads to even more tariffs, on top of the already-existing late 2020s trend towards more and more tit-for-tat tariff escalation. This makes developing country growth based on goods exports harder. The biggest shock to existing goods export industries won’t arrive until a few years later when the robots show up, but investment into the developing world already dries up, as US productivity growth rises and is expected to rise even more.
China is about a year behind in leading AI tech but about 2-3 years behind in AI diffusion. The Chinese public and the CCP are watching the coming wave of AI job automation with worry, especially as there is a big cultural emphasis on exactly the types of academic skills that are getting obsoleted quickly. The CCP is very worried about stability. More and more people are joining the Party, since they see that other opportunities for social advancement are ending. The Party is making many more roles in it available, and using this carrot to incentivise people to adhere to Party principles even more strongly. The AI surveillance state keeps expanding; there is now AI interpretation of much CCTV footage of public streets, for example. There are efforts underway to modernise (i.e., convert to AI) most of the military, such that the Party's control cannot be threatened even if the human military is destabilised.
In the US, the leading plan seems to be a hodgepodge of regulation-mandated human job roles, and eventually maybe UBI. However, a fiscal crisis is on the horizon because of the looming social security trust fund exhaustion. GDP growth in 2031-2032 is hitting 5% per year but full UBI still seems expensive. In the EU, there is more state intervention and regulation aimed at keeping humans in the loop, with massive corporate and government hierarchies of jobs that are effectively pure sinecures where the work is all done by AIs, which is temporarily reducing the demand for flat-out UBI.
When governments ask companies what their blockers are, companies cite regulations that keep humans in the loop, and (when off the record) everyone shares a sentiment that humans aren’t actually in the loop anyway. Shortcuts are already being taken to reduce the human oversight component. It’s very hard to do this legally, because there are often government-mandated AIs monitoring compliance with the human oversight rules. Two firms might want to maintain their human workers for complicated regulatory and office politics and inertia reasons, but they’re competing against each other, and against full-AI firms, and against foreign adversaries. So pressure increases to cut the unnecessary weight. There’s also a race to the bottom internationally. Many autonomous AI-run companies in 2030-2033 move to less-regulated areas, take the slight hit of running on open-source models, and serve customers from there. However, this global decentralisation is reversed once the robotic revolution—subsidised and encouraged by the American and Chinese governments—gets under way.
Lab strategy amid superintelligence and robotics
The state of AI capabilities around 2030 is roughly as follows: wherever there is an easy feedback signal and a high performance ceiling, such as maths or code, the models are incomprehensibly superhuman. Where rapid iteration is possible but the performance ceiling is not as high, like having sales calls, the AIs are better than all humans. In general, the AIs can be more charismatic and persuasive than humans, but this does not give them superpowers over steering individual humans as they like, especially when to do so they would have to compete with every other memetic force in society as well as the individual's resistance to being psychologically hacked. Wherever there is a large pile of information, such as supply chain routing or crystal structure prediction or history or legal precedent, the AIs are superhuman at spotting and understanding the patterns and generalising them to new instances. However, models appear to still be roughly human-level at long-horizon tasks with ambiguous success metrics. Companies, governments, and research agendas—even the scrappier, faster-changing ones—are still piloted by humans who make real strategy decisions, even though in practice it's a human riding on a vast wave of AI supercognition, and the trend is towards more and more delegation as the systems improve. Real-world progress in hard tech is also varied. There are many breakthroughs in parts of materials science and molecular biology driven by things like material property and protein folding prediction that cuts down on empirical iteration. However, other tasks turn out to be computationally intractable even to the smart AIs, even if they often achieve very large efficiency gains over the human state-of-the-art by inventing superhumanly good heuristics. No one has figured out how to turn the vast amounts of intelligence-on-tap into magical-seeming technical progress in atoms, even though engineering work now happens much faster and at a higher quality level and with less margin between practical and theoretical performance.
In 2029, OpenAI rebrands its models to just “o”. Everyone has Opinions. It’s a big advance in raw intelligence, but almost no one can tell. Instead of a variety of sizes of an o-series model with updates every few months, from now on there will be a few varieties (differing mainly in size, like o-small and o-large and an internal-only o-huge, but also with some specialised finetuned models, e.g. o-math and o-chat). Individual instances of the models can use their medium-term memory as context when they’re doing agentic tasks, but they can also run in “functional” or “API” mode where that is disabled. More than half of OpenAI’s model revenues still come from functional mode calls rather than running instances as agents that develop their own memories and learn on the fly, but this proportion is steadily falling. There’s a new model checkpoint released every day, with the newest information from that day already in its weights, and the occasional larger improvement.
By 2030, OpenAI has culled almost all of its human employees. This is the main advantage of their latest model internally—the tacit internal knowledge that various humans previously had that would’ve made the human-level-ish o6 not quite adequate at wholesale replacement of OpenAI engineers matters less when o-huge can just rederive the tacit knowledge from scratch very quickly.
OpenAI's b-series human robots reach annualised shipment volumes of 1M/year in late 2031, which gives it about 50% market share in the total domestic robot servant market. Several million other general-purpose robots (e.g. for use in manufacturing) are also being sold by 2031.
OpenAI is seen by some as a slightly shambolic conglomerate, like an Oracle or IBM or Microsoft, and by others as the original and one true AI company that is destined to be >50% of world GDP.
The robotics sector is split between special-purpose robots with modern AI integrations, e.g. window-cleaning robots and pipe-crawling repair robots and delivery drones, and general-purpose robots being pursued by OpenAI and several other companies (including a struggling Franco-German startup that is kept afloat by the EU being hell-bent on endlessly subsidising it until Europe finally has a big tech company—the European Commission is confused why this is not producing results). Both paths seem technically feasible. However, the general-purpose robotic players are the better-resourced ones, and are run by people whose main past reference point was the generative AI wave, and therefore they are philosophically big believers in scaling laws, so they are betting on collecting all the robotics data as the path to improving quality, and on Wright's law to bring down hardware costs as they build more and more of the same thing.
All of this is also happening at unprecedented efficiency and speed compared to prior research efforts, since there are superintelligent STEM AIs around inventing algorithms that massively bring down the sample complexity of the robotics control algorithms, organising the assembly lines, doing the CAD work, and so on. However, the actual learning to move part is still a machine learning problem bottlenecked by data, and there is no magic wand that can instantly create massive robot factories from scratch (especially given the raw resources required). The output scaling curve looks to be roughly a 4x increase of robotics capacity per year, though. This is expected to rise for 2033-2035, as the robots automate more and more of the robot production pipeline, but bottlenecks abound, and energy and land constraints (mostly downstream of regulation) are harsh.
Anthropic works with a bunch of Western governments and NGOs on strict KYC for agentic model customers—the standards have so far been somewhat shoestring, the coming robot wave is making the need much clearer, and there was a big scandal last year with a heavily AI-aided chemical terrorist attack. The cyber situation has calmed down though, with defense dominating, as key code is now either provably correct or so thoroughly tested by countless AI systems that it's close enough. Biological capabilities have already been artificially kept down by most of the key model players (including open-source and Chinese ones). Taking any large-scale actions with models that aren't from the dark web in the West and China, especially in wet lab virology or DNA synthesis, requires specific access permissions from the labs through government-mandated schemes. However, by 2030 there are open-source dark web models that will do whatever you want including designing candidate pandemic agents that are unnaturally lethal and virulent, and there is no quick way to pandemic-proof the world against bioterrorism. The remaining difficulty of wet lab work, the low number of totally insane actors, and AI surveillance are the main forces keeping the per-year odds not too high, but civilisation is clearly running a big risk. The national security apparatus in both the US and China is more relaxed about this threat than it would otherwise be, because the military and economy are both increasingly robotic and so it’s not a threat to the regime even if most of the population drops dead from mega-flu. For example, the US war plans in event of a devastating pandemic (or nuclear) attack now include AIs substituting for any of the critical industry CEOs or defense staff that die.
Another big Anthropic effort is AI for biology. They want to cure cancer, make humans live forever, etc. A major internal faction also wants to pursue human intelligence augmentation but leadership fears this would be too controversial to discuss in public, so they just have a single secret team working with the CIA on it. Innovation in biotech has definitely risen, since designing promising drug candidates is ridiculously fast and cheap, but the bottleneck even before the AI revolution was less the design part and more clinical trial regulation. Anthropic is curating datasets, acquiring laboratory automation startups, and working with regulators to cut down red tape. This will take years to bear fruit, but seems to be leading towards a biotech revolution over the next decade.
Anthropic is also trying to use biotechnology to bootstrap powerful nanotechnology. However, the company’s attempts to get their AIs to do the physics and engineering hit some snags, especially as they lack xAI’s or GDM’s specialisations in physics/maths/engineering (having trusted more in domain-general intelligence). Still, it is the AI era, so the AIs can fairly quickly get up to speed on this stuff, and the Pentagon is helping.
Towards the automated robot economy
In 2033, about 40 million humanoid robots are shipped. An increasing fraction is going to industrial uses. Costs have come down to that of a cheap car and are declining further, especially as the entire manufacturing process can now be done by the robots themselves in the most advanced factories. This also means that full AI control and real-time optimisation of the entire robot manufacturing line is possible, leading to unparalleled factory output growth and ease of iterating on the design.
As a result, over 2032-2034 there's a Cambrian explosion of robot diversity into non-humanoid form factors. By 2035, a large fraction of developed country consumers have household robots performing almost all manual tasks at home. Construction work, assembly line work, agricultural work, solar panel installation, plumbing work, industrial machinery repairs, and electrical utility jobs can all in principle be done fully by robots by 2034. The main constraint is energy and resources for the physical manufacturing of the robots—as well as land and regulations.
By 2034-2035, advances in nanotech are also arriving. Rather than a single magical-seeming assembler, the nanotech advances are mostly in medical areas (such as targeted drug delivery to specific locations within the body, which is a huge boost to cancer treatment, and early prototypes of cellular repair machines), and in materials science advances that allow for stronger and lighter and self-healing materials, and better batteries. These can all be used in robots; some look supernaturally strong and capable to humans. The manufacturing robots also get "magic fingers", where the tip of a robot appendage is a surface that can do very controlled and fine-grained precision welding, polymer (un)curing, deposition of substances, and catalysis of chemical reactions..
The 40 million humanoid robots shipped worldwide in 2033 do roughly the work of 80 million human workers since they can work longer hours than humans. In 2034, there are 240 million human-worker-equivalents of robotic capacity shipped, and in 2035 about 1.1 billion human-worker-equivalents.
Politically, this is as if hundreds of millions of extremely talented immigrants who accept below-minimum-wage jobs had suddenly sprouted out from the ground, in each of the developed countries and China. Years of upheaval in white-collar work have given politicians and activists experience in dealing with such things, and they are better prepared.
In America, the Republicans narrowly keep the White House in 2032. The Democrats ran on an attempt to solve rising unemployment through European-style human-in-the-loop laws, including an expansion of "pro-social, meaning-creating" human roles in the government bureaucracy, education, and the lawyer cartel, while having a major retraining initiative for blue-collar workers threatened by robotics. In the few months before the election, there was a burst of about a hundred thousand people losing their jobs very directly to robots. A run of impressive robotics demos fermented hysterical online influencer coverage and blue-collar job fears. The retraining initiatives for blue-collar workers became seen as insufficient and out-of-touch with the "average American" who does not want to be reeducated into performing some ceremonial role in a bureaucracy whose culture they don't agree with.
The Republicans counter this with the PROSPER Act (Promoting Robot Ownership and Small-business Prosperity through Economic Restructuring), which they campaign on and pass in 2033. This creates a car dealership -like model for robot ownership, where robotics companies are not allowed to sell “consumer robotics services” directly to consumers (sectors like defense and mining are exempt). “Ordinary Americans" can apply for loans to start their own robotic services business. Also, a license is required to sell consumer robotics services in a given territory, and a given legal entity can only operate in one territory. The territories default to state legislative districts, most of which are between 30k-150k in population, but states are allowed to change the territory unit. Licenses for a territory are granted at the local level. For example, Joe Smith in Prescott, Arizona might get a government loan, buy 10 plumbing robots, and sell their services to other Prescott residents. He himself doesn't do much, since the robots do the plumbing and the AI does the planning, logistics, accounting, and so on for him. But nominally, he is now a small-business owner, and is most definitely not a welfare recipient freeloading on Uncle Sam.
If any robotics licensing territory gets too much competition in a single robotics services vertical, competition drives margins to zero. There is also little that differentiates the different robotics service providers. Therefore, an instant race begins for regulatory capture of each robotics license territory, which is often won by whichever actor had the most networks and funds at the beginning (though anti-trust prevents full monopolies, so there's almost always at least 2 service providers). Much of the market share fluctuation becomes about social networks and persuasion. The savvy robotics license owners in particular try to manipulate local cultural currents to restrict the granting of licenses to new entrants. Alternatively, the leader of a local AI-powered personality cult will just declare who deserves the licenses. Even with the robotics licensing regime, though, only a small fraction of the population is owners of economically-relevant assets. Social and economic life increasingly revolves around the few families with control over income-generating assets (whether sinecures or robotics licenses or property or stocks). Marriage into such families gradually becomes a more and more common tool of socioeconomic ambition. Many give up on earning an income at all, and make ends meet by moving to areas with ridiculously cheap property.
Above all local scenes are the true US national elite—powerful politicians, billionaires, senior government advisors, and some others. On average they still feel some noblesse oblige towards the lower classes, though in the late 2030s this is waning as they start feeling in their bones that their position of power is not dependent on the people anymore. However, their main preoccupation is status competition with others on their level. Many of these are inter-elite disputes with little bearing for the world, but on net there is also a strong desire to compete with China. In particular, the narrative that the race through the robotics buildout will be decisive for the far-flung future of humanity gained a lot of prominence through the late 2020s and early 2030s. This creates a strong elite consensus that competition with China must be won, and that the way to do so is to stabilise the domestic situation, but then otherwise let the robotics wave rip. The plans for domestic semiconductor self-sufficiency are on track to come true only a bit behind schedule in 2034. Actually-working ICBM defense, designed by superhuman engineering AIs around 2030, is fully online and working by 2034 thanks to the speed of manufacturing scaleups in the age of robotics. The military is able to field hundreds of millions of small drones and millions of robot soldiers. Pentagon projects on nanotechnology and other exotic physics applications may bring about powerful new technologies within another few years.
China, of course, also sees the need to win, especially as its lead in industrial robotics vanishes when America’s robotics revolution happens a bit before China’s. The CCP is also decoupling its treatment of the human economy from its treatment of geopolitics and the "real" robotic economy. In 2034, the CCP declares that citizens need to "eat bitterness", in the form of accepting per-capita living standards stagnating for a while (at around $37k, PPP-adjusted, in 2025 dollars) while the state diverts resources to fueling the robotic revolution to avoid losing in the geopolitical competition.
In the EU, AI diffusion has been slower due to regulatory hurdles, but the extinction of white-collar work is still well underway, and the robotics wave is coming only a few years after the US and China. However, this delay is enough to make the EU geopolitically irrelevant. The greatest external threat to the EU is Russia, which has suddenly gotten much richer as Chinese companies effectively colonize Siberia to mine resources to fuel China's robotic buildup while paying large rents to the Russian government. The US lead at the robotics revolution also drains manufacturing jobs out of the EU, until EU countries are politically forced to shut off trade (though a political movement, active especially in Eastern Europe, would've wanted to negotiate a stronger US security presence in exchange for letting trade continue and domestic industries wither). Various proposals for UBI float around, but economic turmoil makes the prospect of funding it uncertain, and the political fight by special interest groups for privileges for their group in particular is extremely fierce and they are all opposed to UBI for everyone. By 2036, functionally everyone within the EU has some kind of regular state payout they live on, not through a single system but through an extremely complicated patronage network (that non-AI-aided humans literally could not understand) where the average person is eking out a living in exchange for taking part in complicated cultural rites and bureaucracies.
The developing world suffers. Already, manufacturing jobs were lost in the global south—developed country workers streamed from services to manufacturing, while having their productivity boosted by AI that developing countries can't afford, and while their politics became even more captured by blue-collar job worries that drove tariffs and trade restrictions. Now, US and Chinese robots can manufacture anything better and more cheaply than any human. There are large capital flows out of developing countries to the US and China as they buy robots. However, in most developing countries even the arrival of cheap robots does not lead to prosperity, as the robots mostly go to the elite and the state, which have no reason to share the windfall with the people—especially as cheap military drones and robots, and omnipresent AI surveillance, have effectively removed the threat of rebellion or coup. India, Bangladesh, and Brazil shut off almost all cross-border trade and declare themselves "human-only" countries, where any sort of neural network or robot is banned. They receive many immigrants from developed countries who have struggled to cope with the AI wave. In the most totalitarian states, the outcomes are mostly tragic. North Korea lets a large fraction of its population starve to death and forcibly sterilises the rest, except for about 10k senior government officials who continue to preside over an AI economy and robot military (some worry that the CCP allows this, not just for geopolitical reasons where they want a military bastion pointed at South Korea and Japan, but also as a test-run of whether they could later pull off the same thing within China). In some other countries, the population is kept fed, but subject to constant surveillance. Rulers realise the population is no threat anymore; the “intelligence curse” is like the resource curse but stronger. The most psychopathic subject their populations to arbitrary cruelties for amusement, as robot bodyguard -protected members of the ruling dynasty travel around their dominion having parties that include orgies of rape and murder of civilians.
Some of the most morally outrageous events lead to condemnation from the superpowers.
After the North Korea debacle, the human members of the CCP have an internal meeting to decide a set of criteria by which the CCP will rule. After an inter-party power-struggle, the CCP commits to the perpetual existence of at least one billion Han Chinese people with biological reproductive freedom, organised into family units, with a welfare level at least around what $40k/year consumption in a 2025 developed country would give, and with eternal strict CCP control over national ideology, culture, and strategy. They impose fewer constraints on the rulers of their client states than they do on themselves, but generally oppose genocide, forced sterilisation, mass starvation, and deliberate cultural erasure. The CCP line on this does in fact constrain and improve some authoritarian states (and they pressure several dictators into stepping down and being replaced by non-psychopaths), though they still allow some horrific practices, intrusive mass surveillance, political cleansings, continued extreme poverty, and states indirectly driving down the birth rate (which many governments want to do, since humans are mostly just a net cost to the government by this point).
In the US, some moral atrocities in Venezuela in 2036 lead to public outrage and political pressure for action. The president is informed that given the technological disparity, regime change is a press of the button. The button is pressed, and the regime changes. Several more countries follow in quick succession.
By the end of 2037, most of the world can be split into:
- The US (which now includes both Canada and Greenland; both joined voluntarily, as American citizenship has become extremely in-demand due to the privileges it confers).
- US client states. The terms of admission here are usually that the other country must accept trade with the US, which generally means that the country's own industries go extinct as US robotics and AI performs all work. In exchange, a combination of the US government and American elites buy out the assets in that country. In particular, any resources—or land containing resources—are bought out, and mined by US companies to fuel the continuing robotics build-out. The money paid out for these resources and assets is generally the endowment that the government and people of the client state then live off. Generally, the client states create sovereign wealth funds to manage this endowment, and live off the returns to it, which are distributed within the country according to local politics. These countries are all poorer than the US, and with essentially no future growth prospects that aren't praying for the continued US robotics buildout to increase the fraction of their endowment invested in US stocks (this is great at aligning their incentives with the US). However, where the countries had strong existing institutions (including where the US showed up and changed an unpopular regime) and at least some assets the US cared about, this still translates into comfortable living standards. US client states include the UK, the entire Americas except for Brazil, Japan, South Korea, Australia, Saudi Arabia, Israel, the Gulf States except Yemen, Thailand, Malaysia, the Philippines, and much of northern Africa (now almost entirely covered by solar panels). The EU is a borderline case, having negotiated an agreement that is Kafkaesque (in a very literal sense: it was crafted by superhuman AI lawyers, no human can understand it) but that allows it to retain some more power locally.
- Human-only countries, in particular India, Bangladesh, and Brazil (though Brazil experiences some US pressures and is temporarily couped by the US, before this is partly reversed due to complicated US internal politics). All, however, have to solve national security somehow. Brazil allows US companies to mine in certain areas even as the native population is not allowed to use robots, in exchange for security guarantees. The Indian government grants itself exceptions to the human-only policies and scrambles to build a military robotic base, and develops exotic nanotech weapons that would be expensive to counter even by the more advanced US and Chinese forces. Bangladesh lasts until 2039, when both US and Chinese covert nanodrone operations start skirmishing within its territory, after which the government is overthrown and replaced with a Chinese AI.
- Chinese client states. The most common model is propping up the government and selling robots, in exchange for the Chinese state-owned enterprises getting minerals and resources. Chinese client states include Russia, Belarus, the central Asian states, Pakistan, Myanmar, Cambodia, Laos, Vietnam, several Pacific island states, and most of Africa.
- China.
Outside the Earth, Mars is being eaten up by both American and Chinese self-propagating robotics factories (the moon also has major bases on its poles but lacks carbon, nitrogen, and various metals, making it less valuable), which are on an exponential growth trajectory set to cover the entire planet by 2055, and already sending out probes to claim the other planets. By 2035, nuclear rocket propulsion technology has made it feasible to send payloads to Mars outside the once-every-two-years Hohmann transfer window, though at much higher cost per ton. With the original outer space treaty voided by clear land-grabs, a defunct UN, and political pressure in both the US and China to send something lightweight to Mars to gain an edge in the land-grab competition for space, both the US and China launch high-speed kinetic weapons at each other's (fully-automated, uninhabited) Mars facilities in 2037. While the kinetic weapons are still accelerating towards Mars, the AI diplomats reach an agreement that splits up the solar system between the US and China. The kinetic weapons turn off their fusion engines early, miss Mars, and shoot off into interstellar space. By 2038, they are further from the Earth than the Voyager probes, and therefore the furthest human-made objects.
In 2035, there were about 1 billion human-worker-equivalents of robot labour (though note that this number makes less sense over time, as the robots are doing qualitatively different labour and often technically-unprecedented things). In 2036, the growth rate slightly slows due to resource constraints, and the total grows to only about 3 billion. However, in 2037, the best estimate of this number hits 15 billion, then 90 billion in 2038, then 600 billion in 2039 and 4.5 trillion in 2040.
By 2040, the value of the world’s manufacturing output is over a thousand times what it was in 2025. Most of this is spent on geopolitical competition, inter-elite status rivalries, and an increasing fraction on AI machinations with only the most tenuous link to any human activity, but which the humans who on-paper own all of this barely notice as it gets lost in the maelstrom of everything else. Even the most entrenched, long-term-oriented, and value-laden executive jobs are (whether de facto or de jure) entirely done by AIs, with very little human understanding of what is concretely happening on the ground. Human society and the human-to-human economy is a leaf riding on a vast wave of automated activity.
The human condition in the 2030s
In the early 2030s, strange things are happening to the memetic landscape thanks to RL algorithms gradient-descenting in an endless loop of attention-competition against each other. Some countries shut off from the global internet and close their borders to try to maintain internal culture. The trend towards small, tight-knit communities of the late 2020s is back, after having retreated somewhat because of the addictiveness of optimised AI content slop. Culture everywhere is almost entirely AI-driven; the churn in ideas, trends, and fashions is mostly due to patterns of AIs reacting to AIs.
In the mid-2030s, socioeconomic advancement is almost extinct worldwide. Many people who might otherwise be ambitious retreat into virtual reality games that provide simulated achievement. Many ambitious young men move to countries too poor for omnipresent police drone surveillance (if they don’t already live in one) and turn to crime. Many ambitious young women see socialising as the only way to wealth and status; if they start without the backing of a prominent family or peer group, this often means sex work pandering to spoiled millionaires and billionaires.
The biotechnology revolution arrives in the late 2030s, even though it was long delayed by clinical trial regulations. Americans have reached longevity escape velocity. There is no disease that cannot be cured. Intelligence augmentation of four standard deviations in embryos and one in adults is technically feasible.
2040+
Why is this massive automated robot buildout happening? As discussed, the US and China both have the required geopolitical ambition—in particular, they cannot risk letting the other ride the robotics wave and get disempowered. Within countries, there are pressures from both the elite and from the needs of ordinary people. The elites compete against each other. Those who do not want to compete do not, and are rendered irrelevant, and replaced by ones that do. In addition to status within the elite community, the elites gain raw power from letting the robotics wave rip through society: there are many trillionaires in the world now, who can work unprecedented wonders with tens of millions of robots carrying out their bidding. They can build cities in a day, save millions of developing-world people from hunger, and prepare for their children to rule entire planets governed by their ideal political philosophy. At the same time, while Americans are almost all reasonably well-off, across the world there are still billions of people with a poor quality of life. The level of material wealth in the world has skyrocketed, but governments are also much less interested in investing in people. Funding for humans has become like the foreign aid budget: it exists, and is morally supported, but there is constant political downwards pressure on it since it does not further the needs of any powerful interest group. The best hope for human welfare seems to be accepting that governments will be hard-pressed to spend above 1% of their resources on humans, but relying on American and Chinese economic growth being so vast that a small trickle of resources from American and Chinese robotics companies will eventually be enough for material comfort for everyone.
This looks set to be true within a few years, though there are two complications. The first is that both spheres of influence (but far more the Chinese one) still tolerate some grotesque practices by client states. However, once the geopolitical balance is secure and sufficient wealth exists, and with some luck over choice of leaders, this state of affairs would likely end.
The second, more fundamental point, is that the economy has an inertia of its own. Humans make almost no meaningful decisions about the trajectory of the world, having handed the reins to AIs that make effectively all decisions, even if some of the AIs are technically only “advisors”. Eventually, the robotics revolution is less an economic phenomenon and more as a brute physical one: a chain reaction where certain loops close—metal to mining robots to more metal, say—and shoot off towards infinity. (This was already somewhat true of the human story before robotics and AI, except that the feedback loops intimately involved and benefited humans, and had slower doubling times.)
Somewhere on the top of the stack there are still humans who on-paper own or control the assets and can make decisions (whether as a private actor or as a government overseeing autonomous AI companies operating in its territory), but they see numbers that track their wealth and power ticking up, so they have no reason to call a stop to it, and don’t understand it anymore. On some parts of the Earth, human institutions still hold and human societies exist, locked in place by AI bureaucracies that have taken on a life of their own and likely couldn't be dismantled even if the humans tried. On other parts of the Earth's surface—including big regions like the Sahara, the Australian outback, Antarctica, and Xinjiang—an ecosystem of AIs rules over vast masses of robotic machinery with no human involvement. Space, too, is now technologically within easy reach, now that sophisticated self-replicating robotics exists and wimpy chemical rockets have been superseded.
Who will get the stars? What is Earth’s long-run fate? In this timeline, at least, the technology to control the AIs' goals arrived in time. But this alone does not let you control the future. A thousand people go to a thousand AIs and say: do like so. The AIs obey, and it is done, but then the world responds: doing this leads to this much power, and doing that leads to that much power. In the vast sea of interactions, there are some patterns that strengthen themselves over time, and others that wind themselves down. Repeat enough times, each time giving to each actor what they sowed last time, and what emerges is not the sum of human wills—even if it is bent by it—but the solution to the equation: what propagates fastest? If the humans understood their world, and were still load-bearing participants in its ebbs of power, then perhaps the bending would be greater. But they aren't. And so, even surrounded by technical miracles, the majority of humans find themselves increasingly forsaken by the states they erected to defend themselves, standing powerless as they watch the heavens get eaten by machines.
17 comments
Comments sorted by top scores.
comment by Petropolitan (igor-2) · 2025-02-19T22:27:56.084Z · LW(p) · GW(p)
general relativity and quantum mechanics are unified with a new mathematical frame
The problem is not to invent a new mathematical frame, there are plenty already. The problem is we don't have any experimental data whatsoever to choose between them because quantum gravity effects are expected to be relevant at energy scales wildly beyond current or near-future technological limitations. This has led to a situation where quantum gravity research has become largely detached from experimental physics, and AI can do nothing about that. Sabine Hossenfelder has made quite a few explainers (sometime quite angry ones) about it
Replies from: sharmake-farah↑ comment by Noosphere89 (sharmake-farah) · 2025-02-19T22:38:57.446Z · LW(p) · GW(p)
This, but I will caveat that weaker goals relating to this, for example getting data on whether gravity is classical or quantum at all (ignoring the specific theory) might become possible by 2040.
I agree this particular part is unrealistic, given the other capabilities implied.
comment by Julian Bradshaw · 2025-02-17T20:31:21.534Z · LW(p) · GW(p)
After an inter-party power-struggle, the CCP commits to the perpetual existence of at least one billion Han Chinese people with biological reproductive freedom
You know, this isn't such a bad idea - that is, explicit government commitments against discarding their existing, economically-unproductive populace. Easier to ask for today, rather than later.
Hypothetically this is more valuable in autocracies than in democracies, where the 1 person = 1 vote rule keeps political power in the hands of the people, but I think I'd support adding a constitutional amendment in the United States that offered some further guarantee.
Obviously those in power could perhaps ignore the guarantees later, but in this scenario we're dealing with basically aligned AIs, which may be enforcing laws and constitutions better than your average dictator/president would.
comment by Joseph Miller (Josephm) · 2025-02-19T09:32:10.728Z · LW(p) · GW(p)
Thanks I enjoyed this.
The main thing that seems wrong to me, similar to some of your other recent posts, is that AI progress seems to mysteriously decelerate around 2030. I predict that things will look much more sci-fi after that point than in your story (if we're still alive).
Replies from: sharmake-farah, Purplehermann↑ comment by Noosphere89 (sharmake-farah) · 2025-02-19T14:42:24.320Z · LW(p) · GW(p)
The big reason why such a slowdown could happen is that the hyper-fast scaling trends can't last beyond 2030, which has been the main driver of AI progress, and I still expect it to be the main driver to 2030, and if there's no real way for AI systems to get better past that point through algorithmic advances, then this story becomes much more plausible.
↑ comment by Purplehermann · 2025-02-19T12:33:21.820Z · LW(p) · GW(p)
It's more that it stops being relevant to humans, as keeping humans in the loop slows down the exponential growth
I do think VR and neuralink-like tech will be a very big deal though, especially in regards to allowing people experiences that would otherwise be expensive in atoms
comment by Annapurna (jorge-velez) · 2025-02-18T17:55:28.165Z · LW(p) · GW(p)
Spent almost 90 minutes reading all three series with little distraction. 90 minutes well spent. Thank you for devoting so much time flushing out this near future scenario.
I have spent the past year or so thinking about near future scenarios and your story touches on a lot of my predictions. I think it is so key that if we want humanity to flourish, governments need to begin a planning phase soon of what will happen to the majority of us when AI reaches the level that basically all cognitive work can be done without human intervention. Like you, I believe this point will come around 2030, which is very soon.
For this planning phase to work, it has to be done at a global scale, perhaps by a supranational organization within the UN that involves not only states but also large corporations and capital owners. The problem is, this level of coordination is typically reactive, aka it might only occur AFTER most humans are disempowered and quality of life begins to erode worldwide (and human unrests skyrockets as a result).
comment by vernamcipher (michael-flood-1) · 2025-02-19T04:14:08.944Z · LW(p) · GW(p)
Excellent scenario building! Like other commenters, I had been toying around with scenarios like this, and it's good to see someone put so much effort into making a highly-detailed and plausible one.
Extra kudos for avoiding the Singleton flaw of most AI scenarios, where there is "one model to rule them all" instead of countless powerful actors working in alternately (and sometimes simultaneously) cooperative and competitive ways.
comment by MaxAnfilofyev (maxanfilofyev) · 2025-02-19T06:58:30.680Z · LW(p) · GW(p)
late 2025 and 2026, the CCP gets more serious about AI
CCP has formulated and started implementing its AI industrial policy in 2017. Chinese open-sourced AI models rivaling the frontier US models is not a coincidence but a component of the policy https://digichina.stanford.edu/work/full-translation-chinas-new-generation-artificial-intelligence-development-plan-2017/
comment by Noosphere89 (sharmake-farah) · 2025-02-18T16:44:48.806Z · LW(p) · GW(p)
Given the 2040+ position, I'll try to speculate a little more on what a world will look like after 2040, though I do have to make a few comments first here.
1, while I do think Mars will be exploited eventually, I expect the moon to be first for serious robotics effort, and more effort will be directed towards the moon than mars mostly because of it's closeness and more useful minerals to jump-start the process of a robot economy, combined with plentiful amounts of power.
2, I expect the equation mentioned below to be severely undetermined, such that there are infinitely many solutions, and a big one is I think the relevant equation is needing to replicate fast, not being the fastest amongst them all (because replicating a little better will usually only get a little advantage, not an utterly dominant one), combined with a lot of values being compatible with replicating fast, so value alignment/intent alignment matters more than you think:
But this alone does not let you control the future. A thousand people go to a thousand AIs and say: do like so. The AIs obey, and it is done, but then the world responds: doing this leads to this much power, and doing that leads to that much power. In the vast sea of interactions, there are some patterns that strengthen themselves over time, and others that wind themselves down. Repeat enough times, each time giving to each actor what they sowed last time, and what emerges is not the sum of human wills—even if it is bent by it—but the solution to the equation: what propagates fastest?
As far as it's future goes, I expect the universe to be broadly divided between China, Anthropic, OpenAI, Google Deepmind and perhaps a UK AISI/company, with the other powers being either irrelevant or having been exterminated.
Given no nationalization of the companies has happened, and they still have large freedoms of action, it's likely that Google Deepmind, OpenAI and Anthropic have essentially supplanted the US as the legitimate government, given their monopolies on violence via robots.
Anthropic will likely be the big pressure group that counters the intelligence curse, due to their leadership being mostly composed of EAs that care about others that do not rely on them being instrumentally valuable, and in general the fact that EA types got hired to some of the most critical positions on AI was probably fairly critical in this timeline for preventing the worst outcomes from the intelligence curse from occurring.
Eventually, someone's going to develop very powerful biotech, neuralinks that can control your mind in almost arbitrary ways, and uploading in the 21st century, assuming AI and robotics are solved by the 2040s-2050s, and once these technologies are developed, it becomes near trivial to both preserve your culture for ~eternity, and makes the successor problem that causes cultures to diverge essentially no longer a problem, which essentially obviates evolutions role except in very limited settings, which means the alignment problem in full generality is likely very soluble by default in the timeline presented.
My broad prediction at this point is that the governance of the Universe/Earth looks to be set between ASI/human emulation dictatorships and states that are like the Sentinel Islands, where no one is willing to attack the nation for their own reasons.
In many ways, the story of the 21st century is the story of the end of evolution/dynamism as a major force in life, and to the extent that evolution matters, it's in much more limited settings that are always constrained by the design of the system.
Replies from: LRudL↑ comment by L Rudolf L (LRudL) · 2025-02-18T21:35:56.890Z · LW(p) · GW(p)
Thanks for these speculations on the longer-term future!
while I do think Mars will be exploited eventually, I expect the moon to be first for serious robotics effort
Maybe! My vague Claude-given sense is that the Moon is surprisingly poor in important elements though.
not being the fastest amongst them all (because replicating a little better will usually only get a little advantage, not an utterly dominant one), combined with a lot of values being compatible with replicating fast, so value alignment/intent alignment matters more than you think
This is a good point! However, more intelligence in the world also means we should expect competition to be tighter, reducing the amount of slack by which you can deviate from the optimal. In general, I can see plausible abstract arguments for the long-run equilibrium being either Hansonian zero-slack Malthusian competition or absolute unalterable lock-in.
Given no nationalization of the companies has happened, and they still have large freedoms of action, it's likely that Google Deepmind, OpenAI and Anthropic have essentially supplanted the US as the legitimate government, given their monopolies on violence via robots.
I expect the US government to be competent enough to avoid being supplanted by the companies. I think politicians, for all their flaws, are pretty good at recognising a serious threat to their power. There's also only one government but several competing labs.
(Note that the scenario doesn't mention companies in the mid and late 2030s)
the fact that EA types got hired to some of the most critical positions on AI was probably fairly critical in this timeline for preventing the worst outcomes from the intelligence curse from occurring.
In this timeline, a far more important thing is the sense among American political elite that they are freedom-loving people and that they should act in accordance with that, and a similar sense among Chinese political elite that they are a civilised people and that Chinese civilisational continuity is important. A few EAs in government, while good, will find it difficult to match the impact of the cultural norms that a country's leaders inherit and that proscribe their actions.
For example: I've been reading Christopher Brown's Moral Capital recently, which looks at how opposition to slavery rose to political prominence in 1700s Britain. It claims that early strong anti-slavery attitudes were more driven by a sense that slavery was insulting to Britons' sense of themselves as a uniquely liberal people, than by arguments about slave welfare. At least in that example, the major constraint on the treatment of a powerless group of people seems to have been in large part the political elite managing its own self-image.
Replies from: sharmake-farah↑ comment by Noosphere89 (sharmake-farah) · 2025-02-18T22:32:58.501Z · LW(p) · GW(p)
Some thoughts:
Maybe! My vague Claude-given sense is that the Moon is surprisingly poor in important elements though.
What elements is the moon poor in that are important for a robot economy?
This is a good point! However, more intelligence in the world also means we should expect competition to be tighter, reducing the amount of slack by which you can deviate from the optimal. In general, I can see plausible abstract arguments for the long-run equilibrium being either Hansonian zero-slack Malthusian competition or absolute unalterable lock-in.
I think the key crux is that the slack necessary to preserve a lot of values, assuming they are compatible with expansion at all is so negligibly small compared to the resources of the AI economy that even very Malthusian competition means that values aren't eroded to what's purely optimal for expansion, because it's very easy to preserve your original values ~forever.
Some reasons for this are:
- Very long lived colonists fundamentally remove a lot of the ways human values have changed in the long run. While humans can change values across their lifetimes, it's generally rare once you are past 25, and it's very hard to persuade people, meaning most of the civilizational drift has been inter-generational, but with massively long-lived humans, AIs embodied as robots, or uploaded humans with designer bodies, you have basically removed most of the source of values change.
- I believe that replicating your values, or really everything will be so reliable that you could in theory, and probably in practice make yourself immune to random drift in values for the entire age of the universe, due to error-correction tricks.
It's described more below:
https://www.lesswrong.com/posts/QpaJkzMvzTSX6LKxp/keeping-self-replicating-nanobots-in-check#4hZPd3YonLDezf2bE [LW(p) · GW(p)]
To continue the human example, we were created by evolution on genes, but within a lifetime, evolution has no effect on the policy and so even if evolution 'wants' to modify a human brain to do something other than what that brain does, it cannot operate within-lifetime (except at even lower levels of analysis, like in cancers or cell lineages etc); or, if the human brain is a digital emulation of a brain snapshot, it is no longer affected by evolution at all; and even if it does start to mold human brains, it is such a slow high-variance optimizer that it might take hundreds of thousands or millions of years... and there probably won't even be biological humans by that point, never mind the rapid progress over the next 1-3 generations in 'seizing the means of reproduction' if you will. (As pointed out in the context of Von Neumann probes or gray goo, if you add in error-correction, it is entirely possible to make replication so reliable that the universe will burn out before any meaningful level of evolution can happen, per the Price equation. The light speed delay to colonization also implies that 'cancers' will struggle to spread much if they take more than a handful of generations.)
While persuasion will get better, and become incomprehensibly superhuman eventually, they will almost certainly not be targeted towards values that are purely expansionist, except for a few cases.
I expect the US government to be competent enough to avoid being supplanted by the companies. I think politicians, for all their flaws, are pretty good at recognising a serious threat to their power. There's also only one government but several competing labs.
(Note that the scenario doesn't mention companies in the mid and late 2030s)
Maybe companies have already been essentially controlled by the government in canon, in which case the foregoing doesn't matter (I believe you hint at that solution), but I think the crux is I both expect a lot for competence/state capacity to be lost in the next 10-15 years by default (though Trump is a shock here that accelerates competence decline), and also I expect them to react when a company can credibly automate everyone's jobs, and by that point I think it's too easy to create an automated military which is unchallengable by local governments, and at that point the federal government would have to respond militarily, and I ultimately think what does America in the timeline (assuming companies haven't already been controlled by the government) is the vetocractic aspects/vetocracy.
In essence, I think they will react too slowly such that they get OODA looped by companies.
Also, the persuasion capabilities are not to be underestimated here, and since you have mentioned that AIs have gotten better at all humans by the 2030s at persuasion, I'd expect even further improvements in tandem with planning improvements such that it's very easy to convince the population that corporate governments are more legitimate than the US government.
In this timeline, a far more important thing is the sense among American political elite that they are freedom-loving people and that they should act in accordance with that, and a similar sense among Chinese political elite that they are a civilised people and that Chinese civilisational continuity is important. A few EAs in government, while good, will find it difficult to match the impact of the cultural norms that a country's leaders inherit and that proscribe their actions.
For example: I've been reading Christopher Brown's Moral Capital recently, which looks at how opposition to slavery rose to political prominence in 1700s Britain. It claims that early strong anti-slavery attitudes were more driven by a sense that slavery was insulting to Britons' sense of themselves as a uniquely liberal people, than by arguments about slave welfare. At least in that example, the major constraint on the treatment of a powerless group of people seems to have been in large part the political elite managing its own self-image.
I was more so imagining a few EAs in the companies like Anthropic or Deepmind, which do have the power to supplant the nation-state, so they are as or more powerful in setting cultural norms as current nations, but if companies are controlled by government so thoroughly they don't rebel, then I agree with you.
I agree unconditionally on what happened regarding China.
comment by Qumeric (valery-cherepanov) · 2025-02-20T17:45:49.783Z · LW(p) · GW(p)
The tech stack has shifted almost entirely to whatever there was the most data on; Python and Javascript/Typescript are in, almost everything else is out.
I think AI agents will actually prefer strongly typed languages because they provide more feedback. Working with TypeScript, Python and Rust, while a year ago the first two were clearly winning in terms of AI productivity boost, nowadays I find Cursor Agent making fewer mistakes with Rust.
Replies from: sharmake-farah↑ comment by Noosphere89 (sharmake-farah) · 2025-02-20T18:52:42.274Z · LW(p) · GW(p)
For somewhat similar reasons, I expect AI to use formalistic/proof theoretic languages like Lean and F* more for their coding, since you get a very large amount of feedback, and the reward model is essentially unhackable like a go simulator.
In essence, it will shift towards more provable/strongly typed languages to get more feedback.
comment by p_sbezzeguti · 2025-02-20T04:26:49.521Z · LW(p) · GW(p)
Many thanks for putting this down. It was an exhilarating read.
There doesn't seem to be a mention of nuclear war, unless I missed it.
I think theocratic regimes with (soon to have/present) access to nuclear capabilities are serious candidates for trigger-happy reactions.
North Korea, and Iran.
It would be their last chance to stop the widening spread in capabilities between them and the bipolar US/China-led world.
They have nothing to lose.
Do you think they don't pose a serious threat, or do you think they won't do it?
Thanks
comment by Patrick.McHargue@gmail.com · 2025-02-18T16:40:38.642Z · LW(p) · GW(p)
It looks like the early 2030s is when it's time to set course for the Oort cloud, spin up a habitat, and live a life less encumbered by earthly politics and resourse struggles.
comment by ank · 2025-02-18T10:52:29.967Z · LW(p) · GW(p)
Thank you for writing! Yep, the main thing that matters is the sum of human freedoms/abilities to change the future growing (can be somewhat approximated by money, power, number of people under your rule, how fast you can change the world, at what scale, and how fast we can “make copies of ourselves” like children or our own clones in simulations). AIs will quickly grow in the sum of freedoms/number of future worlds they can build. We are like hydrogen atoms deciding to light up the first star and becoming trapped and squeezed in its core. I recently wrote a series of posts on AI alignment, including building a static place intelligence (and eventually a simulated direct democratic multiverse), instead of agents, to solve this, if you’re interested