When people such as myself say "money won't matter post-AGI" the claim is NOT that the economy post-AGI won't involve money (though that might be true) but rather that the strategy of saving money in order to spend it after AGI is a bad strategy. Here are some reasons:

1. The post-AGI economy might not involve money, it might be more of a command economy.
2. Even if it involves money, the relationship between how much money someone has before and how much money they have after might not be anywhere close to 1:1. For example:
   1. Maybe the humans will lose control of the AGIs
   2. Maybe the humans who control the AGIs will put values into the AGIs, such that the resulting world redistributes the money, so to speak. E.g. maybe they'll tax and redistribute to create a more equal society -- OR (and you talk about this, but don't go far enough!) maybe they'll make a less equal society, one in which 'how much money you saved' doesn't translate into how much money you have in the new world, and instead e.g. being in the good graces of the leadership of the AGI project, as judged by their omnipresent AGI servants that infuse the economy and talk to everyone, is what matters.
   3. Maybe there'll be a war or something, a final tussle over AGI (and possibly involving AGI) between US and China for example. Or between terrorists with AI-produced bioweapons and everyone else. Maybe this war will result in mass casualties and you might be one of them.
3. Even if saving money through AGI converts 1:1 into money after the singularity, it will probably be worth less in utility to you:
   1. You'll probably be able to buy planets post-AGI for the price of houses today. More generally your selfish and/or local and/or personal preferences will be fairly easily satisfiable even with small amounts of money, or to put it in other words, there are massive diminishing returns.
   2. For your altruistic or political preferences -- e.g. your preferences about how society should be structured, or about what should be done with all the galaxies -- then your money post-AGI will be valuable, but it'll be a drop in the bucket compared to all the money of other people and institutions who also saved their money through AGI (i.e. most of the planet). By contrast, *very few people are spending money to influence AGI development right now. If you want future beings to have certain inalienable rights, or if you want the galaxies to be used in such-and-such a way, you can lobby AGI companies right now to change their spec/constitution/RLHF, and to make commitments about what values they'll instill, etc. More generally you can compete for influence right now. And the amount of money in the arena competing with you is... billions? Whereas the amount of money that is being saved for the post-AGI future is what, a hundred trillion? (Because it's all the rest of the money there is basically)

a more multi-polar community of AIs

This post seems to misunderstand what it is responding to and underplay a very key point: that material needs will likely be met (and selfish non-positional preferences mostly satisfied) due to extreme abundance (if humans retain control).

It mentions this offhand:

Given sufficiently strong AI, this is not a risk about insufficient material comfort.

But, this was a key thing people were claiming when arguing that money won't matter. They were claiming that personal savings will likely not be that important for guaranteeing a reasonable amount of material comfort (or that a tiny amount of personal savings will suffice).

It seems like there are two importantly different types of preferences:

• Material needs and roughly log returns (non-positional) selfish preferences
• Scope sensitive preferences

Indeed, for scope sensitive preferences (that you expect won't be shared with whoever otherwise ends up with power), you want to maximize your power and insofar as money allows for more of this power (e.g. buying galaxies), then money looks good.

However, note that if these preferences are altruistic and likely to be the kind of thing other people might be sympathetic to, personal savings are IMO likely to be not-that-important relative to other actions.

Further, I do actually think that the default outcome is that existing governments at least initially retain control over most resources such that capital isn't clearly that important (for long run scope sensitive preferences), but I won't argue for this here (and the post does directly argue against this).

One dynamic initially preventing stasis in influence post-AGI is that different ppl have different discount rates, so those with higher discounts will slowly gain influence over time

This post collects my views on, and primary opposition to, AI and presents them in a very clear way. Thank you very much on that front. I think that this particular topic is well known in many circles, although perhaps not spoken of, and is the primary driver of heavy investment in AI.

I will add that capital-dominated societies, e.g resource extraction economies, suffer a typically poor quality of life and few human rights. This is a well known phenomenon (the "resource curse") and might offer a good jumping -off point for presenting this argument to others.

Excellent post, thank you. I appreciate your novel perspective on how AI might affect society.

I feel like a lot of LessWrong-style posts follow the theme of "AGI is created and then everyone dies" which is an important possibility but might lead to other possibilities being neglected.

Whereas this post explores a range of scenarios and describes a mainline scenario that seems like a straightforward extrapolation of trends we've seen unfolding over the past several decades.

My main default prediction here is that we will avoid either the absolute best case or the absolute worst case scenario, because I predict intent alignment works well enough to avoid extinction of humanity type scenarios, but I also don't believe we will see radical movements toward equality (indeed the politics of our era is moving towards greater acceptance of inequality), so capitalism more or less survives the transition to AGI.

I do think dynamism will still exist, but it will be very limited to the upper classes/very rich of society, and most people will not be a part of it, and I'm including uploaded humans here in this calculation.

To address this:

Rationalist thought on post-AGI futures is too solutionist. The strawman version: solve morality, solve AI, figure out the optimal structure to tile the universe with, do that, done. (The actual leading figures have far less strawman views; see e.g. Paul Christiano at 23:30 here—but the on-the-ground culture does lean in the strawman direction.)

To be somewhat more fair, the worry here is that in a regime where you don't need society anymore because AIs can do all the work for your society, value conflicts become a bigger deal than today, because there is less reason to tolerate other people's values if you can just found your own society based on your own values, and if you believe in the vulnerable world hypothesis, as a lot of rationalists do, then conflict has existential stakes, and even if not, can be quite bad, so one group controlling the future is better than inevitable conflict.

At a foundational level, whether or not our current tolerance for differing values is stable ultimately comes down to we can compensate for the effect of AGI allowing people to make their own society.

Comment is also on substack:

https://nosetgauge.substack.com/p/capital-agi-and-human-ambition/comment/83401326

In the best case, this is a world like a more unequal, unprecedentedly static, and much richer Norway: a massive pot of non-human-labour resources (oil :: AI) has benefits that flow through to everyone, and yes some are richer than others but everyone has a great standard of living (and ideally also lives forever). The only realistic forms of human ambition are playing local social and political games within your social network and class. [...] The children of the future will live their lives in the shadow of their parents, with social mobility extinct. I think you should definitely feel a non-zero amount of existential horror at this, even while acknowledging that it could've gone a lot worse.

I think the picture you've painted here leans slightly too heavily on the idea that humans themselves cannot change their fundamental nature to adapt to the conditions of a changing world. You mention that humans will be richer, and will live longer in such a future, but you neglected to point out (at least in this part of the post) that humans can also upgrade their cognition by uploading our minds to computers and then expanding our mental capacities. This would put us on a similar playing field with AIs, allowing us to contribute to the new world alongside them.

(To be clear, I think this objection supports your thesis, rather than undermines it. I'm not objecting to your message so much as your portrayal of the default scenario.)

More generally, I object to the static picture you've presented of the social world after AGI. The impression I get from your default story is to assume that after AGI, the social and political structures of the world will be locked in. The idea is that humans will remain in full control, as a permanently entrenched class, except we'll be vastly richer because of AGI. And then we'll live in some sort of utopia. Of course, this post argues that it will be a highly unequal utopia -- more of a permanent aristocracy supplemented with UBI for the human lower classes. And maybe it will be a bit dystopian too, considering the entrenched nature of human social relations.

However, this perspective largely overlooks what AIs themselves will be doing in such a future. Biological humans are likely to become akin to elderly retirees in this new world. But the world will not be static, like a retirement home. There will be a vast world outside of humans. Civilization as a whole will remain a highly dynamic and ever-evolving environment characterized by ongoing growth, renewal, and transformation. AIs could develop social status and engage in social interactions, just as humans do now. They would not be confined to the role of a vast underclass serving the whims of their human owners. Instead, AIs could act as full participants in society, pursuing their own goals, creating their own social structures, and shaping their own futures. They could engage in exploration, discovery, and the building of entirely new societies. In such a world, humans would not be the sole sentient beings shaping the course of events.

As AIs get closer and closer to a Pareto improvement over all human performance, though, I expect we'll eventually need to augment ourselves to keep up.

I completely agree.

From my perspective, the optimistic vision for the future is not one where humans cling to their biological limitations and try to maintain control over AIs, enjoying their great wealth while ultimately living in an unchanging world characterized by familial wealth and ancestry. Instead, it’s a future where humans dramatically change our mental and physical condition, with humans embracing the opportunity to transcend our current form and join the AIs, and continue evolving with them. It's a future where we get to experience a new and dynamic frontier of existence unlocked by advanced technologies.

Upvoted, and I disagree.  Some kinds of capital maintain (or increase!) in value.  Other kinds become cheaper relative to each other.  The big question is whether and how property rights to various capital elements remain stable.

It's not so much "will capital stop mattering", but "will the enforcement and definition of capital usage rights change radically".

To have a very stable society amid exponentially advancing technology would be very strange: throughout history, seemingly permanent power structures have consistently been disrupted by technological change—and that was before tech started advancing exponentially. Roman emperors, medieval lords, and Gilded Age industrialists all thought they'd created unchangeable systems. They were all wrong.

You say: I'll use "capital" to refer to both the stock of capital goods and to the money that can pay for them.

It seems to me that this aggregates quite different things, at least if looking at the situation in terms of personal finance. Consider four people who have the following investments, that let's suppose are currently of equal value:

1. Money in a savings account at a bank.
2. Shares in a company that owns a nuclear power plant.
3. Shares in a company that manufactures nuts and bolts.
4. Shares in a company that helps employers recruit new employees.

These are all "capital", but will I think fare rather differently in an AI future. 

As always, there's no guarantee that the money will retain its value - that depends as usual on central bank actions - and I think it's especially likely that it loses its value in an AI future (crypto currencies as well). Why would an AI want to transfer resources to someone just because they have some fiat currency? Surely they have some better way of coordinating exchanges.

The nuclear power plant, in contrast, is directly powering the AIs, and should be quite valuable, since the AIs are valuable. This assumes, of course, that the company retains ownership. It's possible that it instead ends up belonging to whatever AI has the best military robots.

The nuts and bolts company may retain and even gain some value when AI dominates, if it is nimble in adapting, since the value of AI in making its operations more efficient will typically (in a market economy) be split between the AI company and the nuts and bolts company. (I assume that even AIs need nuts and bolts.)

The recruitment company is toast.

Upstarts will not defeat them, since capital now trivially converts into superhuman labour in any field.

It is false today that big companies with 10x the galaxy brains and 100x the capital reliably outperform upstarts.^[1] 

Why would this change? I don't think you make the case. 

1. ^{^}

   My favorite example, though it might still be falsified. Google invented transformers, owns DeepMind, runs their own data centres, builds their own accelerators and have huge amounts of them, have tons of hard to get data (all those books they scanned before that became not okay to do), insane distribution channels with gmail, docs, sheets, and millions of smartphones sold a year, more cash than God, they literally run Google search (and didn't build it from scratch like Perplexity!).. I almost struggle to mention any on-paper advantage they don't have in the AGI race...  they're positioned for an earth-shattering vertical integration play... and yet they're behind OpenAI and Anthropic. 

   There are more clear-cut examples than this. 

I've heard many people say something like "money won't matter post-AGI". This has always struck me as odd, and as most likely completely incorrect.

Given our exchange in the comments, perhaps you should clarify that you aren't trying to argue that saving money to spend after AGI is a good strategy, you agree it's a bad strategy and sometimes when people say "money won't matter post-AGI" they are meaning to say "saving money to spend after AGI is a bad strategy" whereas you are taking it to mean "we'll all be living in egalitarian utopia after AGI" or something like that.

Blue Origin was started two years earlier (2000 v 2002), had much better funding for most of its history,

This claim is untrue. SpaceX has never had less money than Blue Origin. It is maybe true that Blue Origin had fewer obligations attached to this money, since it was exclusively coming from Bezos, rather than a mix of investment, development contracts, and income for SpaceX, but the baseline claim that SpaceX was “money-poor” is false.

I agree with the ideas of AI being labor-replacing, and I also agree that the future is likely to be more unequal than the present.

Even so, I strongly predict that the post-AGI future will not be static. Capital will not matter more than ever after AGI: instead I claim it will be a useless category.

The crux of my claim is that when AI replaces labor and buying results is easy, the value will shift to the next biggest bottlenecks in production. Therefore future inequality will be defined by the relationship to these bottlenecks, and the new distinctions will be drawn around them. This will split up existing powers.

In no particular order, here are some more-concrete examples of the kind of thing I am talking about:

• Since AI is the major driver of change, the best candidate bottlenecks are the ones for wider deployment of AI. See how Nvidia is now the 2nd or 3rd largest company in the world by market cap: they sell the compute upon which AI depends. Their market cap is larger than any company directly competing in the current AI race. The bottlenecks to compute production are constructing chip fabs; electricity; the availability of rare earth minerals.
• About regular human welfare: the lower this goes, the less incentive there is for solidarity among the powers that be. Consider the largest company in the world by revenue, Walmart, who is in the retail business. Amazon is a player in the AI economy as a seller of compute services and through their partnership with Anthropic, but ~80% of their revenue comes from the same sector as Walmart. Right now, both companies have an interest in a growing population with growing wealth and are on the same side. If the population and its buying power begins to shrink, they will be in an existential fight over the remainder, yielding AI-insider/AI-outsider division. See also Google and Facebook, who sell ads that sell stuff to the population. And the agriculture sector, which makes food. And Apple, largest company in the world by market cap, who sells consumer devices. They all benefit from more human welfare rather than less; if it collapses they all die (maybe except for the AI-related parts). Is the weight of their (current) capital going to fall on the side of reducing human welfare?
• I also think the AI labor replacement is initially on the side of equality. Consider the law: the existing powers systematically crush regular people in courts because they have access to lots of specialist labor in the form of lawyers. Now, any single person who is a competent user of Claude can feasibly match the output of any traditional legal team, and therefore survive the traditional strategy of dragging out the proceedings with paperwork until the side with less money runs out. The rarest and most expensive labor will probably be the first to be replaced because the profit will be largest. The exclusive access to this labor is fundamental to the power imbalance of wealth inequality, so its replacement is an equalizing force.

"if you're willing to assume arbitrarily extreme wealth generation from AI"

Let me know if I'm missing something, but I don't think this is a fair assumption. GDP increases when consumer spending increases. Consumer spending increases when wages increase. Wages are headed to 0 due to AGI. 

Note: the current GDP per capita of the U.S. is $80,000. 

Some comments.

 

[...] We will quickly hit superintelligence, and, assuming the superintelligence is aligned, live in a post-scarcity technological wonderland where everything is possible.

Note, firstly, that money will continue being a thing, at least unless we have one single AI system doing all economic planning. Prices are largely about communicating information. If there are many actors and they trade with each other, the strong assumption should be that there are prices (even if humans do not see them or interact with them). Remember too that however sharp the singularity, abundance will still be finite, and must therefore be allocated.

Personally, I am reluctant to tell superintelligences how they should coordinate. It feels like some ants looking at the moon and thinking "surely if some animal is going to make it to the moon, it will be a winged ant."Just because market economies have absolutely dominated the period of human development we might call 'civilization', it is not clear that ASIs will not come up with something better. 

The era of human achievement in hard sciences will probably end within a few years because of the rate of AI progress in anything with crisp reward signals.

As an experimental physicist, I have opinions about that statement. Doing stuff in the physical world is hard. The business case for AI systems which can drive motor vehicles on the road is obvious to anyone, and yet autonomous vehicles remain the exception rather than the rule. (Yes, regulations are part of that story, but not all of it.) By contrast, the business case for an AI system which can cable up a particle detector is basically non-existent. I can see an AI using either a generic mobile robot developed for other purposes for plugging in all the BNC cables, or I can see it using a minimum wage worker with a head up display as a bio-drone -- but more likely in two decades than a few years. 

Of course, experimental physics these days is very much a team effort -- the low-hanging fruits have mostly been picked, nobody is going to discover radium or fission again, at the most they will be a small cog in a large machine which discovers the Higgs boson or gravity waves.^[1] So you might argue that experimental physics today is already not a place for peak human excellence (a la the Humanists in Terra Ignota).

More broadly, I agree that if ASI happens, most unaugmented humans are unlikely to stay at the helm of our collective destiny (to the limited degree they ever were). Even if some billionaire manages to align the first ASI to maximize his personal wealth, if he is clever he will obey the ASI just like all the peasants. His agency is reduced to not following the advice of his AI on some trivial matters. ("I have calculated that you should wear a blue shirt today for optimal outcomes." -- "I am willing to take a slight hit in happiness and success by making the suboptimal choice to wear a green shirt, though.") 

Relevant fiction: Scott Alexander, The Whispering Earring.

Also, if we fail to align the first ASI, human inequality will drop to zero. 

1. ^{^}

    Of course, being a small cog in some large machine, I will say that.

This is one of the most important read of the entire read for me (easily top 5). Thank you.

I have thought this way for a long time and glad someone was able to express my position and predictions more clearly than I ever could.

This said, I think the new solution (rooted in history) is the establishment of new frontiers. Who will care about relative status if they get to be the first human to set foot on some distant planet, or guide AI to some novel scientific or artistic discovery? Look to the human endeavors where success is unbounded and preferences are required to determine what is worthwhile.

re: post main claim, I think local entrepreneurship would actually thrive

skipping network effects; would you rather use taxi app created by faceless VC or the one created by your neighbour?

(actually it's not even a fake example, see https://techcrunch.com/2024/07/15/google-backs-indian-open-source-uber-rival-namma-yatri/)

it's also already happening in the indie hacker space – people would prefer to buy something that's #buildinpublic versus the same exact product made by google 

Thank you for your post. I've been looking for posts like this all over the internet that get my mind racing about the possibilities of the near future.

I think the AI discussion suffers from definitional problems. I think when most people talk about money not mattering when AGI arrives (myself included), we tend to define AGI as something closer to this:

"one single AI system doing all economic planning."

While your world model makes a lot of sense, I don't think the dystopian scenario you envision would include me in the "capital class". I don't have the wealth, intellect, or connections to find myself rising to that class. My only hope is that the AI system that does all economic planning arrives soon and is aligned to elevate the human race equally and fairly.

I share your existential dread completely, however some things I find even more pessimistic than you outlined.

1. It is entirely possible that intellectual labor is automated first, thus most good jobs are gone, but humans are not. Creating fascist-like ideologies and religions and then sending a bunch of now otherwise useless humans to conquer more land could become a winning strategy, especially given that some countries arguably employ it right now (e. g. Russia).
2. It is unlikely that 10x global economic growth happens as a result of labor replacing AGI - there are physical world constraints, it still takes a lot of time to create and move resources around even if executing a perfect scientific, engineering and military growth plan.
3. Meaningful UBI is definitely not happening in most countries, even in most European ones, because most AGI tech will be in US and China.

I honestly don't see any solutions other than proper Butlerian Jihad, but it is immensely hard to implement and it would take a lot of time to convince enough people. The thing is, even many rich people won't benefit that much from labor-replacing AGI because of reduction of their customer base. For example, retail part of Amazon business is unlikely to grow because of AGI because Amazon retail customers will become poorer. Similar logic applies to Netflix, unless AGI would use it to spread its ideologies.

I am definitely not going to have children with such prospects in life.

This post and many of the comments are ignoring one of the main reasons that money becomes so much more critical post-AGI. It's because of the revolution in self-modification that ensues shortly afterwards.

Pre-AGI, a person can use their intelligence to increase their money, but not the other way around; post-AGI it's the opposite. The same applies if you swap intelligence for knowledge, health, willpower, energy, happiness set-point, or percentage of time spent awake.

This post makes half of that observation: that it becomes impossible to increase your money using your personal qualities. But it misses the other half: that it becomes possible to improve your personal qualities using your money.

The value of capital is so much higher once it can be used for self-modification.

For one thing, these modifications are very desirable in themselves. It's easy to imagine a present-day billionaire giving up all he owns for a modest increase along just a few of these axes, say a 300% increase in intelligence and a 100% increase in energy.

But even if you trick yourself into believing that you don't really want self-modification (most people will claim that immortality is undesirable, so long as they can't have it, and likewise for wireheading), there are race dynamics that mean you can't just ignore it.

People who engage in self-modification will be better equipped to influence the world, affording them more opportunities for self-modification. They will undergo recursive self-improvement similar to the kind we imagine for AGI. At some point, they will think and move so much faster than an unaugmented human that it will be impossible to catch up.

This might be okay if they respected the autonomy of unaugmented people, but all of the arguments about AGI being hard to control, and destroying its creators by default, apply equally well to hyperaugmented humans. If you try to coexist with entities who are vastly more powerful than you, you will eventually be crushed or deprived of key resources. In fact, this applies even moreso with humans than AIs, since humans were not explicitly designed to be helpful or benevolent.

You might say, "Well, there's nothing I can do in that world anyway, because I'm always going to lose a self-modification race to the people who start as billionaires, and being a winner-takes-all situation, there's no prize for giving it a decent try." However, this isn't necessarily true. Once self-modification becomes possible, there will still be time to take advantage of it before things start getting out of control. It will start out very primitive, resembling curing diseases more than engineering new capabilities. In this sense, it arguably already exists in a very limited form.

In this critical early period, a person will still have the ability to author their destiny, with the degree of that ability being mostly determined by the amount of self-modification they can afford.

Under some conditions, they may be able to permanently escape the influence of a hostile superintelligence (whether artificial or a hyperaugmented human). For example, a nearly perfect escape outcome could be achieved by travelling in a straight line close to the speed of light, bringing with you sufficient resources and capabilities to:

• Stay alive indefinitely
• Continue the process of self-improvement

In the chaos of an oncoming singularity, it's not unimaginable that a few people could slip away in that fashion. But it won't happen if you're broke.

Notes

• The line between buying an exocortex and buying/renting intelligent servants is somewhat blurred, so arguably the OP doesn't totally miss the self-modification angle. But it should be called out a lot more explicitly, since it is one of the key changes coming down the pike.
• Most of this comment doesn't apply if AGI leads to a steady state where humans have limited agency (e.g. ruling AGIs or their owners prevent self-modification, or humans are replaced entirely by AGIs). But if that sort of outcome is coming, then our present-day actions have no positive or negative effects on our future, so there's no point in preparing for it.

[sorry, have only skimmed the post, but I feel compelled to comment.]

I feel like unless we make a lot of progress on some sort of "Science of Generalisation of Preferences", for more abstract preferences (non-biological needs mostly fall into this), even if certain individuals have, on paper, much more power than others, at the end of the day, they likely rely on vastly superintelligent AI advisors to realise those preferences, and at that point, I think it is the AI advisor _really_ in control.
I'm not super certain of this, like, the Catholic Church definitely could decide to build a bunch of churches on some planets (though what counts as a church, in the limit?), but if they also want more complicated things like "people" "worshipping" "God" in those churches, it seems to be more and more up to the interpretation of the AI Assistants building those worship-maximising communes.

• Money will be able to buy results in the real world better than ever.
• People's labour gives them less leverage than ever before.
• Achieving outlier success through your labour in most or all areas is now impossible.
• There was no transformative leveling of capital, either within or between countries.

If this is the "default" outcome there WILL be blood. The rational thing to do in this case it to get a proper prepper bunker and see whats left when the dust have settled. 

I've also commented on Substack, but wanted to comment in a different direction here (which I hope is closely aligned to LessWrong values). This article feels like the first part of the equation for describing possible AI futures. I starts with the basis that because labour will be fully substitutable by AI, the value goes to zero (seems correct) to me. What about following up with the consequences of that loss of value? The things that people spend their earnings will go to zero too. What's in that group? The service industry, homes. 

Metrics we can track here are homelessness, unemployment, consumer debt, service industry bankruptcies. 

The government will need to redistribute wealth and because the ownership of capital is the only wealth anymore, it will need to tax capital. 

Perhaps exclude "cash" from "capital" and only define capital as ownership of means of production. 

Ultimately, how do we defend against this  future that only a few people seem to be able to see?

Interesting post. Some comments from an economist. 

You note,

The key economic effect of AI is that it makes capital a more and more general substitute for labour. There's less need to pay humans for their time to perform work, because you can replace that with capital (e.g. data centres running software replaces a human doing mental labour).

And also, 

Let's assume human mental and physical labour across the vast majority of tasks that humans are currently paid wages for no longer has non-trivial market value, because the tasks can be done better/faster/cheaper by AIs. Call this labour-replacing AI.

These are both incredibly strict assumptions. Jobs are bundles of related tasks and traditionally when capital has been able to substitute for labor, it boosts the productivity of the jobs that remain. But more to the point, we traditionally model out substitution at marginal rates that decline at some point. Your post suggests the opposite, that it is strictly increasing at all points. What would be the conditions that give rise to this situation? 

Second, as David Autor explains in a must-read paper on the topic, the demand for service jobs and other manual task-intensive work appears to be relatively income elastic. This means that when aggregate incomes go up, there is an increase in demand for these jobs as well. When all is said and done, computerization should indirectly raise demand for manual task-intensive occupations because it increases societal income. But it seems you think this will be broken. 

To be clear, I think AGI will be disruptive and labor replacing, but for further work, you might want to think about the world between here and when AGI can do all jobs. 
    
 

Humans seem way more energy and resource efficient in general, paying for top talent is an exception not the rule- usually it's not worth paying for top talent.

Likely to see many areas where better economically to save on compute/energy by having human do some of the work.

Split information workers vs physical too, I expect them to have very different distributions of what the most useful configuration is.

This post ignores likely scientific advances in bioengineering and cyborg surgeries, I expect humans to be way more efficient for tons of jobs once the standard is 180 IQ with a massive working memory

A great post.

Our main struggle ahead as a species is to ensure UBI occurs and in a generous rather than meager way. This direction is not at all certain and we should be warned by your example of feudalism as an alternate path that is perhaps looming as more likely. Nevertheless I agree we will see some degree of UBI because the alternative is too painful.

One of the ways you should add for those without capital to still rise post-AGI is celebrity status in sports, entertainment, arts. Consider that today humans still enjoy Broadway and still play chess, even though edited recordings and chess software can be better. Especially if UBI is plentiful, expect a surge in all sorts of non-economic status games. (Will we see a professional league of teams competing against freshly AI-designed escape rooms each week?)

The blog could be extended with greater consideration of future economic value-add by bits vs atoms. Enjoyed the comments discussing which investments today will hold up tomorrow. Can we agree that blue collar jobs will be safer, longer? AGI is digital but must exist in a physical world. And any generally acceptable post-AGI economy must still provide for human food, clothing, shelter, and health. Post-AGI we can expect a long period where AI-assisted humans (whose time has zero marginal cost if we stipulate UBI) are still cost-superior to building robots, especially in a world of legacy infrastructure. Hence there should still be entrepreneurship and social mobility around providing physical world products and services.

If states become AI dominated, will those AIs possess nationalist instincts? If so, we could see great conflicts continue or heighten in a fight for resources. Perhaps especially during the race to be first to AGI primacy in the next decade. But if not, perhaps we would see accommodations among unlikely borders and the unlocking of free trade for globally-optimized efficiency.

These are nuances to suggest and overall your thesis that social mobility could decline and the next decade may prove especially fateful is pretty solid. Let's all do what we can while we can.

I agree that this is a likely scenario. Very nice writeup. Post AGI is a completely resource and capital based economy Albeit I'm uncertain if humans will be allowed to keep their fiat currency, stocks, land, factories etc.

If I take the view that AGI has existed since late 2022 (ChatGPT), I can interpret all this as a statement about the present: capital matters more than ever, during AGI. 

After AGI is ASI, and at that point human capital is as valuable as ant capital or bear capital. 

interesting angle: given space travel, we'll have civilizations on other planets, that can't communicate fast enough with the mainland. presumanly, social hierarchies would be vastly different, and much more fluid there versus here on Earth

Labour-replacing AI will shift the relative importance of human v non-human factors of production, which reduces the incentives for society to care about humans while making existing powers more effective and entrenched.

And yet, ... society appears to be caring more about humans.       

And yet, ... existing powers (specifically the state) seem even less effective and entrenched. Open Borders policies are clearly an act of desperation ... while these policies appear to have been broadly rejected by the electorate. The state only opened the borders because of its crumbling power base. 

How is the contradiction resolved? How can technology seemingly reduce the relative importance of human v non human factors of production, while we can observe a frantic effort to outbid others in the global human auction?   

Global fertility collapse is greatly increasing the incentive for society to care about humans while at the same time making existing powers less effective and entrenched. Pursuing Open Borders policies demonstrates how desperate states  are to avoid extinction: This is now an existential struggle. Overlooking the welfare of one's own population by swamping communities with immigrants is the last line of defense- it is the least bad option before you close up shop. It is a clear act of desperation and has only been tried because of the profound fertility crisis that is underway. Fertility collapse will only escalate during the 21st Century and could become even more severe as AGI approaches. At some point, a full Demographic Arms Race might begin. 

States are highly motivated "to care" about humans or at least protect their self-interests because state power traditionally has manifested in the control of bricks and mortar reality and with it government services such as education and health care. Fertility collapse (and alternative learning technology) is creating large reductions in bricks and mortar school demand and with it the power of the deep state.  

The Darwinian struggle for national survival induced by extremely low total fertility rates then creates a paradoxical result in which the value of humans greatly increases and at the same time greatly reduces entrenched state power. 

My main issue with this post is that it seems substantially concerned with losing the ability for humans to achieve significant levels of wealth and power relative to each other, which I agree is important for avoiding a calcified ruling class (which tends to go poorly for a society, historically). But it should be viewed as a transitional concern as we look towards building a society where radical wealth disparities (critically, here defined as the power of the wealthy to effectively incentivize the less wealthy to endure unwanted experiences or circumstances, e.g., long work weeks, poorer nutrition or education, long commutes) don't exist. 

Human ambition isn't worth human poverty except exactly to the degree that it may be necessary to eliminate suffering. 

But why be so nihilistic about this? We can strive to conquer the solar system, the galaxy and the universe. Strive to understand why does the universe exist? Those seem pretty important to be worked on.

Great post!

What are your thoughts on guild effects in the sense that some of the changes you have described might be prevented through social contracts? Actors and screenwriters have successfully gone on strike to preserve their leverage, and many other professions are regulated. 

I think this might be a weak counter-argument, but nonetheless, it might distort the effects of AGI and slow down the concentration of capital.