A brief history of the automated corporation
post by owencb · 2024-11-04T14:35:04.906Z · LW · GW · 1 commentsThis is a link post for https://strangecities.substack.com/p/a-brief-history-of-the-automated
Contents
Looking back from 2041 Remarks None 1 comment
Looking back from 2041
When people in the early 21st Century imagined an AI-empowered economy, they tended to project person-like AI entities doing the work. “There will be demand for agent-like systems,” they argued, “so we’ll see AI labs making agents which can then be deployed to various problems”.
We now know that that isn’t how it played out. But what led to the largely automated corporations that we see today? Let’s revisit the history:
- In the mid–late 2020s, as AI systems became reasonably reliable at many tasks, workers across the economy started consulting them more on an everyday basis
- People and companies started more collection of data showing exactly what they wanted from different tasks
- Systematizers and managers began building company workflows around the automation of tasks
- They would build systems to get things into shapes known to work especially well for automation — in many cases using off-the-shelf software solutions — and direct more of the work into these routes
- In many cases, the automation of a particular tasks involved brief invocation of specialized agent-like systems; but nothing like the long-term general purpose actors imagined in science fiction
- As best practices emerged for automating taskflows, in the early–mid 2030s we saw the start of widespread automation of automation — people used specialized AI systems (or consultants relying on such systems) to advise on which parts of the workflow should be automated and how
- For a while, human experts and managers kept a close eye on these automated loops, to catch and correct errors
- But it wasn’t long before these management processes themselves were largely automatable (or redundant), and humans just stayed in loop for the high-level decisions about how to arrange different workflows and keep them integrated with the parts still done by humans
- Although there are some great anecdotes of failures during that time, the broad trend was towards it being economically efficient to automate larger and larger swathes of work
- At this stage, many companies were still run by people who were slow adopters of technology
- Over the mid and late 2030s, many of these went out of business, as they failed to be competitive on price
- There was significant social unhappiness at the shocks to the labour market
- Lagging behind the automation of existing workflows was the automation of creating new workflows
- Still, this was pioneered by management consultancies, who had access to some of the best data sets about what worked well in what circumstances
- The first fully-automated corporations, with no human workers, were seen in 2032 — but these were mostly gimmicks
- They had human boards playing a role somewhat like that of management — and they weren’t terribly successful
- Still, they proved the concept, and over the next few years the rise of fully automated management layers was tremendous
- Many companies in this period ended up with a human board of directors, and human employees performing some tasks which were particularly well-suited for humans, but effectively no humans in management
- It was not until the cheap general-purpose robots of the late 2030s that many firms eschewed human workers even for those physical tasks which hadn’t already merited specialised robots
- Many companies in this period ended up with a human board of directors, and human employees performing some tasks which were particularly well-suited for humans, but effectively no humans in management
- In many jurisdictions, there was until recently (and in several jurisdictions there still is) a requirement for the officers of the company to be human; and except in two small pioneering countries, it’s still required that the board of directors be human
- But even people nominally in these “human-required” roles are increasingly turning to AI systems to do much or all of their work
- This approaches the ecosystem we see today, where many companies (and a clear majority of new companies) are essentially AI-run: the basic case for them is proposed by AI systems, and AI builds out all of the core systems
- Best practices continue to evolve, as they are now best practices for automated corporations, which differ from the best practices in the world where humans played important roles
- We did see a significant slide towards large conglomerates and “mega-corporations”, as it was generally the biggest companies with the most data on successful management practices worked well who were in the best position to start new firms
- This was significantly stopped by regulators intervening to break up monopolies
- Regulators showed greater willingness to take large actions here than in the human era, as there was normally less loss of efficiency from breaking up monopolies
- This was significantly stopped by regulators intervening to break up monopolies
- To date, the concerns of the doom-mongers about AI catastrophes from corporations without human oversight have not materialized — while there were some harms (and consequent large lawsuits) caused by automated firms, research shows that on average these firms have caused significantly less litigateable harm than the human-run firms they replaced
- Some researchers remain concerned about the possibility of “triggering events” for mass errors by automated systems
- In some countries, governments concerned about fragility have supported an ecosystem with varied management software; but in other jurisdictions we see effective monocultures
- There are widespread beliefs that these systems are doing damage to the fabric of society, but there is no consensus on the nature or degree of the alleged harms, and the companies accused usually paint the concerns as being grounded in unhappiness about the displacement of human workers
- Some researchers remain concerned about the possibility of “triggering events” for mass errors by automated systems
- However, concerns about systematizing unethical — and sometimes even illegal — behaviour in automated corporations have been vindicated; research indicates this is still happening at significant scale
- After the scandals and lawsuits of 2036, the US and EU each passed laws to ensure that the service providers would be liable (and in some cases criminally responsible) if their services were deemed to be accomplices in breaking the law
- Since then, the main service providers have been clear that their services cannot be used in such roles
- However, there is a large grey-market economy of small companies which provide (second-tier but unfettered) services to a smallish number of firms (which may use them for functions which benefit from a lack of scruples, and top-tier services for functions which do not)
- Occasionally their clients are found to have behaved illegally; the small service companies then go bankrupt; but the bet was good in expectation for their owners
- Various regulatory responses have been proposed
- Estonia has recently been innovating with automated regulators to keep up with automated corporations — the ability to have more thorough oversight of firms in principle makes up for their ability to act with little human oversight
- In most jurisdictions, regulators have been much slower to adopt new technology than the firms they are regulating
- This is partially because there is resistance to the idea that legislating should be turned over to automated services; and partially due to highly organized (and “automated” would be a safe bet) lobbying campaigns
- There are some pushing for more international harmonization on these topics, arguing that much of the corporate abuse is not illegal per se, but arises from aggressively pursuing loopholes and differences between jurisdictions to extract competitive advantage
- However, there is a large grey-market economy of small companies which provide (second-tier but unfettered) services to a smallish number of firms (which may use them for functions which benefit from a lack of scruples, and top-tier services for functions which do not)
Today, there are a few instances of fully autonomous corporations, with no human control even in theory, as well as a larger number of fully autonomous AI agents, generally created by hobbyists or activists. However, while intriguing (and suggestive about how the future might unfold), to date these remain a tiny fraction.
And although AI for research has been one of the slower applications to find a niche for properly automated groups (with many cases of AI used at the management level coordinating human researchers, who in turn make use of AI research assistants; although this varies by field), it still appears to have made a difference. On most measures, technological progress was around 1.5–2x faster in the period 2030–2035, compared to a decade earlier (2020–2025), and the second half of the 2030s was faster again. Moreover, in the last couple of years we have been seeing an increase in successes out of purely automated research groups. A controversial AI-produced paper published in Science earlier this year claimed that the rate of technological progress is now ten times faster than it was at the turn of the century. Since IJ Good first coined the idea of an intelligence explosion, 75 years ago last year, people have wondered if we will someday see a blistering rate of progress, that is hard to wrap our heads around. Perhaps we are, finally, standing on the cusp — and the automated corporations we have developed stand ready to work, integrating the fruits of that explosion back into human society.
Remarks
As is perhaps obvious: this is not a prediction that this is how the future will play out. Rather, it’s an exploration of one way that it might play out — and of some of the challenges that might arise if it did.
Thanks to Raymond Douglas, Max Dalton, and Tom Davidson, and Adam Bales, for helpful comments.
1 comments
Comments sorted by top scores.
comment by Viliam · 2024-11-08T14:49:39.882Z · LW(p) · GW(p)
Why do most humans in 2041 still need to work 40 hours a week? The answer is complicated, but to keep this comment simple, let's focus on a few factors that even a hypothetical reader from 2024 would understand.
In most countries, government regulation requires humans in the loop. These might seem like bullshit jobs, but that doesn't make the competition for them any less fierce. An average person cannot get a good job without good credentials (required for regulatory reasons), and good credentials are expensive; it often takes a lifetime to pay back the school debt. It doesn't matter whether the things taught at school are useful in any practical sense (the few remaining human teachers mostly agree that they are not), but they are required by law. The official reasoning is that general education keeps us human (note: this is simplified to the level of strawman, but I am trying to keep it simple for the hypothetical 2024 reader unfamiliar with the culture wars of 2041).
With the exception of a few things such as rent, most things today are significantly cheaper than they used to be in 2024. On the other hand, there are new expenses, many of them related to AI. Some aspects of life got complicated, for example contracts of all kinds. To put it bluntly, you need the latest AI to safely navigate the legal minefield created by the latest AI. Trying to save money by using a cheaper version of AI that is several weeks obsolete is generally considered a very bad idea, and will probably cost you more in long run, because you have no idea what you sign (and you should generally assume that the form was optimized to extract as much value from you as legally possible, otherwise the company would be leaving money on table). You either spend a large part of your income on AI services... or you risk joining the underclass at the first accident; there is not much of a middle way. If you can't afford the "business version" of the latest AI, you can get one that is supported by advertising -- the less you pay for it, the more you should expect the AI agent to optimize for the goals of the advertisers rather than your personal goals. (Oh, "advertisement" today no longer means trying to influence the humans. Humans are mostly irrelevant. It means influencing the AI agents that make most of the everyday decisions. As a simple example, you can pay the AI agents to buy your products rather than your competitor's products, even if they are somewhat more expensive or worse, and to defend this choice to human users using individually optimized arguments.)
There is increasingly addictive... well, basically everything. I am afraid that a far [? · GW]-mode description will fail to convey how strong the effect is when experienced in near mode, but basically: The salesmen of old have used only a few dozen simple techniques (such as smiling at you, looking in your eyes, repeating your name, trying to anchor you to a higher price and then giving you a discount, creating a false sense of urgency, etc.) which were only statistically effective and often failed or backfired for you, the modern ones come to you with a full AI-powered analysis of your personality (yes, there are regulations against this, but they are trivially circumvented), and they have probably already spent a few previous months trying to influence you in all known ways (bots pretending to be humans contacting you on social networks and nudging you in the desired direction, advertising in your AI agent if you use the cheaper version, subliminal advertising on the streets flashing when the screen detects you looking at it, etc.) which makes is almost impossible to resist; in many cases the humans believe that the interaction was actually their own idea, and quite often they fall in love with the salesperson.
Some people suggest that this is a problem humanity should focus on solving, but the respected economists (and more importantly, their AI advisors) mostly shrug and say: "revealed preferences".