The case for multi-decade AI timelines [Linkpost]

I still think full automation of remote work in 10 years is plausible, because it’s what we would predict if we straightforwardly extrapolate current rates of revenue growth and assume no slowdown. However, I would only give this outcome around 30% chance.

In an important sense I feel like Ege and I are not actually far off here. I'm at more like 65-70% on this. I think this overall recommends quite similar actions. Perhaps we have a more important disagreement regarding something like P(AGI within 3 years), for which I'm at approx. 25-30% and Ege might be very low (my probability mass is somewhat concentrated in the next 3 years due to an expectation that compute and algorithmic effort scaling will slow down around 2029 if AGI or close isn't achieved).

My guess is that this disagreement is less important to make progress on than disagreements regarding takeoff speeds/dynamics and alignment difficulty.

I think the main crux with AI 2027 is the very possibility of software-only singularity, rather than details of human economy, or details of software-only singularity. For people feeling skeptical about it, it's hard to find legible arguments that it can happen, and so they tend to lose interest in discussing its details, instead discussing other systems such as the datacenter-relevant parts of the economy, which doesn't chip away at this crux of the disagreement. I talk more about this in another comment [LW(p) · GW(p)] in response to Ege Erdil's post.

A well-known example where a system different from human industry [LW(p) · GW(p)] matters more for the dynamics of AI takeoff is nanotech, which makes the inputs derived from human industry irrelevant for scaling compute.

Many people are skeptical of nanotech, but alternatively there's macroscopic biotech, designing animal robots assembled with metamorphosis from small mobile things analogous to fruit flies, which can double biomass every 1-3 days (an impossible level of growth compared to the human economy). This only depends on procuring power and raw materials (rather than human technology) and provides manipulation in the physical world to assemble mines and factories and such (possibly even directly growing biological compute). There is much more of a proof of concept for this kind of thing than for nanotech.

• It underrates the difficulty of automating the job of a researcher. Real world work environments are messy and contain lots of detail that are neglected in an abstraction purely focused on writing code and reasoning about the results of experiments. As a result, we shouldn’t expect automating AI R&D to be much easier than automating remote work in general.

I basically agree. The reason I expect AI R&D automation to happen before the rest of remote work isn't because I think it's fundamentally much easier, but because (a) companies will try to automate it before other remote work tasks, and relatedly (b) because companies have access to more data and expertise for AI R&D than other fields.

Another potential crux[^related] is that Ege's world view seemingly doesn't depend at all on AIs which are much faster and smarter than any human. As far as I can tell, it doesn't enter into his modeling of takeoff (or timelines to full automation of remote work which partially depends on something more like takeoff).

On my views this makes a huge difference because a large number of domains would go much faster with much more (serial and smarter) intelligence. My sense is that a civilization where the smartest human was today's median human and also everyone's brain operated 50x slower^[1] would in fact make technological progress much slower. Similarly, if AIs were as much smarter than the smartest humans as the smartest human is smarter than the median human and also ran 50x faster than humans (and operated at greater scale than the smartest humans with hundreds of thousands of copies all at 50x speed for over 10 million parallel worker equivalents putting aside the advantages of serial work and intelligence), then we'd see lots of sectors go much faster.

My sense is that Ege bullet bites on this and thinks that slowing everyone down wouldn't make a big difference, but I find this surprising. Or maybe his views are that parallelism is nearly as good as speed and intelligence and sectors naturally scale up parallel worker equivalents to match up with other inputs, so we're bottlenecking on some other inputs in the important cases.

@Vladimir_Nesov [LW · GW] I republished the post, you may reply to @faul_sname [LW · GW].