For every choice of AGI difficulty, conditioning on gradual take-off implies shorter timelines.

The actual claim is “For every choice of AGI difficulty, conditioning on gradual take-off implies shorter timelines.” I’m OK with that claim.

However, the title is [was—it was changed in response to this comment! :) ] “Gradual take-off implies shorter timelines”, and I think that’s very misleading.

Also, the claim towards the top is “All else being equal, gradual take-off implies shorter timelines.”, and I think that’s somewhat misleading.

I think we both agree that it’s perfectly consistent for Person A to expect FOOM next week, and it’s perfectly consistent for Person B to expect a gradual takeoff between 2150 and 2200. You would say “Person A thinks AGI is much easier than Person B”, so that’s not “all else being equal”.

But I would say: “all else being equal” is kinda a hard thing to think about, and maybe poorly defined in general. After all, any two scenarios wherein one is fast-takeoff and the other is slow-takeoff will differ in a great many ways. Which of those ways is an “else” that would violate the “all else being equal” assumption? For example, are we going to hold “total research effort” fixed, or are we going to hold “total human research effort” fixed, or are we going to hold “resource allocation decision criteria” fixed? Well maybe there’s a sensible way to answer that, or maybe not, but regardless, I think it’s not intuitive.

So I think “Holding AGI difficulty fixed…” is much better than “All else being equal…”, and much much better than the post title which omits that caveat altogether.

I think the confusion stems from the word "gradual." It seems there's wide agreement that we have two dimensions of the AI timelines problem:

• Growth rate pattern. This could be linear, polynomial, exponential, or hyperbolic, and perhaps include a shift between two or more trends. Choice of coefficients matters too. Linear growth with a high slope could reach the threshold faster than exponential growth with small coefficients.
• AI difficulty

Paul's description of "gradualism" as stated actually does make a statement about both dimensions, though it's also somewhat ambiguous:

There will be a complete 4 year interval in which world output doubles, before the first 1 year interval in which world output doubles. (Similarly, we’ll see an 8 year doubling before a 2 year doubling, etc.)

US GDP last doubled from 1998-2021, a 23-year interval. Paul's statement seems to imply that a doubling can occur in no less than 25% the time of its predecessor. Under this assumption, we'd have at least 3 doublings left, over the course of at least 8 years, before doublings can occur in less than a year. This still allows AI takeoff by 2030; it just suggest that we'll be getting a warning shot over the next few years.

Another way to frame this, then, is that "For any choice of AI difficulty, faster pre-takeoff growth rates imply shorter timelines."

This statement is literally and obviously true, and avoids the counterintuitive aspect introduced by the word "gradual."

I agree with this post that the accelerative forces of gradual take-off (e.g., "economic value... more funding... freeing up people to work on AI...") are important and not everyone considers them when thinking through timelines.

However, I think the specific argument that "Gradual take-off implies shorter timelines" requires a prior belief that not everyone shares, such as a prior that an AGI of difficulty D will occur in the same year in both timelines. I don't think such a prior is implied by "conditioned on a given level of “AGI difficulty”". Here are two example priors, one that leads to "Gradual take-off implies shorter timelines" and one that leads to the opposite. The first sentence of each is most important.

Gradual take-off implies shorter timelines
Step 1: (Prior) Set AGI of difficulty D to occur at the same year Y in the gradual and sudden take-off timelines.
Step 2: Notice that the gradual take-off timeline has AIs of difficulties like 0.5D sooner, which would make AGI occur sooner than Y because of the accelerative forces of "economic value... more funding... freeing up people to work on AI..." etc. Therefore, move AGI occurrence in gradual take-off from Y to some year before Y, such as 0.5Y.

=> AGI occurs at 0.5Y in the gradual timeline and Y in the sudden timeline.

Gradual take-off implies longer timelines
Step 1: (Prior) Set AI of difficulty 0.5D to occur at the same year Y in the gradual and sudden take-off timelines. To fill in AGI of difficulty D in each timeline, suppose that both are superlinear but sudden AGI arrives at exactly Y and gradual AGI arrives at 1.5Y.
Step 2: Notice that the gradual take-off timeline has AIs of difficulties like 0.25D sooner, which would make AGI occur sooner than Y because of the accelerative forces of "economic value... more funding... freeing up people to work on AI..." etc. Therefore, move 0.5D AI occurrence in gradual take-off from Y to some year before Y, such as Y/2, and move AGI occurrence in gradual take-off correspondingly from 1.5Y to 1.25Y.

=> AGI occurs at 1.25Y in the gradual timeline and Y in the sudden timeline.

By the way, this is separate from Stefan_Schubert's critique [LW(p) · GW(p)] that very short timelines are possible with sudden take-off but not with gradual take-off, which I personally think can be considered a counterexample if we treat the impossibility of gradual take-off as "long" but not really a counterexample if we just consider the shortness comparison to be indeterminate because there are no very short gradual timelines.

For every choice of AGI difficulty, conditioning on gradual take-off implies shorter timelines.

What would you say about the following argument?

• Suppose that we get AGI tomorrow because of a fast take-off. If so timelines will be extremely short.
• If we instead suppose that take-off will be gradual, then it seems impossible for timelines to be that short.
• So in this scenario - this choice of AGI difficulty - conditioning on gradual take-off doesn't seem to imply shorter timelines.
• So that's a counterexample to the claim that for every choice of AGI difficulty, conditioning on gradual take-off implies shorter timelines.

I'm not sure whether it does justice to your reasoning, but if so, I'd be interested to learn where it goes wrong.

I think there's a problem with linking together AI research progress and economic output. The AI research -> economic value pipeline takes some time, the research AI models are not necessarily trivially adapted to generating economic value, and businesses need to actually figure out how to use AIs in their workflows. This means that in a gradual take-off world, where unintuitively AI research is happening faster pre-takeoff, the mechanisms which convert AI progress into value might not kickoff soon enough for us to observe them, for instance deepmind might be making so much progress so fast that it doesn't make sense to stop to monetize their current models because they know that they'll get better ones very soon. We can already notice this from the fact that there is so much economic low-hanging fruit remaining for current AI to pluck. It's in the sudden take-off world, where progress pre-takeoff is slower, that it makes sense to invest ressources to monetize your current models because you expect them to last some time before being deprecated. 

Even holding difficulty constant, a gradual takeoff will not result in a shorter timeline if investment is saturated in the sudden timeline (i.e. no additional resources could be effectively deployed).