Thanks for this post! I'll write a fuller response later, but for now I'll say: These arguments prove too much; you could apply them to pretty much any technology (e.g. self-driving cars, 3D printing, reusable rockets, smart phones, VR headsets...). There doesn't seem to be any justification for the 50-year number; it's not like you'd give the same number for those other techs, and you could have made exactly this argument about AI 40 years ago, which would lead to 10-year timelines now. You are just pointing out three reasons in favor of longer timelines and then concluding

it's a bit difficult to see how we will get transformative AI developments in the next 50 years. Even accepting some of the more optimistic assumptions in e.g. Ajeya Cotra's Draft report on AI timelines [LW · GW], it still seems to me that these effects will add a few decades to our timelines before things get really interesting.

Which seems unwarranted to me. I agree that the things you say push in the direction of longer timelines, but there are other arguments one could make that push in the direction of shorter timelines, and it's not like your arguments are so solid that we can just conclude directly from them that timelines are long--and specifically 50+ years long!

Thanks for the nice post! Here's why I disagree :)

Technological deployment lag

Normal technologies require (1) people who know how to use the technology, and (2) people who decide to use the technology. If we're thinking about a "real-deal AGI" that can do pretty much every aspect of a human job but better and cheaper, then (1) isn't an issue because the AGI can jump into existing human roles. It would be less like "technology deployment" and more like a highly-educated exquisitely-skilled immigrant arriving into a labor market. Such a person would have no trouble getting a job, in any of a million different roles, in weeks not decades. For (2), the same "real-deal AGI" would be able to start companies of its own accord, build factories, market products and services, make money, invest it in starting more companies, etc. etc. So it doesn't need anyone to "decide to use the technology" or to invest in the technology.

Regulation will slow things down

I think my main disagreement comes from my thinking of AGI development as being "mostly writing and testing code inside R&D departments", rather than "mostly deploying code to the public and learning from that experience". I agree that it's feasible and likely for the latter activity to get slowed down by regulation, but the former seems much harder to regulate for both political reasons and technical reasons.

The political is: It's easy to get politicians riled up about the algorithms that Facebook is actually using to influence people, and much harder to get politicians riled up about whatever algorithms Facebook is tinkering with (but not actually deploying) in some office building somewhere. I think there would only be political will once we start getting "lab escape accidents" with out-of-control AGIs self-replicating around the internet, or whatever, at which point it may well be too late already.

The technical is: A lot of this development will involve things like open-source frameworks to easily parallelize software, and easier-to-use faster open-source implementations of new algorithms, academic groups publishing papers, and so on. I don't see any precedent or feasible path for the regulation of these kinds of activities, even if there were the political will.

Not that we shouldn't develop political and technical methods to regulate that kind of thing—it seems like worth trying to figure out—just that it seems extremely hard to do and unlikely to happen.

Overestimating the generality of AI technology

My own inside-view story (see here for example [LW · GW]) is that human intelligence is based around a legible learning algorithm, and that researchers in neuroscience and AI are making good progress in working out exactly how that learning algorithm works, especially in the past 5 years. I'm not going to try to sell you on that story here, but fwiw it's a short-ish timelines story that doesn't directly rely on the belief that currently-popular deep learning models are very general, or even necessarily on the right track.

I don't think technological deployment is likely to take that long for AI's. With a physical device like a car or fridge, it takes time for people to set up the factories, and manufacture the devices. AI can be sent across the internet in moments. I don't know how long it takes google to go from say an algorithm that detects streets in satellite images to the results showing up in google maps, but its not anything like the decades it took those physical techs to roll out.

The slow roll-out scenario looks like this, AGI is developed using a technique that fundamentally relies on imitating humans, and requires lots of training data. There aren't nearly enough data from humans that are AI experts to make an AI AI expert. The AI is about as good at AI research as the median human. Or maybe the 80th percentile human. Ie no good at all. The AI design fundamentally requires custom hardware to run at reasonable speeds. Add in some political squabbling and it could take a fair few years before wide use, although there would still be huge economic incentive to create it. 

The fast scenario is the rapidly self improving superintelligence. Where we have oodles of compute by the time we crack the algorithms. All the self improvement happens very fast in software. Then the AI takes over the world. (I question that "a few weeks" is the fastest possible timescale for this. )

(For that matter, the curves on the right of the graph look steeper. It takes less time for an invention to be rolled out nowadays)

For your second point, you can name biases that might make people underestimate timelines, I can name biases that might make people overestimate timelines. (eg Failure to consider techniques not known to you) And it all turns into a bias naming competition. Which is hardly truth tracking at all.

As for regulation, I think its what people are doing in R&D labs, not what is rolled out that matters. And that is harder to regulate. I also explicitly don't expect any AI Chernobyl. I don't strongly predict there won't be an AI Chernobyl either. I feel that if the relevant parties act with the barest modicum of competence,  there won't be an AI Chernobyl. And the people being massively stupid will carry on being massively stupid after any AI Chernobyl.

Re: 1, I think it may be important to note that adoption has gotten quicker (e.g. as visualized in Figure 1 here; linking this instead of the original source since you might find other parts of the article interesting). Does this update you, or were you already taking this into account? 

I broadly agree with these points, and (1) and (3) in particular lead to me to shade the bio anchors estimates upwards by ~5 years (note they are already shaded up somewhat to account for these kinds of effects).

I don't really agree on (2).

I see no strong reason to doubt the narrow version of this thesis. I believe it's likely that, as training scales, we'll progressively see more general and more capable machine learning models that can do a ton of impressive things, both on the stuff we expect them to do well on, and some stuff we didn't expect.

But no matter how hard I try, I don't see any current way of making some descendant of GPT-3, for instance, manage a corporation.

I feel like if you were applying this argument to evolution, you'd conclude that humans would be unable to manage corporations, which seems too much. Humans seem to do things that weren't in the ancestral environment, why not GPTs, for the same reason?

You might say "okay, sure, at some level of scaling GPTs learn enough general reasoning that they can manage a corporation, but there's no reason to believe it's near". But one of the major points of the bio anchors framework is to give a reasonable answer to the question of "at what level of scaling might this work", so I don't think you can argue that current forecasts are ignoring (2).

Perhaps you just mean that most people aren't taking bio anchors into account and that's why (2) applies to them -- that seems plausible, I don't have strong beliefs about what other people are thinking.

Planned summary for the Alignment Newsletter:

This post outlines and argues for three reasons to expect long AI timelines that the author expects are not taken into account in current forecasting efforts:

1. **Technological deployment lag:** Most technologies take decades between when they're first developed and when they become widely impactful.
2. **Overestimating the generality of AI technology:** Just as people in the 1950s and 1960s overestimated the impact of solving chess, it seems likely that current people are overestimating the impact of recent progress, and how far it can scale in the future.
3. **Regulation will slow things down,** as with [nuclear energy](https://rootsofprogress.org/devanney-on-the-nuclear-flop), for example.

You might argue that the first and third points don’t matter, since what we care about is when AGI is _developed_, as opposed to when it becomes widely deployed. However, it seems that we continue to have the opportunity to intervene until the technology becomes widely impactful, and that seems to be the relevant quantity for decision-making. You could have some specific argument like “the AI goes FOOM and very quickly achieves all of its goals” that then implies that the development time is the right thing to forecast, but none of these seem all that obvious.

Planned opinion:

I broadly agree that (1) and (3) don’t seem to be discussed much during forecasting, despite being quite important. (Though see e.g. [value of the long tail](https://www.lesswrong.com/posts/Nbcs5Fe2cxQuzje4K/value-of-the-long-tail).) I disagree with (2): while it is obviously possible that people are overestimating recent progress, or overconfident about how useful scaling will be, there has at least been a lot of thought put into that particular question -- it seems like one of the central questions tackled by <@bio anchors@>(@Draft report on AI timelines@). See more discussion in this [comment thread](https://www.alignmentforum.org/posts/Z5gPrKTR2oDmm6fqJ/three-reasons-to-expect-long-ai-timelines?commentId=F7FNee8Bpa8hemQkd).

i think we die probly this decade, and if not then probly next decade.

i partly explain my short timelines here. the short version is that i think recursive self-improvement (RSI) is not very hard to build. to kill everyone, you don't need lots of compute (though it helps), which means you don't need to be in a lab, which means you're not affected by regulation unless the regulation is "all computers are banned", which it's not going to be. you don't need to build "general" AI (whatever that means), you just need to build RSI. the hardest variable to predict is how many people are gonna be trying to build something like RSI, which is why my prediction is as vague as it is, but i think doom from RSI could happen any day, it just gets more likely on future days than on, say, tomorrow, because more people are trying with more powerful computers and more powerful available AI tech they can use.

I agree that 1-3 need more attention, thanks for raising them.

Many AI scientists in the 1950s and 1960s incorrectly expected that cracking computer chess would automatically crack other tasks as well.

There’s a simple disconnect here between chess and self-supervised learning.  You're probably aware of it but it it's worth mentioning. Chess algorithms were historically designed to win at chess. In contrast, the point of self-supervised learning is to extract representations that are useful in general. For example, to solve a new tasks we can feed the representations into a linear regression, another general algorithm. ML researchers have argued for ages that this should work and we already have plenty of evidence that it does.

I'm curious how you'd estimate the relative importance of these factors. Myself, I think one of them really outweighs the others.