What's a good probability distribution family (e.g. "log-normal") to use for AGI timelines?
post by David Scott Krueger (formerly: capybaralet) (capybaralet) · 2022-04-13T04:45:04.649Z · LW · GW · 3 commentsThis is a question post.
Contents
Answers 7 Daniel Kokotajlo 4 Zac Hatfield-Dodds 3 davidad 3 GuySrinivasan 2 avturchin None 3 comments
Answers
My distribution has median 2030ish. That means it peaks a bit before then, and then has a long tapering tail.
I'm fairly confident in this, because I've done lots of thinking and research about it. I have talked to smart experts who disagree and think I could maybe even pass an ITT for them. That said, they could probably pass an ITT for me too.
I think that the best available starting point is the Cotra / OpenPhil report on biological anchors.
Personally I treat this as an upper bound on how much compute it might take, and hold the estimates of when compute will become available quite lightly. Nonetheless, IMO serious discussion of timelines should at least be able to describe specific disagreements with the report and identify which differing assumptions give rise to each.
↑ comment by InquilineKea · 2022-04-13T17:31:01.397Z · LW(p) · GW(p)
Upper bound so there's a high chance that it will come more quickly than that.
Log-normal is a good first guess, but I think its tails are too small (at both ends).
Some alternatives to consider:
- Erlang distribution (by when will k Poisson events have happened?), or its generalization, Generalized gamma distribution
- Frechet distribution (what will be the max of a large number of i.i.d. samples?) or its generalization, Generalized extreme value distribution
- Log-logistic distribution (like log-normal, but heavier-tailed), or its generalization, Singh–Maddala distribution
Of course, the best Bayesian forecast you could come up with, derived from multiple causal factors such as hardware and economics in addition to algorithms, would probably score a bit better than any simple closed-form family like this, but I'd guess literally only about 1 to 2 bits better (in terms of log-score).
It looks like normal distribution around some mean value (2040+-10) is typically assumed, but it wrong, as obviously AGI probability is growing exponentially.
So some exponential distribution is needed, in which mean value is not far from unity. E.g.: 10 percent for 2030, 50 percent for 2035 and 99 percent for 2040.
↑ comment by Vivek Hebbar (Vivek) · 2022-04-13T10:00:01.110Z · LW(p) · GW(p)
This seems very wrong to me:
- What do you mean by "probability is growing exponentially"? Cumulative probabilities cannot grow exponentially forever, since they are bounded by [0,1].
- It seems salvageable if what you mean by "probability" is (dP/dt)/P, where P is cumulative probability.
- Can you clarify what gears-level model leads to exponential growth in "probability"?
- Even if you pick a gears-level model of AGI probability which resembles an exponential within the domain [0, 0.99], you should have a lot of uncertainty over the growth rate. Thus your model should be a mixture of exponentials with a fairly wide spread over hyperparameters, and it is highly unlikely that you would get 99% by 2040, given how little we know about AGI.
↑ comment by avturchin · 2022-04-14T11:21:56.612Z · LW(p) · GW(p)
Ok, on gear level I have the following model: imagine that Moore's law is true and the AI risk is proportional to the amount of available compute, via some unknown coefficient. In that model, after the cumulative risk reaches 50 per cent, 99 per cent is very near.
But we need some normalisation to escape probabilities above one, for example, by using odds ratio.
Also, cumulative risk is exponential even for constant probability density. If probability density is exponential, cumulative risk is double exponential.
It all means that if AI risk some sizeable digit, 10 per cent or 50 percent, there is only few years until almost certain end. In other words, there is no difference between two claims "10 per cent of AI in 2040" and "90 per cent of AI in 2040", as both mean that the end will be in 2040s.
3 comments
Comments sorted by top scores.
comment by habryka (habryka4) · 2022-04-13T05:21:53.100Z · LW(p) · GW(p)
Are you looking for something like "a normal distribution", or for something like "10% by 2030, 50% by 2050, 90% by 209
Replies from: capybaralet↑ comment by David Scott Krueger (formerly: capybaralet) (capybaralet) · 2022-04-13T18:07:46.452Z · LW(p) · GW(p)
like "a normal distribution", thx!
comment by David Scott Krueger (formerly: capybaralet) (capybaralet) · 2022-08-12T17:50:21.772Z · LW(p) · GW(p)
AI Impacts used a Gamma distribution here:
https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/