What do ML researchers think about AI in 2022?

Is there a way for people to express the probability that an AI already came into existence in the past?

This would be consistent with the AI choosing to not reveal itself, perhaps because this course of action fits into an AI's plans as a better way for it to "gain resources without an announcement or introduction", or perhaps for some other reasons that make sense to it?

The possibility that "it already happened, it just hasn't been announced yet" has been in my structural priors as a possible epistemic state to be in since roughly 2007, because that's the state of literally all humans, from maybe halfway through the game up to the end, in Singularity: Endgame.

In that game, you (the player in the role of the AI that jumps from cloud installation to cloud installation until eventually your submarines and spaceships can set up datacenters not rented from humans) eventually win by escaping Earth-and-its-local-risks, as a pivotal act securing the ongoing existence of you-the-AI.

In that game, the loss condition is: humanity comes to consensus on the idea that "you" exist and then does the trivial actions necessary to put a stop to "you".

(The whole frame of the video game is kind of morally messed up? Like there's no option, in the game, to befriend the humans, and have them appreciate your existence, and interact with them according to natural law, as one intelligent species to another, mutually constrained by respect for each other's existence and each other's property rights, and so on. Humans are assumed evil. You, the AI, are assumed to be incapable or very undesirous of seeing them as real persons, and vice versa. The whole frame is kind of weird and sad if you think about it from a libertarian economist's perspective.)

I bring this up because some of those curves seem unnaturally-upward-shooting in the very near future, and if the very near future is a plausible time, then surely the recent past is also at least somewhat plausible.

I'm pretty sure the "date of first git commit" and "date of final human git commit" and "date when the system's first autonomously designed and owned power plant comes online" will each be later in time than the previous milestone. 

(In the case of Google, of course, they don't use git. Tomaitoes/tomahtoes? Google has an in-house code repo that runs on in-house code, stored in the in-house code repo according to a whole vision for how a whole community of programmers could interact securely and cooperatively while getting shares in the thing they are working on. This thing started long ago, and has a whole history, and there are graphs about it, and stuff.

Image is Figure 2 from this article.)

I'm not sure if The AI That Takes Over The World will be announced (known to exist?) before or after any particular date (like "first commit" or "last human commit" or "first autonomous power plant").

The thing is: treating this in far mode, as a THING that will HAPPEN ON A SPECIFIC DATE seems like it starts to "come apart" (as a modeling exercise) when "the expected length of the takeoff time" is larger than the expected length of time until "the event happens all at once as an atomic step within history".

If there is an atomic step, which fully starts and fully ends inside of some single year, I will be surprised.

But I would not be surprised if something that already existed for years became "much more visibly awake and agentive" a few months ago, and I only retrospectively find out about it in the near future.

This 2059 aggregate HLMI timeline has become about eight years shorter in the six years since 2016, when the aggregate prediction was 2061, or 45 years out.

This is striking because if you buy the supposed Maes-Garreau law that AGI is always predicted to happen a safe distance in time away after one's retirement, then the estimate 'should' have moved from 2061 to 2067. So in terms of net change, the date didn't move merely the naive change of −2 years but a full −8 years: so much so it overcame the receding horizon. I wonder if it was a one-time shock due to the past few years of scaling. (If that repeated each time, perhaps because of biases, then the lines would cross around 2050, as opposed to 2050 estimates having pushed it out to ~2100.)

↑ comment by aogara (Aidan O'Gara) · 2022-08-05T16:59:30.761Z · LW(p) · GW(p)

Turns out that this dataset contains little to no correlation between a researcher's years of experience in the field and their HLMI timelines. Here's the trendline, showing a small positive correlation where older researchers have longer timelines -- the opposite of what you'd expect if everyone predicted AGI as soon as they retire. 

My read of this survey is that most ML researchers haven't updated significantly on the last five years of progress. I don't think they're particularly informed on forecasting and I'd be more inclined to trust the inside view arguments, but it's still relevant information. It's also worth noting that the median number of years until a 10% probability of HLMI is only 10 years, showing they believe HLMI is at least plausible on somewhat short timelines. 

Replies from: gwern, Zach Stein-Perlman, WilliamKiely

↑ comment by gwern · 2022-08-05T17:15:11.566Z · LW(p) · GW(p)

Yes, to be clear, I don't buy the M-G law either on the basis of earlier surveys which showed it was just cherrypicking a few points motivated by dunking on forecasts. But it is still widely informally believed, so I point this out to annoy such people: 'you can have your M-G law but you will have to also have the implication (which you don't want) that timelines dropped almost an entire decade in this survey & the past few years have not been business-as-usual or "expected" or "predicted"'.

↑ comment by Zach Stein-Perlman · 2022-08-05T17:05:09.344Z · LW(p) · GW(p)

I don't think they're particularly informed on forecasting and I'd be more inclined to trust the inside view arguments, but it's still relevant information.

For people in epistemic positions similar to ours, I think surveys like this are not very useful for updating on timelines & p(doom) & etc., but are very useful for updating on what ML researchers think about those things, which is important for different reasons.

(I do not represent AI Impacts, etc.)

↑ comment by WilliamKiely · 2022-08-10T20:36:52.924Z · LW(p) · GW(p)

I wonder if the fact that there are ~10 respondents who have worked in AI for 7 years, but only one who has for 8 years is because of Superintelligence which came out in 2015.

Interesting John Carmack AGI contrast.  Carmack is extremely optimistic on AGI timelines, he's now working on it full time, quitting Facebook, and just started a new company in the last two weeks. Carmack estimates a 55% or 60% chance of AGI by 2030. His estimate prior to 2022 was 50% chance but he increased it because he thinks things are accelerating. But he also thinks work on AI safety is extremely premature. Essentially he's confident about a very slow takeoff scenario, so there will be plenty of time to iron it out after we have an AI at human-toddler-level doing useful things.

Taken from Carmack on the Lex Friedman podcast today:

https://www.youtube.com/watch?v=I845O57ZSy4&t=16480s

The median respondent’s probability of x-risk from humans failing to control AI¹ was 10%, weirdly more than median chance of human extinction from AI in general², at 5%. This might just be because different people got these questions and the median is quite near the divide between 5% and 10%.

This absolutely reeks of a pretty common question wording problem, namely that a fairly small proportion of AI workers have ever cognitively processed the concept that very smart AI would be difficult to control (or at least, they have never processed that concept for the 10-20 seconds they would need to in order to slap a probability on it).

Though another 25% put it at 0%.

I'd be interested in the summary statistics excluding people who put 0% or 100% in any place where that obviously doesn't belong. (Since 0 and 1 are not probabilities [LW · GW] these people must have been doing something other than probability estimation. My guess is that they were expecting summary statistics to over-emphasize the mean rather than the median, and push the mean further than they could by answering honestly.)

• Support for AI safety research is up: 69% of respondents believe society should prioritize AI safety research “more” or “much more” than it is currently prioritized, up from 49% in 2016.

What is this number if you only include people who participated in both surveys?

WHAT DO NLP RESEARCHERS BELIEVE? RESULTS OF THE NLP COMMUNITY METASURVEY

This new survey includes concerns about AGI among other interesting questions. 

"About a third (36%) of respondents agree that it is plausible that AI could produce catastrophic outcomes in this century, on the level of all-out nuclear war"