Prediction Thread: Make Predictions About How Different Factors Affect AGI X-Risk.

mrthink

Prediction Thread: Make Predictions About How Different Factors Affect AGI X-Risk.

post by MrThink (ViktorThink) · 2023-02-27T19:15:05.349Z · LW · GW · 8 comments

In this post you can make several predictions for how different factors affect the probability that the creation of AGI leads to an extinction level catastrophe. This might be useful for planning.

Please let me know if you have other ideas for questions that could be valuable to ask.

Elicit Prediction (forecast.elicit.org/binary/questions/EviuxuEi3)

Predictions based on who develops AGI:

Elicit Prediction (forecast.elicit.org/binary/questions/v4fhohmfO)

Elicit Prediction (forecast.elicit.org/binary/questions/qDtulAocc)

Elicit Prediction (forecast.elicit.org/binary/questions/7kqkKdLCX)

Elicit Prediction (forecast.elicit.org/binary/questions/Ji-f9tH1B)

Predictions based on technology used for developing AGI:

Elicit Prediction (forecast.elicit.org/binary/questions/vEPhQz4Qz)

Elicit Prediction (forecast.elicit.org/binary/questions/pPq7NHe_C)

Prediction based on approach for creating AGI:

Elicit Prediction (forecast.elicit.org/binary/questions/fcZH5b2m8)

Elicit Prediction (forecast.elicit.org/binary/questions/ZvtW_KBSR)

Predictions on how money affects probability of AGI X-risk:

Elicit Prediction (forecast.elicit.org/binary/questions/yAooUTWqA)

Elicit Prediction (forecast.elicit.org/binary/questions/Aa7i8eqn1)

8 comments

Comments sorted by top scores.

comment by JBlack · 2023-02-28T02:16:52.618Z · LW(p) · GW(p)

I'm moderately confident that money would just make things worse, but there's no option for increase in catastrophe risk from increased donations.

Replies from: ViktorThink

↑ comment by MrThink (ViktorThink) · 2023-02-28T08:44:14.353Z · LW(p) · GW(p)

Sadly I could only create questions between 1-99 for some reason, I guess we should interpret 1% to mean 1% or less (including negative).

What makes you think more money would be net negative?

Do you think that it would also be negative if you had 100% of how the money was spent, or would it only apply if other AI Alignment researchers were responsible for the strategy to donate?

Replies from: JBlack

↑ comment by JBlack · 2023-03-02T00:04:46.175Z · LW(p) · GW(p)

I think more money spent right now, even with the best of intentions, is likely to increase capabilities much faster than it reduces risk. I think OpenAI and consequent capability races are turning out to be an example of this.

There are hypothetical worlds where spending an extra ten billion (or a trillion) dollars on AI research with good intentions doesn't do this, but I don't think they're likely to be our world. I don't think that directing who gets the money is likely to prevent it, without pretty major non-monetary controls in addition.

Replies from: ViktorThink

↑ comment by MrThink (ViktorThink) · 2023-03-02T08:23:11.361Z · LW(p) · GW(p)

I do agree that OpenAI is an example of good intentions going wrong, however I think we could learn from that and top researchers would be vary of such risks.

Nevertheless I do think your concerns are valid and is important not to dismiss.

comment by Daniel Kokotajlo (daniel-kokotajlo) · 2023-02-27T20:07:08.316Z · LW(p) · GW(p)

Disclaimer: my answers are wild guesses & poorly calibrated, I spent ~5s on each of them.

comment by Quintin Pope (quintin-pope) · 2023-02-27T19:33:58.875Z · LW(p) · GW(p)

The odds of any particular AGI destroying the world are << than the odds of AGI as a whole destroying the world, so the questions in "Prediction based on approach for creating AGI:" may be miscalibrated.

Replies from: JBlack, ViktorThink

↑ comment by JBlack · 2023-02-28T02:25:06.529Z · LW(p) · GW(p)

I interpreted that question as a conditional probability, and so not required to be strictly less. In particular, P(X causes catastrophe | X was of the given type, and the first AGI developed, on the given date).

Nothing requires that this conditional probability should be less than "some AGI destroys the world conditional on any AGI being developed".

↑ comment by MrThink (ViktorThink) · 2023-02-27T20:00:43.633Z · LW(p) · GW(p)

Excellent point.

I do think that the first AGI developed will have a big effect on the probability of doom, so hopefully it will be some value possible to derive from the question. But it would be interesting to control for what other AIs do, in order to get better calibrated statistics.