Is this true? paulg: [One special thing about AI risk is that people who understand AI well are more worried than people who understand it poorly]

post by tailcalled · 2023-04-01T11:59:45.038Z · LW · GW · 1 comment

This is a question post.

Contents

  Answers
    10 rahulxyz
    5 CuriousApe
None
1 comment

Link to claim.

I've seen this claim before but I haven't seen any direct data supporting it. Does anyone have any resources?

Answers

answer by rahulxyz · 2023-04-01T13:45:28.593Z · LW(p) · GW(p)

There doesn't seem to be many surveys of the general population on doom type scenarios. Most of them seem to be based on bias/weapons type scenario. You could look at something like metaculus but I don't think that's representative of the general population.

Here's a breakdown of AI researchers: https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/ (median /mean of extinction is 5%/14%)

US Public: https://governanceai.github.io/US-Public-Opinion-Report-Jan-2019/general-attitudes-toward-ai.html (12% of americans think it will be "extremely bad i.e extinction)

Based on the very weak data above, it doesn't seem like a huge divergence of opinion specifially for x-risk
 

answer by CuriousApe · 2023-04-01T16:46:00.386Z · LW(p) · GW(p)

Anecdotally this feels very true. Those outside of the AI community feel way more optimistic than those I know who work in AI. The general population who are aware of GPT and LLM's seem way too optimistic. When I talk to people about an AGI capabilities moratorium, AI researchers are way more likely to agree than those not working in AI. 

↑ comment by tailcalled · 2023-04-01T16:53:23.384Z · LW(p) · GW(p)

Where do you know people who work in AI from? And where do you know people in the general population from?

(This might seem like a weird question, so lemme explain. If you e.g. know people who work in AI from LessWrong, and know people in the general population from your family, then these are separate mechanisms and it would seem that these could induce a collider bias, distorting the correlations.)

↑ comment by DFNaiff (danilo-naiff) · 2023-04-01T20:11:32.060Z · LW(p) · GW(p)

That is not my experience at all. Maybe it is because my friends from outside of the AI community are also outside of the tech bubble, but I've seen a lot of pessimism recently with the future of AI. In fact, they seem to easily both the orthogonality and the instrumentality thesis. Although I avoid delving into this topic of human extinction, since I don't want to harm anyone's mental health, the rare times were this topic comes up they seem to easily agree that this is a non-trivial possibility.

I guess the main reason is that, since they are outside of the tech bubble, they don't seem to think that worrying about AI risk is being a Luddite, not truly understanding AI, or something like that. Moreover, since none of them works in AI, they don't take any personal offense with the suggestion that capabilities advance may greatly harm humanity.

1 comment

Comments sorted by top scores.

comment by SomeoneYouOnceKnew · 2023-04-01T20:41:27.177Z · LW(p) · GW(p)

Part of the problem with verifying this is that the number of machine learning people who got into machine learning due to lesswrong. We need more machine learning people whom were able to come to doom conclusions of their own accord, independent of hpmor etc, as a control group.

As far as I can tell, the number worried about doom overlap 1:1 with lesswrong posters/readers, and if it was such a threat, we'd expect there to be some number of people coming to the conclusions independently/of their own accord.