Survey of NLP Researchers: NLP is contributing to AGI progress; major catastrophe plausible
post by Sam Bowman (sbowman) · 2022-08-31T01:39:54.533Z · LW · GW · 6 commentsContents
6 comments
I was part of a group that ran a PhilPapers-style survey and metasurvey targeting NLP researchers who publish at venues like ACL. Results are here (Tweet-thread version). It didn't target AGI timelines, but had some other questions that could be of interest to people here:
- NLP is on a path to AGI: 58% agreed that Understanding the potential development of artificial general intelligence (AGI) and the benefits/risks associated with it should be a significant priority for NLP researchers.
- Related: 57% agreed that Recent developments in large-scale ML modeling (such as in language modeling and reinforcement learning) are significant steps toward the development of AGI.
- AGI could be revolutionary: 73% agreed that In this century, labor automation caused by advances in AI/ML could plausibly lead to economic restructuring and societal changes on at least the scale of the Industrial Revolution.
- AGI could be catastrophic: 36% agreed that It is plausible that decisions made by AI or machine learning systems could cause a catastrophe this century that is at least as bad as an all-out nuclear war.
- 46% of women and 53% of URM respondents agreed.
- The comments suggested that people took a pretty wide range of interpretations to this, including things like OOD robustness failures leading to weapons launches.
- Few scaling maximalists: 17% agreed that Given resources (i.e., compute and data) that could come to exist this century, scaled-up implementations of established existing techniques will be sufficient to practically solve any important real-world problem or application in NLP.
- The metasurvey responses predicted that 47% would agree to this, so there are fewer scaling maximalists than people expected there to be.
- Optimism about ideas from cognitive science: 61% agreed that It is likely that at least one of the five most-cited systems in 2030 will take clear inspiration from specific, non-trivial results from the last 50 years of research into linguistics or cognitive science.
- This strikes me as very optimistic, since it's pretty clearly false about the most cited systems today.
- Optimism about the field: 87% agreed that On net, NLP research continuing into the future will have a positive impact on the world.
- 32% of respondents who agreed that NLP will have a positive future impact on society also agreed that there is a plausible risk of global catastrophe.
- Most NLP research is crap: 67% agreed that A majority of the research being published in NLP is of dubious scientific value.
6 comments
Comments sorted by top scores.
comment by jungofthewon · 2022-09-01T16:05:40.453Z · LW(p) · GW(p)
This was really interesting, thanks for running and sharing! Overall this was a positive update for me.
Results are here
I think this just links to PhilPapers not your survey results?
Replies from: Evan R. Murphy, sbowman↑ comment by Evan R. Murphy · 2022-09-11T00:36:18.244Z · LW(p) · GW(p)
Can you say more about how this was a positive update for you?
Replies from: jungofthewon↑ comment by jungofthewon · 2022-09-12T00:03:46.907Z · LW(p) · GW(p)
Sure! Prior to this survey I would have thought:
- Fewer NLP researchers would have taken AGI seriously, identified understanding its risks as a significant priority, and considered it catastrophic.
- I particularly found it interesting that underrepresented researcher groups were more concerned (though less surprising in hindsight, especially considering the diversity of interpretations of catastrophe). I wonder how well the alignment community is doing with outreach to those groups.
- There were more scaling maximalists (like the survey respondents did)
I was also encouraged that the majority of people thought the majority of research is crap.
...Though not sure how that math exactly works out. Unless people are self-aware of their publishing crap :P
↑ comment by Sam Bowman (sbowman) · 2022-09-01T16:40:09.458Z · LW(p) · GW(p)
Thanks! Fixed link.
comment by Zoe Williams (GreyArea) · 2022-09-06T00:16:55.662Z · LW(p) · GW(p)
Super interesting, thanks!
If you were running it again, you might want to think about standardizing the wording of the questions - it varies from 'will / is' to 'is likely' to 'plausible' and this can make it hard to compare between questions. Plausible in particular is quite a fuzzy word, for some it might mean 1% or more, for others it might just mean it's not completely impossible / if a movie had that storyline, they'd be okay with it.
↑ comment by Sam Bowman (sbowman) · 2022-09-06T16:37:44.966Z · LW(p) · GW(p)
Fair. For better or worse, a lot of this variation came from piloting—we got a lot of nudges from pilot participants to move toward framings that were perceived as controversial or up for debate.