[LINK] The errors, insights and lessons of famous AI predictions

post by Stuart_Armstrong · 2014-04-28T09:41:40.218Z · LW · GW · Legacy · 9 comments

Contents

9 comments

The Journal of Experimental & Theoretical Artificial Intelligence has - finally! - published our paper "The errors, insights and lessons of famous AI predictions – and what they mean for the future":

Predicting the development of artificial intelligence (AI) is a difficult project – but a vital one, according to some analysts. AI predictions are already abound: but are they reliable? This paper starts by proposing a decomposition schema for classifying them. Then it constructs a variety of theoretical tools for analysing, judging and improving them. These tools are demonstrated by careful analysis of five famous AI predictions: the initial Dartmouth conference, Dreyfus's criticism of AI, Searle's Chinese room paper, Kurzweil's predictions in the Age of Spiritual Machines, and Omohundro's ‘AI drives’ paper. These case studies illustrate several important principles, such as the general overconfidence of experts, the superiority of models over expert judgement and the need for greater uncertainty in all types of predictions. The general reliability of expert judgement in AI timeline predictions is shown to be poor, a result that fits in with previous studies of expert competence.

The paper was written by me (Stuart Armstrong), Kaj Sotala and Seán S. Ó hÉigeartaigh, and is similar to the series of Less Wrong posts starting here and here.

9 comments

Comments sorted by top scores.

comment by [deleted] · 2014-04-29T01:13:30.421Z · LW(p) · GW(p)

Well done getting published in the Journal! That is an interesting read and it was nice to see the predictions graphed which really highlights the range of results.

In terms of the paywalled site, could you put a copy (or draft copy if there are copyright issues) on a more public site, like MIRI? I could read it through my work, but cant distribute as it is watermarked with employers name on each page.

Replies from: Stuart_Armstrong, Stuart_Armstrong
↑ comment by Stuart_Armstrong · 2014-04-29T09:22:32.738Z · LW(p) · GW(p)

Looking into this now...

comment by Kaj_Sotala · 2014-04-28T14:47:52.754Z · LW(p) · GW(p)

The paper was written by me (Stuart Armstrong), Kaj Sotala and Seán S. Ó hÉigeartaigh

That's very generous of Stuart to say, but in all honesty my name's mostly just there because the paper draws on the set of predictions that I helped classify. I didn't have very much to do with the actual writing of it.

Replies from: Stuart_Armstrong
↑ comment by Stuart_Armstrong · 2014-04-28T14:49:41.433Z · LW(p) · GW(p)

I remember having many useful conversations about the data and how to analyse it.

Replies from: Kaj_Sotala
↑ comment by Kaj_Sotala · 2014-04-28T14:57:23.604Z · LW(p) · GW(p)

Huh. I don't remember having said much that would have been particularly valuable for the stuff in this particular paper, but if you disagree, I guess I'll go along with that. :-)

comment by Shmi (shminux) · 2014-04-28T17:37:30.644Z · LW(p) · GW(p)

Paywalled :(

Replies from: Stuart_Armstrong, Sean_o_h
↑ comment by Sean_o_h · 2014-05-29T16:21:35.151Z · LW(p) · GW(p)

I'm happy to send a copy to anyone who messages me. Stuart, my understanding of the legalese is that we could put up the preprint/final draft pdf - would MIRI be willing to host it?