Overconfident Pessimism

How can you say that people are overconfident if you don't count the pessimistic predictions that turned out to be true?

Role-acting that conflates lack of info with skepticism. If you're acting out the role of a dutifully skeptical scientist, you say that things are a long way away unless you have strong info that they're about to happen, for fear of making a mistaken prediction that makes you sound like a mere enthusiast. Failed negative predictions don't count against the role - they don't reduce your prestige to that of a mere enthusiast. Just imagine it as a serious frown vs. a childish smile; they're trying to give a serious frown, and avoid childish smiles. You're dealing with a behavioral rule for when to agree with verbal statements, not a coherent state of uncertainty. Both "we know very little about the future" and "AI is fifty years off" sound like a serious frown, so they agree with both.

I doubt there's very much more to it than that.

A premortem might be a useful technique to use to counteract overconfident pessimism. It's another variant on "consider the opposite", discussed in Kahneman's Thinking, Fast and Slow and this LW post (among other places). With standard overconfidence (e.g. "our project will succeed"), you do a premortem by saying "it's two months from now and the project has failed; why did it fail?" Suddenly your brain can generate various pathways to failure which had been hidden.

With overconfident pessimism ("there is no way that we'll develop that technology in the next 50 years"), you'd do a premortem by saying "it's 20 years from now and we have that technology; how did we get it?"

Can we compile better lists of false predictions?

With some of them, it's not clear whether they were even intended as predictions; they may have been intended as statements about the technology at the time. Taken in that sense, they may have been true:

"I think there is a world market for maybe five computers." -- Thomas Watson, chairman of IBM, 1943.

Was there much demand for computers as they existed in 1943?

There's also the problem that some of the claims may not have been intended as claims about what will happen, but about what should happen. For example, the Darwin quote:

"I see no good reasons why the views given in this volume should shock the religious sensibilities of anyone." -- Charles Darwin, The Origin Of Species, 1869.

That doesn't entail people's religious sensibilities won't be shocked for bad reasons.

Also, these lists don't cite their sources very well.

How do we know that calibration training will improve our calibration on long-term predictions? It would seem that we necessarily have little evidence about the efficacy of short-term calibration training on calibrating long-term predictions.

I've been uncomfortable for a while with statements like Eliezer's remark that:

When heavier-than-air flight or atomic energy was a hundred years off, it looked fifty years off or impossible; when it was five years off, it still looked fifty years off or impossible.

This really is picking and choosing specific technological examples rather than looking at the overall pattern. In 1964, five years before the first moon landing, it looked a few years off but certainly not a hundred years off.

Perhaps the best online tool for calibration training is PredictionBook.com

I strongly agree with this. I've used it to make a variety of predictions, including tech predictions. One issue it does have is that there's no easy categorization so one can't use it for example to see at a glance whether one's tech predictions are more or less accurate than one's predictions about politics or other subjects.

Mathematicians seldom try and predict when major problems will be solved, because they recognise that insight is very hard to predict.

Noteworthy counterexample: Soon after the Feit-Thompson theorem, people started talking about classifying all finite simple groups, but this was because Gorenstein had a specific blueprint that was thought might be able to get the full result. But even then, the time period was shorter.

In cases like the Riemann hypothesis we have a few ideas of things that might work, but none look that promising, and results one would expect to fall first, like the Lindelof hypothesis remain apparently unassailable. So one major sign of a problem being genuinely far off is that even to our eyes, much simpler problems look far off. I'm not sure how to apply that to AI. Do modern practical successes like machine learning count plausibly as successes of related minor aspects? It will be a lot easier to tell after there's some form of general AI and we have more of an idea about its structure. Similar issues apply to almost any future tech.

It don't think it's just a matter of pessimism -- the phenomenon whereby “I cannot predict X” somehow becomes “X is not going to happen” can also happen when X is a bad thing (e.g. “there will be a major earthquake in L'Aquila in the next few days”).

EDIT: If you've heard the story about the convicted scientists, please read the article linked to, as most news coverage seriously distorted the story.

The examples make the point that it's possible to be too pessimistic, and too confident in that pessimism. However, maybe we can figure out when we should be confidently pessimistic.

For example, we can be very confidently pessimistic about the prospects for squaring the circle or inventing perpetual motion. Here we have mathematical proofs of impossibility. I think we can be almost as confidently pessimistic about the near-term prospects for practical near-light-speed travel. Here we have a good understanding of the scope of the problem and of the capabilities of all practical sources of propulsion, and we can see that those capabilities are nowhere near enough.

Let's not just leave it at "it's possible to be too pessimistic." How can we identify problems about which we can be confidently pessimistic?

I'm not sure how comparable AI and nuclear fusion are. AI is clearly a case where new insights are needed, but fusion -- to the best of my knowledge, anyway -- might be obtainable via sufficient grinding. So while certainly people are still looking for new insights into nuclear fusion, one could perhaps predict how long it would obtain fusion by pure grinding. And if you expect no new insights, you would consider this a good estimate -- although as you point out, expecting no new insights is probably pessimistic.

Not directly related to the topic, but since you're speaking of PredictionBook, there is a question I would like to ask : it seems from http://predictionbook.com/predictions that the PredictionBook crowd is mostly calibrated, on average, except on the extrema (100%/0%). How does that match with the "people are broadly overconfident" studies ? The two dataset seem contradictory to me. I notice I'm confused.

I could pop explanations like "people on prediction book are not representatives of people in general" or "the kind of predictions made on prediction book isn't the same" but they sound more like rationalizations (popping an explanation with poor data backing it to avoid admitting confusion), so I don't accept them.

Does anyone here has better answers (or data back to my "guesses") on that data contradiction ?

We still don't have economical flying cars. :(

(Doing something cheaply and a million times is a lot harder than doing it expensively and once.)

Now I'm wondering how this kind of bias operates outside of science, and specifically with what confidence we can expect insane things to be disregarded.

More in detail, I'm wondering how long homeopathy can survive while all experts can attest that it's not useful. The case of Miracle Mineral Supplement which Eliezer mentioned recently, seems to show that people will stop doing absurd things, when it is shown exactly how absurd they are. The question is, how long does it take for this to happen? After all, people still read horoscopes!

A virtuous doubt that applies to us:

Doubt that LessWrong is gifted.

Here, I consider the opposite.

I realize this isn't a prediction that people are overconfident and pessimistic about, but these are such great concepts, I would like to apply them everywhere.

In addition to my experiences where LessWrongers have been surprisingly overconfident and pessimistic about IQ claims, Yvain has also made an observation in the 2012 survey with "some people have been pretty quick to ridicule this survey's intelligence numbers as completely useless and impossible and so on". It's not just me who is encountering this, so I wonder if this might be common mistake that LessWrongers make.

Some pessimists expect a linear grown in sofistication, with a few insights along the way, but not sequential insights from one single group. Safe AI designs are more hard than unsafe ones. Normally hard problems are solved after the easy, and by different people. If FAI is a hard-rather-than-easy project, the results will appear after the unsafe AGIs. However, this could not be the case, if AI research change.

It's probably also worth noting that widely-known and accepted prediction will, in turn, have an influence on what they predict. When a physicist has an idea for a very expensive experiment that may lead to nuclear fusion and has to get funding, he's more likely to get funding from an agency that feels that nuclear fusion is 'just around the corner' than from one that feels that nuclear fusion is 'impossible within the next fifty years'.

And before that, of course, the physicist himself has to decide in which direction he will take his research. The R&D departments of commercial enterprises will usually want to direct their research towards outcomes that are likely to result in something that can be solved, and therefore sold, reasonably soon. Therefore, most commercial enterprises are unlikely to run across any unexpected insights.