After critical event W happens, they still won't believe you

For big jumpy events, look at the reactions to nuclear chain reactions, Sputnik, ENIGMA, penicillin, the Wright brothers, polio vaccine...

Then consider the process of gradual change with respect to the Internet, solar power, crop yields...

I do tend to think that Aubrey de Grey's argument holds some water. That is, it's not so much general society that will be influenced as wealthy elites. Elites seem more likely to update when they read about a 2x mouse. I suppose the Less Wrong response to this argument would be: how many of them are signed up for cryonics? But cryonics is a lot harder to believe than life extension. You need to buy pattern identity theory and nanotechnology and Hanson's value of life calculations. In the case of LE, all you have to believe is that the techniques that worked on the mouse will, likely, be useful in treating human senescence. And anyway, Aubrey hopes to first convince the gerontology community and then the public at large. This approach has worked for climate science and a similar approach may work for AI risk.

I don't find either example convincing about the general point. Since I'm stupid I'll fail to spot that the mouse example uses fictional evidence and is best ignored

We are all pretty sick of seeing a headline "Cure for Alzheimer's disease!!!" and clicking through to the article only to find that it is cured in mice, knock-out mice, with a missing gene, and therefore suffering from a disease a little like human Alzheimer. The treatment turns out to be injecting them with the protein that the missing gene codes for. Relevance to human health: zero.

Mice are very short lived. We expect big boosts in life span by invoking mechanisms already present in humans and already working to provide humans with much longer life spans than mice. We don't expect big boosts in the life span of mice to herald very much for human health. Cats would be different. If pet cats started living 34 years instead of 17, their owners would certainly be saying "I want what Felix is getting."

The sophistication of AI is a tricky thing to measure. I think that we are safe from unfriendly AI for a few years yet, not so much because humans suck at programming computers, but because they suck in a particular way. Some humans can sit at a keyboard typing in hundreds of thousands of lines of code specific to a particular challenge and achieve great things. We can call that sophistication if we like, but it isn't going to go foom. The next big challenge requires a repeat of the heroic efforts, and generates another big pile of worn out keyboards. We suck at programming in the sense that we need to spend years typing in the code ourselves, we cannot write code that writes code.

Original visions of AI imagined a positronic brain in an anthropomorphic body. The robot could drive a car, play a violin, cook dinner, and beat you at chess. It was general purpose.

If one saw the distinction between special purpose and general purpose as the key issue, one might wonder: what would failure look like? I think the original vision would fail if one had separate robots, one for driving cars and flying airplanes, a second for playing musical instruments, a third to cook and clean, and fourth to play games such as chess, bridge, and baduk.

We have separate hand-crafted computer programs for chess and bridge and baduk. That is worse than failure.

Examples the other way.

After the Wright brothers people did believe in powered, heavier-than-air flight. Aircraft really took of after that. One crappy little hop in the most favourable weather and suddenly every-one's a believer.

Sputnik. Dreamers had been building rockets since the 1930's, and being laughed at. The German V2 was no laughing matter, but it was designed to crash into the ground and destroy things, which put an ugh field around thinking about what it meant. Then comes 1957. Beep, beep, beep! Suddenly every-one's a believer and twelve years later Buzz Aldrin and the other guy are standing on the moon :-)

The Battle of Cambrai is two examples of people "getting it". First people understood before the end of 1914 that the day of the horse-mounted cavalry charge was over. The Hussites had war wagons in 1420 so there was a long history of rejecting that kind of technology. But after event W1 (machine guns and barbed wire defeating horses) it only took three years before the first tank-mounted cavalry charge. I think we tend to miss understand this by measuring time in lives lost rather than in years. Yes, the adoption of armoured tanks was very slow if you count the delay in lives, but it couldn't have come much faster in months.

The second point is that first world war tanks were crap. The Cambrai salient was abandoned. The tanks were slow and always broke down, because they were too heavy and yet the armour was merely bullet proof. There only protection against artillery was that the gun laying techniques of the time were ill suited to moving targets. The deployment of tanks in the first world war fall short of being the critical event W. One expects the horrors of trench warfare to fade and military doctrine to go back to horses and charges in brightly coloured uniforms.

In reality the disappointing performance of the tanks didn't cause military thinkers to miss their significance. Governments did believe and developed doctrines of Blitzkreig and Cruiser tanks. Even a weak W can turn every-one into believers.

As far as I know, people have predicted every single big economic impact from technology well in advance, in the strong sense of making appropriate plans, making indicative utterances, etc. (I was claiming a few year's warning in the piece you are responding to, which is pretty minimal). Do you think there are counterexamples? You are claiming that a completely unprecedented will happen with very high probability. If you don't think that requires strong arguments to justify than I am confused, and if you think you've provided strong arguments I'm confused too.

I agree that AI has the potential to develop extremely quickly, in a way that only a handful of other technologies did. As far as I can tell the best reason to suspect that AI might be a surprise is that it is possible that only theoretical insights are needed, and we do have empirical evidence that sometimes people will be blindsided by a new mathematical proof. But again, as far as I know that has never resulted in a surprising economic impact, not even a modest one (and even in the domain of proofs, most of them don't blindside people, and there are strong arguments that AI is a harder problem than the problems that one person solves in isolation from the community---for example, literally thousands of times more effort has been put into it). A priori you might say "well, writing better conceptual algorithms is basically the same as proofs---and also sometimes blindsides people---and the total economic value of algorithms is massive at this point, so surely we would sometimes see huge jumps" but as far as I know you would be wrong.

There seems to be a big gap between the sort of problem on which progress is rapid and surprising, and the sort of problem on which progress would have an economic impact. There are a number of reasons to suspect this a priori (lots of people work on economically relevant problems, lots of people try to pay attention to development in those areas because it actually matters, economically relevant problems tend to have lots of moving pieces and require lots of work to get right, lots of people create working intermediate versions because those tend to also have economic impact, etc. etc.) and it seems to be an extremely strong empirical trend.

Like I said, I agree that AI has the potential to develop surprisingly quickly. I would say that 10% is a reasonable probability for such a surprising development (we have seen only a few cases of tech developments which could plausibly have rapid scale-up in economic significance; we also have empirical evidence from the nature of the relationship between theoretical progress and practical progress on software performance). This is a huge deal and something that people don't take nearly seriously enough. But your position on this question seems perplexing, and it doesn't seem surprising to me that most AI researchers dismiss it (and most other serious observers follow their lead, since your claim appears to be resting on a detailed view about the nature of AI, and it seems reasonable to believe people who have done serious work on AI when trying to evaluate such claims).

Making clear arguments for a more moderate and defensible conclusions seems like a good idea, and the sort of thing that would probably cause reasonable AI researchers to take the scenario more seriously.

You mention Deep Blue beating Kasparov. This sounds look a good test case. I know that there were times when it was very controversial whether computers would ever be able to beat humans in chess - Wikipedia gives the example of a 1960s MIT professor who claimed that "no computer program could defeat even a 10-year-old child at chess". And it seems to me that by the time Deep Blue beat Kasparov, most people in the know agreed it would happen someday even if they didn't think Deep Blue itself would be the winner. A quick Google search doesn't pull up enough data to allow me to craft a full narrative of "people gradually became more and more willing to believe computers could beat grand masters with each incremental advance in chess technology", but it seems like the sort of thing that probably happened.

I think the economics example is a poor analogy, because it's a question about laws and not a question of gradual creeping recognition of a new technology. It also ignores one of the most important factors at play here - the recategorization of genres from "science fiction nerdery" to "something that will happen eventually" to "something that might happen in my lifetime and I should prepare for it."

Example 1: "After a 2-year-old mouse is rejuvenated to allow 3 years of additional life, society will realize that human rejuvenation is possible, turn against deathism as the prospect of lifespan / healthspan extension starts to seem real, and demand a huge Manhattan Project to get it done."

A quick and dirty Google search reveals:

Cost of Manhattan Project in 2012 dollars: 30 billion

Pharma R&D budget in 2012: 70 billion

http://www.fiercebiotech.com/special-reports/biopharmas-top-rd-spenders-2012

http://nuclearsecrecy.com/blog/2013/05/17/the-price-of-the-manhattan-project/

My version of Example 2 sounds more like "at some point, Watson might badly misdiagnose a human patient, or a bunch of self-driving cars might cause a terrible accident, or more inscrutable algorithms will do more inscrutable things, and this sort of thing might cause public opinion to turn against AI entirely in the same way that it turned against nuclear power."

In general and across all instances I can think of so far, I do not agree with the part of your futurological forecast in which you reason, "After event W happens, everyone will see the truth of proposition X, leading them to endorse Y and agree with me about policy decision Z."

Sir Karl Popper came to the same conclusion in his 1963 book Conjectures and Refutations. So did Harold Walsby in his 1947 book The Domain of Ideologies. You're in good company.

At the Edge question 2009 6 people spoke of immortality (de Grey not included). and 17 people spoke of superintelligence/humans 2.0.

This seems like evidence for Aubrey's point of view.

Of all the things that 151 top scientists could think they'd live to see, that more than 10% converged on that without previous communication in that stuff is perplexing for anyone who was a transhumanist in 2005.

I agree that almost no actual individual will change his or her mind. But humanity as a whole can change its mind, as young impressionable scientists look around, take in the available evidence, and then commit themselves to a position and career trajectory based on that evidence.

I do think there is a lot of truth to that. Reminds me of the people who said in the 1990's "Well, as soon as the arctic ice cap start to melt, then the climate deniers will admit that climate change is real", but of course that haven't happened either.

I do wonder, though, if that's equally true for all of those fields. For example, in terms of anti-aging technology, it seems to me that the whole status quo is driven by a very deep and fundamental sense that aging is basically unchangeable, and that that's the only thing that makes it acceptable to people, and I would suspect that anything that starts to change or threaten that perception even a little bit could cause radical alterations to how people act.

AI, though, is a field where it's especially hard for a lay people to get an idea of either how fast things are advancing or how much farther they have to go before a human-level GAI becomes possible; everyone is used to a constant stream of "cool new things" coming out of the tech fields, but it's much harder to get an idea of what the larger picture is. I also think that technology news generally isn't covered very well in the media, which makes it even harder. Fundamentally, I think that most people do understand that computers are a transformative technology, it's just that they think that the computer revolution has already mostly happened.

"When the author of the original data admits that he fabricated it, people will stop believing that vaccines cause autism."

Funny that I just hit this with my comment from yesterday:

http://lesswrong.com/lw/hoz/do_earths_with_slower_economic_growth_have_a/95hm

Idea: robots can be visibly unfriendly without being anywhere near a FOOM. This will help promote awareness.

I think this is different from your examples.

A mouse living to the ripe old age of 5? Well, everyone has precedent for animals living that long, and we also have precedent for medicine doing all sorts of amazing things for the health of mice that seem never to translate into treatments for people.

Economic meltdowns? Economics is so political that you expect the populace to update on evidence? You might as well expect them to update on evidence in respect to religion.

Meanwhile, we are so very primed to suspect the worst of computers.

Huh, my first thought on seeing the title was that this would be about Richard Stallman.

I wonder how a prolonged event Y, something with enough marketability to capture the public's eye for some time, might change opinions on the truth of proposition X. Something along the lines of the calculus, Gutenberg's printing press, the advent of manoeuvrable cannons, flintlocks, quantum mechanics, electric stoves (harnessed electricity), the concept of a national debt, etcetera.

I'd be interested in what if any effects the American National Security Agency scandal, or a worldwide marketing campaign by Finland advertising their healthcare system will or would have to this end.

Presumably the best you can do solution-wise is to try and move policy options through a series of "middle stages" towards either optimal results, or more likely the best result you can realistically get?

EDIT: Also- how DID the economists figure it out anyway? I would have thought that although circumstances can incease or reduce it inflationary effects would be inevitable if you increased the money supply that much.

The two examples here seem to not have alarming/obvious enough Ws. It seems like you are arguing against a straw-man who makes bad predictions, based on something like a typical mind fallacy.

Making a mouse live for 5 years isn't going to get anyone's attention. When they can make a housecat live to be 60, then we'll talk.

What would be examples of a critical event in AI?

Exactly. This is part of the reason I will win the bet, i.e. it is the reason the first super intelligent AI will be programmed without attention to Friendliness.

if you look at what happens when Congresscritters question Bernanke, you will find that they are all terribly, terribly concerned about inflation

I imagine that Congress is full of busy people who don't have time to follow blogs that cover every topic they debate. Did they get any testimony from expert economists when questioning Bernanke?

"After event W happens, everyone will see the truth of proposition X, leading them to endorse Y and agree with me about policy decision Z."

Isn't this just a standard application of Bayesianism? I.e. after event W happens, people will consider proposition X to be somewhat more likely, thereby making them more favorable to Y and Z. The stronger evidence event W is, the more people will update and the further they will update. But no one piece of evidence is likely to totally convince everyone immediately, nor should it.

For instance, if "a 2-year-old mouse is rejuvenated to allow 3 years of additional life" that's some evidence for lifespan extension and I would update accordingly, but not all the way to infinite life extension is obviously possible and worthy of reshaping the economy around. If a two-year old mouse trained in a maze is rejuvenated and still remembers the maze it was trained in, that's much stronger evidence for useful lifespan extension, and I start to think maybe we should fund a Manhattan Project for human rejuvenation. If a 20-year-old signing chimpanzee is rejuvenated to allow 30 years of additional life, and still remembers how to sign, and perhaps can tell us facts from its previous life, that's really strong evidence. (Though that might actually require language skills beyond those of a non-rejuvenated signing chimpanzee.)

On the other hand, if the two-year old mouse is rejuvenated but seems to have forgotten the maze, or the chimpanzee is rejuvenated, but not only has forgotten how to sign, but is demonstrably mentally handicapped relative to other chimpanzees, that's actually evidence against life extension.

The wheel of progress turns slowly but it does turn. Expecting no one to update at all on new evidence is as equally wrong as expecting everyone to update from 0 to 1 overnight. After event W happens, most scientists who know what W means, young and old, will increase their estimate of the truth of proposition X; and after event W2, and then W3, happen they'll update still more. I've seen this happen repeatedly over my lifetime. For instance, in the last 20 years pretty much all astrophysicists have come to believe in a positive cosmological constant, dark matter, and dark energy, as implausible as those ideas sounded 40 years ago, because the evidence has piled up one observation at a time. Certainly some astrophysicists have retired and died and been replaced in those 20 years; but absolutely those scientists remaining have changed their minds. The rest of society will follow along as soon as it becomes important for them to do so. In astrophysics that day of relevance may never come, but in more down-to-earth subjects it may take years but not a lifetime's worth of years.

I'd missed the mouse "rejuvenation for 3 more years of life" result (did you mean cryo freeze -> revive, or something else?). Could you supply a cite?

This is the very definition of the status quo bias.