Changing accepted public opinion and Skynet

Your environmentalism examples raise another issue. What good is it convincing people of the importance of friendly AI if they respond with similarly ineffective actions? If widespread acceptance of the importance of the environment has led primarily to ineffective behaviours like unplugging phone chargers, washing and sorting containers for recycling and other activities of dubious benefit while at the same time achieving little with regards to reductions in CO2 emissions or slowing the destruction of rainforests then why should we expect widespread acceptance of the importance of friendly AI to actually aid in the development of friendly AI?

Other than donating to the singularity institute it is not even obvious to me what the average person could do to 'further the cause' if they were to accept its importance. There seems a fairly high chance that you would instead get useless or counter productive responses given widespread popular acceptance.

When Eliezer writes about the "miracle" of evolved morality, he reminds me of that bit from H.M.S. Pinafore where the singers are heaping praise on Rafe Rackstraw for being born an Englishman "despite all the temptations to belong to other nations". We can imagine that they might have sung quite a similar song in French.

One thing that might help change the opinion of people about friendly AI is to make some progress on it. For example, if Eliezer has had any interesting ideas about how to do it in the last five years of thinking about it, it could be helpful to communicate them.

A case that is credible to a large number of people needs to be made that this is a high-probability near-term problem. Without that it's just a scary sci-fi movie, and frankly there are scarier sci-fi movie concepts out there (e.g. bioterror). Making an analogy with a nuclear bomb is simply not an effective argument. People were not persuaded about global warming with a "greenhouse" analogy. That sort of thing creates a sort of dim level of awareness, but "AI might kill us" is not some new idea; everybody is already aware of that -- just like they are aware that a meteor might wipe us out, aliens might invade, or an engineered virus or new life form could kill us all. Which of those things get attention from policy-makers and their advisers, and why?

Besides the weakness of relying on analogy, this analogy isn't even all that good -- it takes concerted and advanced targeted technical dedication to make a nuclear FOOM fast enough to "explode". It's a reasonably simple matter to make it FOOM slowly and provide us with electrical power to enhance our standard of living.

If the message is "don't build Skynet", funding agencies will say "ok, we won't fund Skynet" and AI researchers will say "I'm not building Skynet". If somebody is working on a dangerous project, name names and point fingers.

GIve a chain of reasoning. If some of you rationalists have concluded a significant probability of an AI FOOM coming soon, all you have to do is explicate the reasoning and probabilities involved. If your conclusion is justified, if your ratiocination is sound, you must be able to explicate it in a convincing way, or else how are you so confident in it?

This isn't really an "awareness" issue -- because it's scary and in some sense reasonable it makes a great story, thus hour after hour of TV, movie blockbusters stretching back through decades, novel after novel after novel.

Make a convincing case and people will start to be convinced by it. I know you think you have already, but you haven't.

A couple things come to mind. The first is that we have to figure out how much we value attention from people that are not initially rational about it (as this determines how much Dark Art to use). I can see the extra publicity as helping, but if it gets the cause associated with "nut jobs", then it may pass under the radar of the rational folk and do more harm than good.

The other thing that comes to mind is that this looks like an example of people using "far" reasoning. Learning how to get people to analyze the situation in "near" mode seems like it would be a very valuable tool in general (if anyone has any ideas, make a top level post!)

To give a couple examples, I recently talked to my roommate about the risks of AI, and on the surface he agreed it was a big deal. However, he didn't make the connection "maybe cancer fundraisers aren't the best way to spend my charity time", and I don't think he'll actually do anything differently.

I talked to another friend about the same thing, and it scared the living crap out of him. He asked "How can you go on living like normal instead of working to fix it!?". So far, so good. He looked like he was using the "near" method of thinking about it.

The catch though is that his conclusion was "Since there's a small probability of me making the difference, I'd prefer to 'stick my head in the sand' and forget I heard this.". This might actually be the rational response for someone that 1) doesn't care about more than a small group of people, and 2) defects on true PD.

To recruit people like this it seems like we'd need to turn it into an iterated prisoners dilemma. If you caught as much flak for not donating to the FAI cause as you do for not recycling, then a lot more people would donate at least something.

If "AI will be dangerous to the world" became a socially accepted factoid you would get it spilling over in all sorts of unintended fashions. It might not be socially acceptable to use Wolfram Alpha as it is too AI-ish,

The simplest way to change public opinion is manually. Skynet seems like an adequate solution to me.

The biggest problem with the movies, besides the inconsistencies as to whether causality is changeable or not, is why Skynet bothers dealing with the humans once it's broken their ability to prevent it from launching itself into space. Sending a single self-replicating seed factory to the Moon is what a reasonable AI would do.

The Terminator movies exploit the primal human fear of being exterminated by a rival tribe, putting AI in the role once filled by extraterrestrials: outsiders with great power who want to destroy all of 'us'. The pattern is tedious and predictable.

As William has pointed out, AI running amok is already a standard trope. In fact, Asimov invented his three laws way back when as a way of getting past the cliche, and writing stories where it wasn't a given that the machine would turn on its creator. But the cliche is still alive and well. Asimov himself had the robots taking over in the end, in "That Thou Art Mindful of Him" and the prequels to the "Foundation" trilogy.

The people that the world needs to take FAI seriously are the people working on AI. That's what, thousands at the most? And surely they have all heard of the issue by now. What is their view on it?

As people's preferences and decisions are concerned, there are trusted tools, trusted analysis methods. People use them, because they know indirectly that their output will be better than what their intuitive gut feeling outputs. And thus, people use complicated engineering practices to build bridges, instead of just drawing a concept and proceeding to fill the image with building material.

But these tools are rarely used to refactor people's minds. People may accept conclusions chosen by experts, and allow them to install policies, as they know this is a prudent thing to do, but at the same time they may refuse to accept the fundamental assumptions on which the experts' advice is based. This is non-technical perspective, Pirsig's "romantic" attitude.

The only tools that get adopted are those that demonstrate practical results, independently of their internal construction. Adoption of new tools doesn't change minds, and thus doesn't allow to adopt new tools that were refused before. This is a slower process, driven more by social norm than support by the tool of reason, too technical to become part of people's minds, to help in their decisions.

Argument will only convince smart people who are strong enough in rationality to allow their reason to reshape their attitudes. For others, the tools of blind social norm and dark arts are the only option, where practical effect appears only in the endgame.

My sense is that most people aren't concerned about Skynet for the same reason that they're not concerned about robots, zombies, pirates, faeries, aliens, dragons, and ninjas. (Homework: which of those are things to worry about, and why/why not?)

Also, this article could do without the rant against environmentalism and your roommate. Examples are useful to understanding one's main point, but this article seems to be overwhelmed by its sole example.

You could probably find a clearer title. Naming an article after an example doesn't seem like a good idea to me. Probably the topic changed while you were writing and you didn't notice. (I claim that it is a coherent essay on public opinion.)

Yes, it is important to know how public opinion changes. But before you try to influence it, you should have a good idea of what you're trying to accomplish and whether it's possible. Recycling and unplugging gadgets are daily activities. That continuity is important to making them popular. Is it possible to make insulating houses fashionable?

Blah, my name is "Anissimov", please spell it correctly!

"The Internet" is probably an interesting case study. It has grown from a very small niche product into a "fundamental right" in a relatively short time. One of the things that probably helped this shift is showing people what the internet could do for them - it became useful. This is understandably a difficult point on which to sell FAI.

Now that that surface analogy is over, how about the teleological analogy? In a way, environmentalism assumes the same mantle as FAI - "do it for the children". Environmentalism has plenty of benefits over FAI - it has fuzzier mascots and more eminent problems - Terminators aren't attacking, but more and more species are becoming extinct.

Environmentalism is still of interest here through the subtopic of climate change. Climate change already deals with some of the problems existential risk at large deals with - its veracity is argued, its importance is disputed, and the math is poorly understood. The next generation serves as a nice fuzzy mascot and the danger is of the dramatically helpful ever inexorably closer variety. Each day you don't recycle, the earth is in more danger, &c. (The greater benefit of a creeping death, "zombie" danger may be that it negates the need for a mathematical understanding of the problem. It becomes "obvious" that the danger is real if it gets closer everyday.)

How can you convince people to solve a harder problem once, rather than every problem that crops up?

One central problem is that people are constantly deluged with information about incipient crises. The Typical Person can not be expected to understand the difference in risk levels indicated by UFAI vs. bioterror vs. thermonuclear war vs global warming, and this is not even a disparagement of the Typical Person. These risks are just impossible to estimate.

But how can we deal with this multitude of potential disasters? Each disaster has some low probability of occurring, but because there are so many of them (swine flu, nuclear EMP attacks, grey goo, complexity... ) we are almost certainly doomed, unless we do something clever. Even if we take preventative measures sufficient to eliminate the risk of one problem (presumably at some enormous expense), we will just get smashed by the next one on the list.

Meta-strategy: find strategies that help defend against all sources of existential risk simultaneously. Candidates:

• moon base
• genetic engineering of humans to be smarter and more disease-resistant
• generic civilizational upgrades, e.g. reducing traffic and improving the economy
• simplification. There is no fundamental reason why complexity always has to increase. Almost everything can be simplified: the law, the economy, software.

I agree that an Unfriendly AI could be a complete disaster for the human race. However, I really don't expect to see an AI that goes FOOM during my lifetime. To be frank, I think I'm far more likely to be killed by a civilization-threatening natural disaster, such as an asteroid impact, supervolcano eruption, or megatsunami, than by an Unfriendly AI. As far as I'm concerned, worrying about Unfriendly AI today is like worrying about global warming in 1862, shortly after people began producing fuel from petroleum. Yes, it's a real problem that will have to be solved - but the people alive today aren't going to be the ones that solve it.

Why is this post being voted negative? It's an important problem for plenty of causes of interest to many rationalists, and is well worth discussing here.