Top 9+2 myths about AI risk

Three more myths, from Luke Muehlhauser:

• We don’t think AI progress is “exponential,” nor that human-level AI is likely ~20 years away.
• We don’t think AIs will want to wipe us out. Rather, we worry they’ll wipe us out because that is the most effective way to satisfy almost any possible goal function one could have.
• AI self-improvement and protection against external modification isn’t just one of many scenarios. Like resource acquisition, self-improvement and protection against external modification are useful for the satisfaction of almost any final goal function.

A similar list by Rob Bensinger:

• Worrying about AGI means worrying about narrow AI
• Worrying about AGI means being confident it’s near
• Worrying about AGI means worrying about “malevolent” AI

I'm not sure I agree with 9. There is a lot of SF out there, and some of it (Roadside Picnic, Stanislaw Lem's works) presents the idea that the universe is inherently uncaring in a very real way. Anthropomorphization is not an inherent feature of the genre, and fiction might enable directly understanding these ideas in a way that argument doesn't make real for some people.

I'm not a Friendliness researcher, but I did once consider whether trying to slow down AI research might be a good idea. Current thinking is probably not, but only because we're forced to live in a third-best world:

First best: Do AI research until just before we're ready to create an AGI. Either Friendliness is already solved by then, or else everyone stop and wait until Friendliness is solved.

Second best: Friendliness looks a lot harder than AGI, and we can't expect everyone to resist the temptation of fame and fortune when the possibility of creating AGI is staring them in the face. So stop or slow down AI research now.

Third best: Don't try to stop or slow down AI research because we don't know how to do it effectively, and doing it ineffectively will just antagonize AI researchers and create PR problems.

There are some people, who honestly think Friendliness-researchers in MIRI and other places actually discourage AI research. Which sounds to me ridiculous, I've never seen such attitude from Friendliness-researchers, nor can even imagine that.

Why is this so ridiculous as to be unimaginable? Isn't the second-best world above actually better than the third-best, if only it was feasible?

That we want to stop AI research. We don’t. Current AI research is very far from the risky areas and abilities. And it’s risk aware AI researchers that are most likely to figure out how to make safe AI.

Is it really the case that nobody interested in AI risk/safety wants to stop or slow down progress in AI research? It seemed to me there was perhaps at least substantial minority that wanted to do this, to buy time.

Nice. This would probably be useful to have near the end of a primer on AI risk, perhaps as a summary of sorts.

I can only talk about those I've interacted with, and I haven't seen AI research blocking being discussed as a viable option.

Given the speed of AI development in other countries, do we know if any of the work on friendly AI is being translated or implemented outside of the US? Or what the level of awareness of AI friendliness issues among AI researchers in non-English speaking countries?

(I realize that IQ isn't an effective test of AI, but this is the article that prompted me wondering: http://www.businessinsider.com/chinese-ai-beat-humans-in-an-iq-test-2015-6. )

I have my own idea of "one simple trick". Might as well solicit opinions here: consider the possibility that developing tools for providing useful summaries of and agent's thinking could substantially lower the risk associated with AGI. If there's an effective way to place trip-wires in the agent's mind to inform us of its thought-crimes, we would know to ignore its protestations and could just pull the plug on it. Or better yet, have the plug be automatically pulled by the monitoring process. Perhaps an agent be able to meditate it's way around our safeguards, so as to shield itself from Jedi mind-tricks?

Certainly a good compilation! It might be even more useful, though, if it contained references to research papers, Bostrom's superintelligence etc., where the arguments are discussed in full detail.

As an example of number 10, consider the Optimalverse. The friendliest death of self-determination I ever did see.

Unfortunately, I'm not quite sure of the point of this post, considering you're posting a reply to news articles on a forum filled with people who understand the mistakes they made in the first place. Perhaps as a repository of rebuttals to common misconceptions posited in the future?

Argument for a powerful AI being unlikely - was this considered before?

One problem I see here is the "lone hero inventor" implicit assumption, namely that there are people optimizing things for their goals on their own and an AI could be extremely powerful at this. I would like to propose a different model.

This model would be that intelligence is primarily a social, communication skill, it is the skill to disassemble (understand, lat. intelligo), play with and reassemble ideas acquired from other people. Like literally what we are doing on this forum. It is conversational. The whole standing on the shoulder of giants thing, not the lone hero thing.

In this model, inventions are made by the whole of humankind, a network, where each brain is a node communicating slightly modified ideas to each other.

In such a network one 10000 IQ node does not get very powerful, it doesn't even make the network very powerful i.e. a friendly AI does not quickly solve mortality even with human help.

The primary reason I think such a model is correct that intelligence means thinking, we think in concepts, and concepts are not really nailed down but they are constantly modified through a social communication process. Atoms used to mean indivisible units, then they became divisible into little ping-pong balls, and then the model was updated into something entirely different by quantum physics, but is quantum physics based atom theory about the same atoms that were once thought to be indivisible or is this a different thing now? Is modern atomic theory still about atoms? What are we even mapping here and where does the map end and the territory begin?

So the point is human knowledge is increased by a social communication process where we keep throwing bleggs at each other, and keep redefining what bleggs and rubes mean now, keep juggling these concept, keep asking what you really mean under bleggs, and so on. Intelligence is this communication ability, it is to disassemble Joe's concept of bleggs and understand how it differs from Jane's concept of bleggs and maybe assemble a new concept that describes both bleggs.

Without this communication, what would be even intelligence? What would lone intelligence be? It is almost a contradictory term in itself. What would a brain alone in a universe intelligere i.e. understand if nothing would talk to it? Just tinker with matter somehow without any communication whatsoever? But even if we imagine such an "idiot inventor genius", some kind of a mega-plumber on steroids instead of an intellectual or academic it needs goals for that kind of tinkering with that material stuff, for that it needs concepts, and concepts come and evolve from a constant social ping-pong.

An AI would be yet another node in our network, and participate in this process of throwing blegg-concepts at each other probably far better than any human can, but still just a node.

I recently gave a talk at an academic science fiction conference about whether sf is useful for thinking about the ethics of cognitive enhancement. I think some of the conclusions are applicable to point 9 too:

(1) Bioethics can work in a "prophetic" and a "regulatory" mode. The first is big picture, proactive and open-ended, dealing with the overall aims we ought to have, possibilities, and values. It is open for speculation. The regulatory mode is about ethical governance of current or near-term practices. Ethicists formulate guidelines, point out problems, and suggest reforms, but their purpose is generally not to rethink these practices from the ground-up or to question the wisdom of the whole enterprise. As the debate about the role of speculative bioethics has shown, mixing the modes can be problematic. (Guyer and Moreno 2004) really takes bioethics to task for using science fiction instead of science to motivate arguments: they point out that this can actually be good if one does it inside the prophetic mode, but a lot of bioethics (like the President's Council on Bioethics at the time) cannot decide what kind of consideration it is.

(2) Is it possible to find out things about the world by making stuff up? (Elgin 2014) argues fictions and thought experiments do exemplify patterns or properties that they share with phenomena in the real world, and hence we can learn something about the realized world from considering fictional worlds (i.e. there is a homeomorphism between them in some domain). It does require the fiction to be imaginative but not lawless: not every fiction or thought experiment has value in telling us something the real or moral world. This is of course why just picking a good or famous piece of fiction as a source of ideas is risky: it was selected not for how it reflects patterns in the real world, but for other reasons.

Considering Eliezer's levels of intelligence in fictional characters is a nice illustration of this: level 1 intelligence characters show some patterns (being goal directed agents) that matter, and level 3 characters actually give examples of rational skilled cognition.

(3) Putting this together, if you want to use fiction in your argument, the argument better be in the more prophetic, open-ended mode (e.g. arguing that there is AI risks of various kind, what values are at stake etc.) and the fiction needs to have pretty not only high standards of not just internal consistency but actual mappability to the real world. If the discussion is on the more regulatory side (e.g. thinking of actual safeguards, risk assessment, institutional strategies) then fiction is unlikely to be helpful, and very likely (due to good story bias, easily inserted political agendas or different interpretations of worldview) introducing biasing or noise elements.

There are of course some exceptions. Hannu Rajaniemi provides a neat technical trick to the AI boxing problem in the second novel of his Quantum Thief trilogy (turn a classical computation into a quantum one that will decohere if it interacts with the outside world). But the fictions most people mention in AI safety discussions are unlikely to be helpful - mostly because very few stories succeed with point (2) (and if they are well written, they hide this fact convincingly!)

Commenting on the first myth, Yudkowsky himself seems to be pretty sure of this when reading his comment here: http://econlog.econlib.org/archives/2016/03/so_far_my_respo.html. I know Yudkowsky's post is written after this LessWrong article, but it still seems relevant to mention.

Following some somewhat misleading articles quoting me, I thought I’d present the top 10 myths about the AI risk thesis:

1) That we’re certain AI will doom us. Certainly not. It’s very hard to be certain of anything involving a technology that doesn’t exist; we’re just claiming that the probability of AI going bad isn’t low enough that we can ignore it.

MISLEADING.

If by "we" you mean the people who have published their thoughts on this matter. I believe I am right in saying that you have in the past referenced Steve Omohundro's paper, in which he says:

Without special precautions, [the AGI] will resist being turned off, will try to break into other machines and make copies of itself, and will try to acquire resources without regard for anyone else’s safety. These potentially harmful behaviors will occur not because they were programmed in at the start, but because of the intrinsic nature of goal driven systems (Omohundro, 2008).

Although this begins with "without special precautions" it then goes on to ONLY list all the ways in which this could happen, with no suggestions about how "special precautions" are even possible.

This cannot be construed as "we’re just claiming that the probability of AI going bad isn’t low enough that we can ignore it." The quote is also inconsistent with your statement "It’s very hard to be certain of anything involving a technology that doesn’t exist", because Omohundro says categorically that this "will occur ... because of the intrinsic nature of goal driven systems".

I picked Omohundro's paper as an example, but there are numerous similar writings from MIRI and FHI. I listed several examples in my AAAI paper (Loosemore, 2014).

2) That humanity will survive, because we’ve always survived before. Many groups of humans haven’t survived contact with more powerful intelligent agents. In the past, those agents were other humans; but they need not be. The universe does not owe us a destiny. In the future, something will survive; it need not be us.

STRAWMAN.

I haven't seen anyone of significance make that claim, so how can it be a "myth about the AI risk thesis"?

3) That uncertainty means that you’re safe. If you’re claiming that AI is impossible, or that it will take countless decades, or that it’ll be safe... you’re not being uncertain, you’re being extremely specific about the future. “No AI risk” is certain; “Possible AI risk” is where we stand.

STRAWMAN.

Again, only a tiny minority have said anything resembling those claims, so how can they be a "myth about the AI risk thesis"?

4) That Terminator robots will be involved. Please? The threat from AI comes from its potential intelligence, not from its ability to clank around slowly with an Austrian accent.

STRAWMAN.

Journalists and bloggers love to put a Terminator picture on their post as an Eyeball Magnet. Why elevate an Eyeball Magnet to the level of a "myth about the AI risk thesis"?

5) That we’re assuming the AI is too dumb to know what we’re asking it. No. A powerful AI will know what we meant to program it to do. But why should it care? And if we could figure out how to program “care about what we meant to ask”, well, then we’d have safe AI.

WRONG.

I published a paper giving a thorough analysis anddebunking of your (MIRI and FHI) claims in that regard -- and yet neither you nor anyone else at MIRI or FHI has ever addressed that analysis. Instead, you simply repeat the nonsense as if no one has ever refuted it. MIRI was also invited to respond when the paper was presented. They refused the invitation.

6) That there’s one simple trick that can solve the whole problem. Many people have proposed that one trick. Some of them could even help (see Holden’s tool AI idea). None of them reduce the risk enough to relax – and many of the tricks contradict each other (you can’t design an AI that’s both a tool and socialising with humans!).

INCOHERENT.

I have heard almost no one propose that there is "one simple trick", so how can it be a "myth" (you need more than a couple of suggestions, for something to be a myth).

More importantly, what happened to your own Point 1, above? You said "It’s very hard to be certain of anything involving a technology that doesn’t exist" -- but now you are making categorical statements (e.g. "None of them reduce the risk enough to relax" and "you can’t design an AI that’s both a tool and socialising with humans!") about the effectiveness of various ideas about that technology that doesn’t exist.

7) That we want to stop AI research. We don’t. Current AI research is very far from the risky areas and abilities. And it’s risk aware AI researchers that are most likely to figure out how to make safe AI.

PARTLY TRUE

......... except for all the discussion at MIRI and FHI regarding Hard Takeoff scenarios. And, again, whence cometh the certainty in the statement "Current AI research is very far from the risky areas and abilities."?

8) That AIs will be more intelligent than us, hence more moral. It’s pretty clear than in humans, high intelligence is no guarantee of morality. Are you really willing to bet the whole future of humanity on the idea that AIs might be different? That in the billions of possible minds out there, there is none that is both dangerous and very intelligent?

MISLEADING, and a STRAWMAN.

Few if any people have made the claim that increased intelligence BY ITSELF guarantees greater morality.

This is misleading because some people have discussed a tendency (not a guarantee) for higher intelligence to lead to greater morality (Mark Waser's papers go into this in some detail). Combining that with the probability of AI going through a singleton bottleneck, and there is a plausible scenario in which AIs themselves enforce a post-singleton constraint on the morality of future systems.

You are also profoundly confused (or naive) about how AI works, when you ask the question "Are you really willing to bet the whole future of humanity on the idea that AIs might be different?" One does not WAIT to find out if the motivation system of a future AI "is different", one DESIGNS the motivation system of a future AI to be either this way or that way.

It could be that an absolute correlation between increased intelligence and increased morality, in humans, is undermined by the existence of a psychopathic-selfish module in the human motivation system. Solution? Remove the module. Not possible to do in humans because of the biology, but trivially easy to do if you are designing an AI along the same lines. And if this IS what is happening in humans, then you can deduce nothing about future AI systems from the observation that "in humans, high intelligence is no guarantee of morality".

9) That science fiction or spiritual ideas are useful ways of understanding AI risk. Science fiction and spirituality are full of human concepts, created by humans, for humans, to communicate human ideas. They need not apply to AI at all, as these could be minds far removed from human concepts, possibly without a body, possibly with no emotions or consciousness, possibly with many new emotions and a different type of consciousness, etc... Anthropomorphising the AIs could lead us completely astray.

MISLEADING and CONFUSED.

This is a confusing mishmash of speculation and assumption.

A sentence like "minds far removed from human concepts" is not grounded in any coherent theory of what a 'mind' is or what a 'concept' is, or how to do comparative measures across minds and concepts. The sentence is vague, science-fictional handwaving.

The same goes for other statements like that the AI might have "no emotions or consciousness". Until you define what you mean by those terms, and give some kind of argument about why the AI would or would not be expected to have them, and what difference it would make, in either case, the statement is just folk psychology dressed up as science.

Lists cannot be comprehensive, but they can adapt and grow, adding more important points: 1) That AIs have to be evil to be dangerous. The majority of the risk comes from indifferent or partially nice AIs. Those that have sone goal to follow, with humanity and its desires just getting in the way – using resources, trying to oppose it, or just not being perfectly efficient for its goal.

MISLEADING and a STRAWMAN.

Yet again, I demonstrated in my 2014 paper that that claim is incoherent. It is predicated on a trivially stupid AI design, and there is no evidence that such a design will ever work in the real world.

If you, or anyone else at MIRI or FHI think that you can answer the demolition of this idea that I presented in the AAAI paper, it is about time you published it.

2) That we believe AI is coming soon. It might; it might not. Even if AI is known to be in the distant future (which isn't known, currently), some of the groundwork is worth laying now).

ACCURATE.

References

Loosemore, R.P.W. (2014). The Maverick Nanny with a Dopamine Drip: Debunking Fallacies in the Theory of AI Motivation. Association for the Advancement of Artificial Intelligence 2014 Spring Symposium, Stanford, CA.

Omohundro, Stephen M. 2008. The Basic AI Drives. In Wang, P., Goertzel, B. and Franklin, S. (Eds), Artificial General Intelligence 2008: Proceedings of the First AGI Conference. Amsterdam: IOS.

My own set of objections to AI risk does not include any of these (except possibly #7); but it's possible that they are unusual and therefore do not qualify as "top 10". Still, FWIW, I remain unconvinced that AI risk is something we should be spending any amount of resources on.

It would be great if someone could compile this list and some more points from the comments in a blog post so that it can be shared more easily.

The reality , is that robots will subvert human society not because they are more intelligent or intent on doing humans harm, rather, they will scupper at least industrialised societies by simply being capable of most of the work now done by humans. It will be simple economics which does the trick, with employers caught between the Scylla of needing to compete with other employers who use robots to replace men (which will mean a catastrophic and irreparable lost of demand) and the Charybdis of having to throw away the idea of laissez faire economics and engage in a command economy, something which political elites raised on worship of the great god Market will have immense difficulty in doing.

Read more at https://livinginamadhouse.wordpress.com/2011/07/01/robotics-and-the-real-sorry-karl-you-got-it-wrong-final-crisis-of-capitalism/