Why will AI be dangerous?

Hi, I'm a skeptic of the Alignment Problem, meaning, I don't expect TAI will be dangerously misaligned. I'll go over some of the reasons why I'm skeptical, and I'll suggest how you could persuade me to take the problem more seriously.

Firstly, I definitely agree with the "insurance" argument. Even if I think that TAI is unlikely to be misaligned in a disastrous way, there is still the possibility of a rouge AI causing disaster, and I 100% agree that it worth investing in solving the Alignment Problem. So I DO NOT think that alignment research should be dismissed, ignored or defunded. In fact I would agree with most people on this forum that alignment research is severely underfunded and not taken seriously enough.

I also agree that misalignment is possible. I agree with the Orthogonality Thesis. Someone could build a paperclip maximizer (for example) if they for some reason wanted to. My opinion is that institutional actors (corporate and military research institutes) will not deploy misaligned AI. They will be aware that their AI is misaligned and will not deploy it until it is aligned, even if that takes a long time, because it simply is not in their self-interest to deploy an AI which is not useful to them. Also, if misaligned AI is deployed, the amount of damage that it could do is severely limited.

Corporate actors will not deploy an AI if it is likely to result in financial, legal or public relations disaster for them. This constraint does not apply to military actors, so if world war breaks out or if the Chinese or US Military-Industrial Complex takes the lead in AI away from the current leaders, US corporate research, that could be disastrous and is a scenario I am very afraid of. However, my primary fear is not of the "Technical" Alignment Problem but of the "Political" Alignment Problem. Allow me to explain.

Intelligence is commonly defined as "the ability to achieve goals" (or something like that). When we talk about AI-induced disaster, the AI is usually imagined to be extremely powerful (in at least one dimension). In other words, the fear is not of weak AI but of Artificial Super Intelligence, and "super" is a way of saying "the AI is super powerful and therefore dangerous." When discussing how disaster could happen, the story often goes "the AI invents grey goo nanotechnology/designs a lethal pathogen/discovers a lower vacuum energy and destroys the world."

What's never explained is how AI could suddenly become so powerful that it can achieve what entire nations of extremely smart people could never do. China, the Soviet Union, Nazi Germany or the US (at our worst) were never able to existentially destroy or totally dominate their enemies, despite extreme efforts to do so. Why would an AI be able to do so, and why would you expect it to happen by accident the moment AI is let out of the box?

A much scarier and much more plausible scenario involves conventional military hardware and police state tactics scaled up to an extreme. Suppose an evil dictator gets control of AGI and builds a fleet of billions of autonomous quadcoptors (drones) armed with ordinary guns. He uses the drones to spy on his people and suppress any attempt at resistance. If this dictator is not stopped by a competing military force, either because said dictator rules the sole superpower/one world government, or because the competing superpowers are also evil dictatorships, then this dictator would be able to oppress everyone and could not be opposed, possibility for the rest of eternity.

The dictator could be an ASI, but the dictator could also be a human being using a sufficiently aligned AI. Either way, we have a serious problem. Not the Technical Alignment Problem, but the Political Alignment Problem. Westerners used to believe that there was some sort of force of nature bending the moral arc towards justice, freedom and democracy as society progressed. IMHO this trend towards better society worked until the year 2001, when 9/11 caused the US to turn backwards (starting with the Patriot Act) and then more recently the rest of the world started to revert towards autocracy for some reason.

The problem is making the future safe and free for the common person. For all of human history, dictators were limited in how cruel and oppressive they could be. If the dictator was so awful that even his rank-and-file police and military would not support him, he would be toppled eventually. AI potentially takes that constraint away. An AI that is infinitely subservient (totally aligned) will follow the dictator's orders no matter how evil he is.

The Political Alignment Problem is probably not solvable -- we can only hope that free people prevail over autocracy. But enough politics, let's go back to the Technical Alignment Problem. Why don't I think it's a concern? Simply put, I don't think early AIs will be so powerful. Our world is built on millennia of legal, economic, technological and military safeguards to prevent individual actors (with the exception of autocrats) from doing too much damage. To my knowledge, the worst damage that any non-state individual actor could do was limited to about 16,000 deaths (upper estimate). Society is designed to limit the damage that bad actors and accidents can do, and there is no reason to believe that rouge AI will be dramatically more powerful than corporations or terrorists.

In summary, my objection is approximately the old response of "why wouldn't we just unplug the damn thing" with the added point that we would be willing to use police or if necessary military force if the AI resists. To be convinced otherwise, I would need to understand why early AIs would become so much more powerful than corporations, terrorists or nation-states.

I think that you raise a crucial point. I find it challenging to explain to people that AI is likely very dangerous. It‘s much easier to explain that pandemics, nuclear wars or environmental crises are dangerous. I think this is mainly due to the abstractness of AI and the concreteness of those other dangers, leading to availability bias.

The most common counterarguments I've heard from people about why AI isn't a serious risk are:

• AI is impossible, and it is just "mechanical" and lacks some magical properties only humans have.
• When we build AIs, we will not embed them with negative human traits such as hate, anger and vengeance.
• Technology has been the most significant driver of improvements in human well-being, and there's no solid evidence that this will change.

I have found that comparing the relationship between humans and chimpanzees with humans and hypothetical AIs is a good explanation that people find compelling. There's plenty of evidence suggesting that chimpanzees are pretty intelligent, but they are just not nearly intelligent enough to influence human decision making. This has resulted in chimps spending their lives in captivity across the globe.

Another good explanation is based on insurance. The probability that your house will be destroyed is small, but it's still prudent to buy home insurance. Suppose you believe that the likelihood that AI will be dangerous is small. Is it not wise that we insure ourselves by dedicating resources towards the development of safe AI?

Nice! I think it would be helpful to add a bunch of links at the end along the lines of "Here are some Respected Thinkers You Respect who agree that AI Risk is a Serious Possibility" and "Here are some lengthier, more rigorous pitches and overviews for you to read if you want to engage more with the basic ideas and arguments sketched here."

I've been thinking of a pitch that starts along these lines:

"You know how you kind of hate Facebook but can't stop using it?" 

I feel like most people I know intuitively understand that.  

I'm still playing with the second half of the pitch.  Something like, "What if it were 200x better at keeping you using it and didn't care that you hate it?" or "What if it had nukes?"

unlike other technologies, an AI disaster might not wait around for you to come clean it up

I think this piece is extremely important, and I would have put it in a more central place. The whole "instrumental goal preservation" argument makes AI risk very different from the knife/electricity/car analogies. It means that you only get one shot, and can't rely on iterative engineering. Without that piece, the argument is effectively (but not exactly) considering only low-stakes alignment.

In fact, I think if we get rid of this piece of the alignment problem, basically all of the difficulty goes away. If you can always try again after something goes wrong, then if a solution exists you will always find it eventually. 

This piece seems like much of what makes the difference between "AI could potentially cause harm" and "AI could potentially be the most important problem in the world". And I think even the most bullish techno-optimist probably won't deny the former claim if you press them on it.

Might follow this up with a post?

I think that the central argument for AI risk has the following structure and could be formulated without LW-slang:

1. AI can have unexpected capability gains which will make it significantly above the human level.
2. Such AI can create dangerous weapons (likely, nanobots).
3. AI will control these weapons.
4. This AI will likely have a dangerous goal system and will not care about human wellbeing.

Depending on who you are talking to for-profit corporations is a good analogy for what is meant by "misaligned". You can then point out that those same organizations are likely to make AI with profit maximization in mind, and might skimp out on moral restraint in favor of being superhumanly good at PR.

Use that comparison with the wrong person and they'll call you a communist.