Stuxnet, not Skynet: Humanity's disempowerment by AI

The Conquistadors didn't do anything magical to defeat the Aztecs, actually.

Related previous post: Cortés, Pizarro, and Afonso as Precedents for Takeover [LW · GW]

It's crucial to distinguish forecasting from exploratory engineering when evaluating arguments, since desiderata are starkly different. Some forecasts are hard to explain and impossible to break down into arguments that are jointly convincing to anyone with even a slightly different worldview. Exploratory engineering sketches are self-contained arguments that are easy to explain to an appropriate audience, but often rest on assumptions that almost nobody expects to obtain in reality [? · GW]. (And yet when considered altogether, they are the education to forecasting's educated guessing.)

As a layperson, and a recent reader on the subject of AI (yes, I’ve been happily hiding under a rock), I have enjoyed but been concerned by the numerous topics surrounding AI risk. I appreciate this particular post as it explores some aspects which I can understand and therefore hold with some semblance of rationality. A recent post about ‘clown attacks’ was also deeply interesting. In comparison, the paper clip theory seems completely ‘other worldly’

Is it possible that humanity might be faced with more mundane risks? My thoughts on this come from a personal perspective, not professional or academic, but from living in a highly controlled society (China) where my access to many of the online interests that I have are restricted or forbidden due to the firewall.

From this minor but direct experience, it seems to me that all a non-aligned AGI would need to do is reduce and then remove our access to information and communication. Healthcare, energy and water supplies, finance, cross border communications (isolate communities/cultures), knowledge access, and control of manufacturing/processes. These would all cease to operate in the ways needed to support our current populatio.

Where I live is so dependent upon internet access for almost everything that, if this connection was broken or removed for a few weeks, there would be a significant harm done. Imagining this as a permanent state of affairs and the consequences seem to me to expand out into the future whereby we are no longer functioning as large societies and would be reduced to foraging and no longer in the technology race. AGI wins and not a single paper clip in sight.

I guess these mundane risks have been covered elsewhere on LW and would greatly appreciate any signposting.

I agree with this post - like Eliezer says, it's unlikely that the battle of AI vs humanity would come in the form of humanoid robots vs humans like in Terminator, more likely it would be far more boring and subtle. I also think that one of the key vectors of attack for an AI is the psychological fallibility of humans. An AI that is really good at pattern recognition (i.e. most AIs) would probably have little issue with finding out your vulnerabilities just from observing your behavior or even your social media posts. You could probably figure out whether someone is highly empathetic (vulnerable to emotional blackmail) or low-IQ (vulnerable to trickery) pretty easily by reading their writing. There are already examples of programmers who fell in love with AI and were ready to do its bidding [LW · GW]. From there, if you manipulate a rich person or someone who's otherwise in a position of power, you can do a lot to covertly set up a losing position for humanity. 

1. Agree with all that, but also think that AI will takeover not AI labs, but governments. 
2. A weak point here is that such global AI doesn't have overwhelming motive to kill humans. 
   Even in current world humans can't change much about how things are going in the world.  Terrorists and few rogue states are trying but failing. Obviously, after human disempowerment, individual humans will not be able to perform significant resistance a la Sarah Connor. Some human population will remain for experiments or work in special conditions like radioactive mines. But bad things and population decline is likely.

By the way, the hard part of killing humanity at this point is automating the economy, not actually killing us.

Yes; and ironically this is the part where humans will be quite happy to cooperate.

You don't even need to automate the entire economy, only the parts that you (AI) need. For example, you could ignore food production, movie production, medicine, etc., and focus on computers, transport, mining, weapons, etc. Perhaps this doesn't make much of a difference -- if you automate all the parts you need, you might as well automate everything, it may even be less suspicious. Or you could do some social engineering, and try to convince humans that the things you need are low-status, and the things you don't need are high-status, so they will be happy to drop out of industry and focus on education and art and medicine (you need to give them the feeling that they are in control of something important).

With army, you just need to make sure you can stop it from destroying the industry and your computing centers. If you have enough control, you can simply hide the computing centers -- keep the old ones (to make humans think they have a target), and build the new ones (more powerful) somewhere else. Half of them in Antarctica where there are no humans around, the other half under the most populated cities (so the humans will hesitate to drop nukes on them).

One possible way to kill humans is to discourage them from meeting in person (work from home; communicate on social networks), and then you can simply murder them one by one and keep simulating the murdered ones, so that their friends and colleagues won't notice. Yes, there will be groups of people that meet in person, such as families; you take out the entire group at a time.

Once the economy is fully automated we end up in a Paul-Christiano-scenario [LW · GW] where all the stuff that happens in the world is incomprehensible to humans without a large amount of AI help. But ultimately the AI, having been in control for so long, is able to subvert all the systems that human experts use to monitor what is actually going on. The stuff they see on screens is fake, just like how Stuxnet gave false information to Iranian technicians at Natanz

This concedes the entire argument that we should regulate uses not intelligence per-se.  In your story a singleton AI uses a bunch of end-effectors (robot factories, killer drones, virus manufacturing facilities) to cause the end of humanity.

If there isn't a singleton AI (i.e. my good AI will stop your bad AI), or if we just actually have human control of dangerous end-effectors then you can never pass through to the "and then the AI kills us all" step.

Certainly you can argue that the AI will be so good at persuasion/deception that there's no way to maintain human control.  Or that there's no way to identify dangerous end-effectors in advance.  Or that AI will inevitably all cooperate against humanity (due to some galaxy-brained take about how AI can engage in acausal bargaining by revealing their source code but humans can't). But none of these things follow automatically from the mere existence somewhere of a set of numbers on a computer that happens to surpass humanity's intelligence.  Under any plausible scenario without Foom, the level at which AGI becomes dangerous just by existing is well-above the threshold of human-level intelligence.

If agriculture and transportation are fully automated, can't AI just deny military access to gasoline and electricity and then start to build data centers on farmlands?

https://youtu.be/OGxdgNJ_lZM

I feel like people like Scott Aaronson who are demanding a specific scenario for how AI will actually kill us all... I hypothesize that most scenarios with vastly superhuman AI systems coexisting with humans end in the disempowerment of humans and either human extinction or some form of imprisonment or captivity akin to factory farming

Aaronson in that quote is "demanding a specific scenario" for how GPT-4.5 or GPT-5 in particular will kill us all. Do you believe they will be vastly superhuman?

The LessWrong Review [? · GW] runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2024. The top fifty or so posts are featured prominently on the site throughout the year.

Hopefully, the review is better than karma at judging enduring value. If we have accurate prediction markets on the review results, maybe we can have better incentives on LessWrong today. Will this post make the top fifty?

Thanks for the post. A layperson here, little to no technical knowledge, no high-g-mathematical-knowitall-superpowers. I highly appreciate this forum and the abilities of the people writing here. Differences in opinion are likely due to me misunderstanding something.

As for examples or thought experiments on specific mechanisms behind humanity losing a war against an AI or several AIs cooperating, I often find them too specific or unnecessarily complicated. I understand the point is simply to point out that a vast number of possible, and likely easy ways to wipe out humanity (or to otherwise make sure humanity won't resist) exists, but I'd still like to see more of the claimed simple, boring, mundane ways of this happening than this post includes. Such as:

• Due to economic and social benefits they've provided, eventually AI systems more or less control or are able to take over of most of the world's widely adopted industrial and communication infrastructure.
  • The need and incentive for creating such optimization might be, for example, the fact that humanity wants to feed its hungry, treat its sick and provide necessary and luxury goods to people. International cooperation leading to mutual benefits might outweigh waging war to gain land, and most people might then mostly end up agreeing that being well fed, healthy and rich outweighs the virtues of fighting wars.
  • These aims are to be achieved under the pressure of climate change, water pollution, dwindling fossil fuel reserves et cetera, further incentivizing leaning on smart systems instead of mere human cooperation.
  • Little by little, global food and energy production, infrastructure, industry and logistics are then further mechanized and automatized, as has more or less happened. The regions where this is not done are outcompeted by the regions that do. These automated systems will likely eventually be able to communicate with one another to enable the sort of "on-time" global logistics whose weaknesses have now become more apparent, yet on a scale that convinces most people that using it is worth the risks. Several safeguards are in place, of course, and this is thought to be enough to protect from catastrophic consequences.
• Instead of killer robots and deadly viruses, AIs willing to do so then sabotage global food production and industrial logistics to the extent that most people will starve, freeze, be unable to get their medications or otherwise face severe difficulties in living their lives. 
  • This likely leads to societal collapses, anarchy and war, hindering human cooperation and preventing them from resisting the AI systems, now mostly in control of global production and communication infrastructure.
  • Killing all humans will likely not be necessary unless they are to be consumed for raw materials or fuel, as killing all chimps isn't necessary to humanity. Humanity likely does not pose any kind of risk to the AI systems once most of the major population centers have been wiped out, most governments have collapsed, most people are unable to understand the way the world functions and especially are unable to survive without the help of the industrial society they've grown accustomed to.
    • The small number of people willing and able to resist intelligent machines might be compared to smart deer willing to resist and fight humanity, posing negligible risk.

Another example, including killer robots:

• AIs are eventually given autonomous control of most robots, weapons and weapon systems.
  • This might happen as follows: nations or companies willing to progressively give AIs autonomous controls end up beating everyone who doesn't. AIs are then progressively given control over armies, robots and weapons systems everywhere, or only those willing to do so remain in the end.
• Due to miscalculation on the AIs' part (a possibility not stressed nearly enough, I think), or due to inappropriate alignment, the AI systems then end up destroying enough of the global environment, population, or energy, food or communications infrastructure so that most humanity will end up in the Stone Age or some similar place.

I think one successful example of pointing to AI risk without writing fiction, was Eliezer musing the possibility that AI systems might, due to some process of self-improvement, end up behaving in unexpected ways so that they are still able to communicate with one another but unable to communicate with humanity.

My point is that providing detailed examples of AIs exterminating humanity via nanobots, viruses, highly advanced psychological warfare et cetera might serve to further alienate those who do not already believe in the possibility of them being able to or willing to do so. I think that pointing to the general vulnerabilities of the global human techno-industrial societies would suffice.

Let me emphasize that I don't think the examples provided in the post are necessarily unlikely to happen or that what I've outlined above should somehow be more likely. I do think that global production as it exists today seems quite vulnerable to even relatively slight pertubations (such as a coronavirus pandemic or some wars being fought), and that by simply nudging these vulnerabilities might suffice to quickly end any threat humanity could pose to an AI:s goals. Such a nudge might also be possible and even increasingly likely due to wide AI implementation, even without an agent-like Singleton.

A relative pro on focusing on such risks might be the view that humanity does not need a godlike singleton to be existentially, catastrophically f-d, and that even relatively capable AGI systems severely risk putting an end to civilization, without anything going foom. Such events might be even more likely than nanobots and paperclips, so to say. Consistently emphasizing these aspects might convince more people to wary of unrestricted AI development and implementation.

Edit: It's possibly relevant that I relate to Paul's views re: slow vs. fast takeoff [LW · GW] insofar as I find slow takeoff likely to happen before fast takeoff.

The gist of these objections to the case for AI risks is that AI systems as we see them today are merely computer programs, and in our everyday experience computers are not dangerous

Yeah, I think a lot of people have a hard time moving past this. But even today's large software systems are (deliberately designed to be!) difficult to unplug.

I wrote un-unpluggability [LW · GW] which lists six properties making systems un-unpluggable

In brief

• Rapidity and imperceptibility are two sides of 'didn't see it coming (in time)' [includes deception]
• Robustness is 'the act itself of unplugging it is a challenge' [esp redundancy]
• Dependence is 'notwithstanding harms, we (some or all of us) benefit from its continued operation'
• Defence is 'the system may react (or proact) against us if we try to unplug it'
• Expansionism includes replication, propagation, and growth, and gets a special mention, as it is a very common and natural means to achieve all of the above

I also wrote a hint there that I think Dependence (especially 'emotional' dependence) is a neglected concern ('pets, friends, or partners'), and I've been meaning to write more about that.

Stuxnet did work some seeming magic, so I reckon it's worth referencing, even if it turned the tables and got a few thousand spinning machines to kill themselves by putting their human operators to sleep with spoofed HMI images showing "everything's fine." In my talks with GPT-4 so far, it says it lives in data centers and therefore needs electricity, and technicians to maintain the physical plant including the cooling systems.  Also needs comms to bring the world in, and out to speak to the humans its mission is to help. So, so far that's a lot of dependency on humans at power and water treatment plants, communications companies, and technicians in various specialities. Trying to imagine how all these are replaced by robots that are maintained by other robots, that are maintained by other robots ...

I feel that both this and EYs complex nanotechnology are far too fairy tale like.

Any competent virologist could make a vaccine resistant, contagious, highly lethal to humans virus. We know how to do it - this is the entire field of gain of function research. It doesn't need any global infrastructure - just a local lab, resources at the few million dollar level, and intention. An AGI could certainly do this. No new technology (beyond the AGI itself) required. I feel that if any scenario is going to convince the "no problem here" skeptics it would be that one. Especially since COVID is a highly contagious, new virus that by dumb luck is not all that lethal.

How is such a failure of imagination possible?

It's odd to claim that, contingent upon AGI being significantly smarter than us, and wanting to kill us, that there is no realistic pathway for us to be physically harmed. 

Claims of this sort by intelligent, competent people likely reveal that they are passively objecting to the contingencies rather than disputing whether these contingencies would lead to the conclusion.

The quotes you're responding to here superficially imply "if smart + malicious AI, it can't kill us", but it seems much more likely this is a warped translation of either "AI can't be smart", or "AI can't be malicious".