To the average human, controlled AI is just as lethal as 'misaligned' AI

Before anything else, I would note that your proposed scenario has winners. In other words, it's a horrible outcome, but in this world where alignment as you've defined it is easy but a group of humans uses it to destroy most of the population, that group of humans likely survives, repopulates the Earth, and continues on into the future.

This, to put it mildly, is a far, far better outcome than we'd get from an AI that wants to kill everyone of its own agency, or that doesn't care about us enough to avoid doing so as a side effect of other actions.

I don't remember where or when, but IIRC EY once wrote that if an evil person or group used ASI to take over the world, his reaction would be to 'weep for joy and declare victory' that any human-level agents whatsoever continue to exist and retain enough control to be able to be said to "use" the AI at all.

That said, yes, if we do figure out how to make an AI that actually does what humans want it to do, or CEV-would-want it to do, when they ask it to do something, then preventing misuse becomes the next major problem we need to solve, or ideally to have solved before building such an AI. And it's not an easy one.

Downvoted because the title seems straightforwardly false while not actually arguing for it (making it a bit clickbaity, but I am more objecting to the fact that it's just false). Indeed, this site has a very large number of arguments and posts about why AIs could indeed kill people (and people with AIs might also kill people, though probably many fewer).

Your post reads a bit strangely. 

At first, I thought you were arguing that AGI might be used by some extremists to wipe out most of humanity for some evil and/or stupid reason.  Which does seem like a real risk.  

Then you went on to point out that someone who thought that was likely might wipe out most of humanity (not including themselves) as a simple survival strategy, since otherwise someone else will wipe them out (along with most other people). As you note, this requires a high level of unconcern for normal moral considerations, which one would think very few people would countenance.

Now comes the strange part... You argue that actually maybe many people would be willing to wipe out most of humanity to save themselves, because...  wiping out most of humanity sounds like a pretty good idea!

I'm glad that in the end you seem to still oppose wiping out most of humanity, but I think you have some factual misconceptions about this, and correcting them is a necessary first step to thinking of how to address the problem.

Concerning climate change, you write: "In the absence of any significant technological developments, sober current trajectory predictions seem to me to range from 'human extinction' to 'catastrophic, but survivable'".

No. Those are not "sober" predictions. They are alarmist claptrap with no scientific basis. You have been lied to. Without getting into details, you might want to contemplate that global temperatures were probably higher than today during the "Holocene Climatic Optimum" around 8000 years ago.  That was the time when civilization developed.  And temperatures were significantly higher in the previous interglacial, around 120,000 years ago.  And the reference point for supposedly-disastrous global warming to come is "pre-industrial" time, which was in the "little ice age", when low temperatures were causing significant hardship. Now, I know that the standard alarmist response is that it's the rate of change that matters.  But things changed pretty quickly at the end of the last ice age, so this is hardly unprecedented. And you shouldn't believe the claims made about rates of change in any case - actual science on this question has stagnated for decades, with remarkably little progress being made on reducing the large uncertainty about how much warming CO2 actually causes.

Next, you say that the modern economy is relatively humane "under conditions of growth, which, under current conditions, depends on a growing population and rising consumption. Under stagnant or deflationary conditions it can be expected to become more cutthroat, violent, undemocratic and unjust."

Certainly, history teaches that a social turn towards violence is quite possible. We haven't transcended human nature.  But the idea that continual growth is needed to keep the economy from deteriorating just has no basis in fact.  Capitalist economies can operate perfectly fine without growth.  Of course, there's no guarantee that the economy will be allowed to operate fine.  There have been many disastrous economic policies in the past.  Again, human nature is still with us, and is complicated. Nobody knows whether social degeneration into poverty and tyranny is more likely with growth or without growth.

Finally, the idea that a world with a small population will be some sort of utopia is also quite disconnected from reality.  That wasn't the way things were historically. And even if it was, it woudn't be stable, since population will grow if there's plenty of food, no disease, no violence, etc. 

So, I think your first step should be to realize that wiping out most of humanity would not be a good thing. At all. That should make it a lot easier to convince other people not to do it.

I think you raise an important point. If we solve alignment, do we still all die?

This has been discussed in the alignment community under the terminology of a "pivotal act". It's often been assumed that an aligned AGI would prevent the creation of more AGIs to prevent both the accidental creation of misaligned AGIs, and the deliberate creation of AGIs that are misaligned to most of humanity's interests, while being aligned to the creator's goals. Your misuse category falls into the latter. So you should search for posts under the term pivotal act. I don't know of any particularly central ones off the top of my head.

However, I think this is worth more discussion. People have started to talk about "multipolar scenarios" in which we have multiple or many human-plus level AGIs. I'm unclear on how people think we'll survive such a scenario, except by not thinking about it a lot. I think this is linked to the shift in predicting a slower takeoff, where AGI doesn't become superintelligent that quickly. But I think the same logic applies, even if we survive for a few years longer.

I hope to be convinced otherwise, but I currently mostly agree with your logic for multipolar scenarios. I think we're probably doomed to die if that's allowed to happen. See What does it take to defend the world against out-of-control AGIs? [AF · GW] for reasons that a single AGI could probably end the world even if friendly AGIs have a headstart in trying to defend it.

I'd summarize my concerns thus: Self-improvement creates an unstable situation to which no game-theoretic cooperative equilibrium applies. It's like playing Diplomacy where the players can change the rules arbitrarily on each turn. If there are many AGIs under human control, one will eventually have goals for the use of Earth at odds with those of humanity at large. This could happen because of an error in its alignment, or because the human(s) controlling it has non-standard beliefs or values.

When this happens, I think it's fairly easy for a self-improving AGI to destroy human civilization (although perhaps not other AGIs with good backup plans). It just needs to put together a hidden (perhaps off-planet, underground or underwater) robotic production facility that can produce new compute and new robots. That's if there's nothing simpler and more clever to do, like diverting an asteroid or inventing a way to produce a black hole. The plans get easier the less you care about using the Earth immediately afterward.

I agree that this merits more consideration.

I also agree that the title should change. LW very much looks down on clickbait titles. I don't think you intended to argue that AI won't kill people, merely that people with AIs will. I believe you can edit the title, and you should.

Edit: I recognized the title and didn't take you to be arguing against autonomous AI as a risk - but it does actually make that claim, so probably best to change it.

because it is more task-specific and therefore technically simpler to achieve than general intelligence, doesn't require escaping its own creators' controls

An argument for danger of human-directed misuse doesn't work as an argument against dangers of AI-directed agentic activity. Both are real, though misuse only becomes an extinction-level problem when AIs are very powerful, at which point the AI-directed activity that is not misuse by humans also becomes relevant. With extinction-level problems, it doesn't matter for allocation of attention which one is worse (since after a critical failure there are no retries with a different allocation to reflect lessons learned), only that either is significant and so both need to be addressed.

If alignment is very easy, misuse becomes important. If it's hard, absence of misuse doesn't help. Though there is also a problem of cultural value drift, where AIs change their own culture very quickly on human timescales without anyone individually steering the outcome (including the AIs), so that at the end of this process (that might take merely months to years) the AIs in charge of civilization no longer care about human welfare, with neither misuse nor prosaic misalignment (in individual principal-agent relationships) being the cause of this outcome.

The super simple claim is:

If an unaligned AI by itself can do near-world-ending damage, an identically powerful AI that is instead alignable to a specific person can do the same damage.

I agree that it could likely do damage, but it does cut off the branches of the risk tree where many AIs are required to do damage in a way that relies on them being similarly internally misaligned, or at least more likely to cooperate amongst themselves than with humans.

So I'm not convinced it's necessarily the same distribution of damage probabilities, but it still leaves a lot of room for doom. E.g. if you really can engineer superspreadable and damaging pathogens, you may not need that many AIs cooperating.