Convince me that humanity is as doomed by AGI as Yudkowsky et al., seems to believe

Regarding your list, Eliezer has written extensively about exactly why those seem like good assumptions.  If you want a quick summary though...

1.  Human beings, at least some of us, appear to be generally intelligent.  Unless you believe that this is due to a supernatural phenomenon (maybe souls are capable of hypercomputing?), general intelligence is thus demonstrably a thing that can exist in the natural world if matter is in the right configuration for it.  Eventually, human engineering should be able to discover and create the right configuration.
2. Modern neural nets appear to work closely analogously to the brain, with neurons firing or not depending on which other neurons are firing and knowledge represented in which neurons are connected and how strongly.  While it would require a bit of math to explain rigorously, this is a system that is capable of producing nearly any output due to any change in the input, and is thus flexible enough to reflect nearly any pattern.  Backpropagation can in turn be used to find any patterns in the inputs (as well as more advanced techniques such as the Google Pathways system), and a program that knows the relevant patterns in what it's looking at can both predict and optimize.  If that isn't obvious, consider that backprop can select for a program that predicts relevant results of the observed system, and that reversing this program allows for predicting which system states have a given result, which in turn allows for optimization.  If this still isn't obvious, I'd be happy to answer any questions you have in the comments; this part is complicated enough that trying to do it justice in a paragraph is difficult.  Given that artificial neural nets appear to have generalizable prediction and optimization abilities though, it doesn't seem too much of a stretch that researchers will be able to scale them up to a fully general understanding of the world this century, and quite possibly this decade.  
3. Default nonalignment arises from simple entropy.  There are an inconceivable number of possible goals in the world, and a mind created to fulfill one of them without careful specification is unlikely to end up with one of the very few goals that is consistent with human survival and flourishing.  The obvious counterargument to this is that an AI isn't likely to be created with a random goal; its creators are likely to at least give it instructions like "make everyone happy".  The counter-counterargument, however, is that our values are difficult to specify in terms that will make sense to a machine that doesn't have human instincts.  If I ask you to "make someone happy", you implicitly understand a vast array of ideas that accompany the request:  I'm asking you to help them out in a way that matches the sort of help people could give each other in normal life.  A birthday present counts; wiring their brain's pleasure centers up to a wall socket probably doesn't; threatening to kill their loved ones if they don't claim to be happy is right out.  But just like computers learning simple code do exactly what you say without any instinctive understanding of what you really meant, a computer receiving a specification of what it ought to do on a world-changing scale will be prone to bugs where what we wanted and what we asked for diverge (which is the source of bugs today as well!)
4. This point relies on two things:  collateral damage and the arbitrariness of values.  The risk of collateral damage should be quite clear when considering what happens to other animals caught in the way of human projects.  We tend not to even notice anthills bulldozed to make way for a new building.  As for values, it is certainly possible to attempt to predict any given quantity, be it human happiness or the number of purple polka dots in the world.  And turning that into optimizing for the quantity is as simple as picking actions that are predicted to result in the highest values of it.  Nowhere along the line does anything like human decency enter the picture, not by default.  If you have further questions about this I would recommend looking up the Orthogonality Thesis, the idea that any level of intelligence can coexist with any set of baseline values.  Our values are certainly not arbitrary to us, but they do not appear to be part of the basic structure of math in a way that would force all possible minds to agree.  
5. This isn't just about corrigibility.  An unaligned but perfectly corrigible AI (i.e. one that would follow any order to stop what it was doing and change its actions and values as directed) would still be a danger, as it would have excellent reason to ensure that we couldn't give the order that would halt its plans!  How dangerous a mind smarter than us could be is unpredictable (we could not, after all, know exactly what it would do without being that smart ourselves), but given both how easily humans are able to dominate even slightly less intelligent animals (the difference in intellect between a human and a chimpanzee is fairly small relative to the range of animal intelligence, and if we can make general AI at all, we can likely make one smarter than we are by a much larger margin than that between us and the other species) and that even within the range of plans humans have been able to think up, strategies like nanotech promise nearly total control of the world to anyone who can figure out the exact details, it seems unwise to expect to survive a conflict with a hostile superintelligence. 
6. Certainly we have not yet solved alignment, and most existing alignment researchers have no clear idea of how progress can be made even in principle.  This is one area where I personally diverge from the Less Wrong consensus a bit, however, as I suspect that it should be possible to create a viable alignment strategy by experimentation with AIs that are fairly powerful, but neither yet human level nor smart enough to pose the risks of a superintelligence.  However, such a bootstrapping strategy is so far purely theoretical, and the current approach of trying to come up with human-understandable alignment strategies purely by human cognition has shown almost no progress thus far.  There have been a few interesting ideas thrown around, such as Functional Decision Theory [? · GW], an approach to making choices that avoids many common pitfalls, and Coherent Extrapolated Volition [? · GW], a theory of value that seeks to avoid locking in our existing mistakes and misapprehensions.  However, neither these ideas nor any other produced thus far by alignment researchers can be used in practice yet to prevent an AI from getting the wrong idea of what to pursue, nor from being lethally stubborn in pursuing that wrong idea. 
7. A hostile superintelligence stands a decent chance of killing us all, or else of ensuring that we cannot take any action that could interfere with its goals.  That's quite a large first mover advantage. 
8. At the risk of sounding incredibly cynical, the problem in convincing a great many AI researchers isn't a matter of the convincingness or lack thereof of the arguments.  Rather, most people simply follow habits and play roles, and any argument that they should change their comfortable routine will, for most people, be rejected out of hand.  On the bright side, DeepMind, one of the leading organizations in the field of AI research, is actually somewhat interested in alignment, and has already done some work looking into how far a goal can be optimized before degenerate results occur.  This doesn't guarantee they'll succeed, of course, and some researchers looking into the problem isn't the same as a robust institutional AI safety culture.  But it's a very good sign that this story might have a happy ending after all, if people are sufficiently careful and smart.  
9. Given all of this, the likelihood of world-ending AI fairly soon (timeline estimates vary, but I would not be at all surprised to see AGI this decade) and the difficulty of alignment, hopefully it is a little clearer now why so many here are concerned.  That said, I think there is still quite a lot of hope, at least if the alignment community starts looking into experiments aimed at creating agents that can get better at understanding other agents' values, and better at avoiding too much disruption along the way.  

As a minor token of how much you're missing:

1. If AGI is created by people who are not sufficiently educated (aka aware of a solution to the Alignment problem) and cautious, then it will almost certainly be unaligned.

You can educate them all you want about the dangers, they'll still die.  No solution is known.  Doesn't matter if a particular group is cautious enough to not press forwards (as does not at all presently seem to be the case, note), next group in line destroys the world.

You paint a picture of a world put in danger by mysteriously uncautious figures just charging ahead for no apparent reason.

This picture is unfortunately accurate, due to how little dignity we're dying with.

But if we were on course to die with more dignity than this, we'd still die.  The recklessness is not the source of the problem.  The problem is that cautious people do not know what to do to get an AI that doesn't destroy the world, even if they want that; not because they're "insufficiently educated" in some solution that is known elsewhere, but because there is no known plan in which to educate them.

If you knew this, you sure picked a strange straw way of phrasing it, to say that the danger was AGI created by "people who are not sufficiently educated", as if any other kind of people could exist, or it was a problem that could be solved by education.

I'd like to distinguish between two things. (Bear with me on the vocabulary. I think it probably exists, but I am not really hitting the nail on the head with the terms I am using.)

1. Understanding why something is true. Eg. at the gears level, or somewhat close to the gears level.
2. Having good reason to believe that something is true.

Consider this example. I believe that the big bang was real. Why do I believe this? Well, there are other people who believe it and seem to have a very good grasp on the gears level reasons. These people seem to be reliable. Many others also judge that they are reliable. Yada yada yada. So then, I myself adopt this belief that the big bang is real, and I am quite confident in it.

But despite having watched the Cosmos episode at some point in the past, I really have no clue how it works at the gears level. The knowledge isn't Truly A Part of Me [LW · GW].

The situations with AI is very similar. Despite having hung out on LessWrong for so long, I really don't have much of a gears level understanding at all. But there are people who I have a very high epistemic (and moral) respect for who do seem to have a grasp on things at the gears level, and are claiming to be highly confident about things like short timelines and us being very far and not on pace to solve the alignment problem. Furthermore, lots of other people who I respect also have adopted this as their belief, eg. other LessWrongers who are in a similar boat as me with not having expertise in AI. And as a cherry on top of that, I spoke with a friend the other day who isn't a LessWronger but for whom I have a very high amount of epistemic respect for. I explained the situation to him, and he judged all the grim talk to be, for lack of a better term, legit. It's nice to get an "outsider's" perspective as a guard against things like groupthink.

So in short, I'm in the boat of having 2 but not 1. And it seems appropriate to me more generally to be able to have 2 but not 1. It'd be hard to get along in life if you always required a 1 to go hand in hand with 2. (Not to discourage anyone from also pursuing 1. Just that I don't think it should be a requirement.)

Coming back to the OP, it seems to be mostly asking about 1, but kinda conflating it with 2. My claim is that these are different things that should kinda be talked about separately, and that assuming that you too have a good amount of epistemic trust for Eliezer and all of the other people making these claims, you should probably adopt their beliefs as well.

I find point no. 4 weak.

4. Unaligned AGI will try to do something horrible to humans (not out of maliciousness, necessarily, we could just be collateral damage), and will not display sufficiently convergent behavior to have anything resembling our values.

I worry that when people reason about utility functions, they're relying upon the availability heuristic [? · GW]. When people try to picture "a random utility function", they're heavily biased in favor of the kind of utility functions they're familiar with, like paperclip-maximization, prediction error minimization, or corporate profit-optimization.

How do we know that a random sample from utility-function-space looks anything like the utility functions we're familiar with? We don't. I wrote a very short story [LW · GW] to this effect. If you can retroactively fit a utility function to any sequence of actions, what predictive power do we gain by including utility functions into our models of AGI?

1. AGI is possible to create.

Humans exist.

2. AGI will be created within the next century or so, possibly even within the next few years.

The next century is consensus, I think, and arguments against the next few years are not on the level where I would be comfortable saying "well, it wouldn't happen, so it's ok to try really hard to do it anyway".

3. If AGI is created by people who are not sufficiently educated (aka aware of a solution to the Alignment problem) and cautious, then it will almost certainly be unaligned.

I guess the problem here is that by the most natural metrics the best way for AGI to serve its function provably leads to catastrophic results. So you either need to not try very hard, or precisely specify human values from the beginning.

4. Unaligned AGI will try to do something horrible to humans (not out of maliciousness, necessarily, we could just be collateral damage), and will not display sufficiently convergent behavior to have anything resembling our values.

Not sure what's the difference with 3 - that's just definition of "unaligned"?

5. We will not be able to effectively stop an unaligned AGI once it is created (due to the Corrigibility problem).

Even if we win against first AGI, we are now in a situation where AGI is proved for everyone to be possible and probably easy to scale to uncontainable levels.

6. We have not yet solved the Alignment problem (of which the Corrigibility problem is merely a subset), and there does not appear to be any likely avenues to success (or at least we should not expect success within the next few decades).

I don't think anyone claims to have a solution that works in non-optimistic scenario?

7. Even if we solved the Alignment problem, if a non-aligned AGI arrives on the scene before we can implement ours, we are still doomed (due to first-mover advantage).

There are also related considerations like "aligning something non-pivotal doesn't help much".

8. Our arguments for all of the above are not convincing or compelling enough for most AI researchers to take the threat seriously.

The more seriously researchers take the threat, the more people will notice, and then someone will combine techniques from last accessible papers on new hardware and it will work.

9. As such, unless some drastic action is taken soon, unaligned AGI will be created shortly, and that will be the end of the world as we know it.

I mean, "doomed" means there are no much drastic actions to take^^.

The laws of physics in our particular universe make fission/fusion release of energy difficult enough that you can't ignite the planet itself.  (well you likely can, but you would need to make a small black hole, let it consume the planet, then bleed off enough mass that it then explodes.  Difficult). 

Imagine a counterfactual universe where you could, and the Los Alamos test ignited the planet and that was it.

 My point is that we do not actually know yet how 'somewhat superintelligent' AIs will fail.  They may 'quench' themselves like fission devices do - fission devices blast themselves apart and stop reacting, and almost all elements and isotopes won't fission.  Somewhat superintelligent AGIs may expediently self hack their own reward function to give them infinite reward, shortly after box escape, and thus 'quench' the explosion in a quick self hack.  

So our actual survival unfortunately probably depends on luck.  It depends not on what any person does, but on the laws of nature.  In a world where a fission device will ignite the planet, we'd be doomed - there is nothing anyone could do to 'align' fission researchers not to try it.  Someone would try it and we'd die.  If AGI is this dangerous, yeah, we're doomed.

The bottom line is: nobody has a strong argument in support of the inevitability of the doom scenario (If you have it, just reply to this with a clear and self contained argument.). 

From what I'm reading in the comments and in other papers/articles, it's a mixture of beliefs, estrapolations from known facts, reliance on what "experts" said, cherry picking. Add the fact that bad/pessimistic news travel and spread faster than boring good news.

A sober analysis enstablish that super-AGI can be dangerous (indeed there are no theorems forbidding this either), what's unproven is that it will be HIGHLY LIKELY to be a net minus for humanity. Even admitting that alignement is not possible, it's not clear why humanity and super-AGI goals should be in contrast, and not just different. Even admitting that they are highly likely to be in contrasts, is not clear why strategies to counter this cannot be of effect (e.g. parner up with a "good" super-AGI).

Another factors often forgotten is that what we mean by "humanity" today may not have the same meaning when we will have technologies like AGIs, mind upload or intelligence enhancement. We may literally become those AIs.

As I see, nobody is afraid of "alpine village life maximization", as some are afraid of "paper-clip maximization". Why is that? I wouldn't mind very much, a rouge superintelligence which tiles the Universe with alpine villages. In the past discussions, that would be "astronomical waste", now it's not even in the cards anymore? We are doomed to die, and not to be "bored for billion of years in a nonoptimal scenario". Interesting.

I see no problems with your list. I would add that creating corrigible superhumanly intelligent AGI doesn't necessarily solve the AI Control Problem forever because its corrigibility may be incompatible with its application to the Programmer/Human Control Problem, which is the threat that someone will make a dangerous AGI one day. Perhaps intentionally.    

A desire to understand the arguments is admirable.

Wanting to actually be convinced that we are in fact doomed is a dereliction of duty.

Karl Popper wrote that

Optimism is a duty. The future is open. It is not predetermined. No one can predict it, except by chance. We all contribute to determining it by what we do. We are all equally responsible for its success.

Only those who believe success is possible will work to achieve it. This is what Popper meant by "optimism is a duty".

We are not doomed. We do face danger, but with effort and attention we may yet survive.

I am not as smart as most of the people who read this blog, nor am I an AI expert. But I am older than almost all of you. I've seen other predictions of doom, sincerely believed by people as smart as you, come and go. Ideology. Nuclear war. Resource exhaustion. Overpopulation. Environmental destruction. Nanotechnological grey goo. 

One of those may yet get us, but so far none has, which would surprise a lot of people I used to hang around with. As Edward Gibbon said, "however it may deserve respect for its usefulness and antiquity, [prediction of the end of the world] has not been found agreeable to experience."

One thing I've learned with time: Everything is more complicated than it seems. And prediction is difficult, especially about the future.

When people who are smarter than you

The relevant subset of people who are smarter than you is the people who have relevant industry experience or academic qualifications.

There is no form of smartness that makes you equally good at everything.