Nick Bostrom's TED talk and setting priorities

If you mostly solve the 'Ageing' and 'Unnecessary Unhappiness' problems, the youthful, happy populous will probably give a lot more weight to 'Things That Might Kill Everyone'

I don't know about putting these things into proper categories, but I'm sure I'd be a lot more worried about the (more distant than a few decades) future if I had a stronger expectation of living to see it and I spent less time being depressed.

TED is more about "here are some cool ideas you haven't been exposed to" rather than "take this as your singular policy prescription for the rest of your lives." Those three ideas may not all carry the same weight but they are connected ideas which most people don't hear about or take seriously.

Putting these other two in the same category as "existential risk" seems like a mistake. This seems especially obvious in the case of (the present, normal rate of) death and existential risk. Bostrom's talk gives an annual death rate of 56 million, whereas if you take future generations into account, a 1% reduction in existential risk could save 10^32 lives.

And if you don't take people who don't exist into account, is it still a mistake?

if you take future generations into account, a 1% reduction in existential risk could save 10^32 lives.

And if you don't solve aging ever, and take future generations into account, then 100% of all human beings ever born will die, which kills at least as many people as any existential risk. (The inevitability of solving aging given enough time but no particular will or effort towards it isn't obvious - so it's not just a matter of saving a few decades worth of 56 million lives a year.)

There is only so much transhumanism one can cram into an 18-minute talk.

See Levels of Action. Creating utopia is analogous to an object-level action, while avoiding extinction would be a meta-level action: avoiding extinction supports the creation of utopia (in fact it is, obviously, necessary). However, at some point, meta-level actions have to bottom out in an object-level action -- otherwise they were pointless.

See also Lost Purposes. If we devote too much of ourselves to avoiding extinction, our values may drift away and we may simply become survival-maximizing agents who never end up creating utopia.

The biggest risk of "existential risk mitigation" is that it will be used by the "precautionary principle" zealots to shut down scientific research. There is some evidence that it has been attempted already, see the fear-mongering associated with the startup of the new collider at CERN.

A slowdown, much less an actual halt, in new science is the one thing I am certain will increase future risks, since it will undercut our ability to deal with any disasters that actually do occur.

One point Hayek made in his last book, The Fatal Conceit, is that you need to be aware of how others will use whatever policies you support. His arguments were primarily about how "well meaning" academics ended up paving the way for the Communist dictatorships, but the general argument is equally true of many other policies.

Anti-aging tech would free up a lot of resources for working on existential risks.

Bostrom is working on a paper showing how existential risk prevention can be seen as the most important task for humanity (PDF and HTML).

From the last section ("Outlook"):

We have seen that reducing existential risk emerges as a dominant priority in many aggregative consequentialist moral theories (and as a very important concern in many other moral theories). The concept of existential risk can thus help the morally or altruistically motivated to identify actions that have the highest expected value. In particular, given certain assumptions, the problem of making the right decision simplifies to that of following the maxipok principle.

I think it is very easy to believe that "death" and "life isn't usually as wonderful as it could be" are as important as existential risk if you weight heavily in favour of the well-being of you, people you know and people in other senses "close" to you.

Caring more about that is also very natural. If I were to tell a typical person that was going to die tomorrow, their reaction would be stronger than is going to die under the same circumstances etc.

Of course, shut up and multiply, but only if you actually care about all of the events equally.