A Quick Guide to Confronting Doom

I agree. I find myself in an epistemic state somewhat like: "I see some good arguments for X. I can't think of any particular counter-argument that makes me confident that X is false. If X is true, it implies there are high-value ways of spending my time that I am not currently doing. Plenty of smart people I know/read believe X; but plenty do not"

It sounds like that should maybe be enough to coax me into taking action about X. But the problem is that I don't think it's that hard to put me in this kind of epistemic state. Eg, if I were to read the right blogs, I think I could be brought into that state for a bunch of different values of X. A few of the top of my head that seem plausible:

• Climate change
• Monetary policy/hyperinflation
• Animal suffering

So I don't feel super trusting of my epistemic state. I guess I feel a sort of epistemic learned helplessness, where I am suspicious of smart bloggers' ability to get me to think an issue is important and worth dedicating my life to.

Not totally sure how to resolve this, though I suppose it would involve some sort of "going off on my own and actually thinking deeply about what it should take to convince me"

...and take out a bunch of loans...

That part really shouldn't be necessary (even if it may be rational, conditional on some assumptions). In the event that you do decide to devote your time to helping, whether for dignity or whatever else, you should be able to get funding to cover most reasonable forms of upskilling and/or seeing-if-you-can-help trial period.

That said, I think step one would be to figure out where your comparative advantage lies (80,000 hours folk may have thoughts, among others). Certainly some people should be upskilling in ML/CS/Math - though an advanced degree may not be most efficient -, but there are other ways to help.

I realize this doesn't address the deciding-what's-true aspect.
I'd note there that I don't think much detailed ML knowledge is necessary to follow Eliezer's arguments on this. Most of the ML-dependent parts can be summarized as [we don't know how to do X], [we don't have any clear plan that we expect will tell us how to do X], similarly for Y, Z, [Either X, Y or Z is necessary for safe AGI].

Beyond that, I think you only need a low prior on our bumping into a good solution while fumbling in the dark and a low prior on sufficient coordination, and things look quite gloomy. Probably you also need to throw in some pessimism on getting safe AI systems to fundamentally improve our alignment research.

to maximise my chance to die with dignity I should quit my job and take out a bunch of loans try to turbo through an advanced degree in machine learning

This is probably pretty tangential to the overall point of your post, but you definitely don't need to take loans for this, since you could apply for funding from Open Philanthropy's early-career funding for individuals interested in improving the long-term future or the Long-Term Future Fund.

You don't have to have a degree in machine learning. Besides machine learning engineering or machine learning research there are plenty of other ways to help reduce existential risk from AI, such as:

• software engineering at Redwood Research or Anthropic
• independent alignment research [AF · GW]
• operations for Redwood Research, Encultured AI, Stanford Existential Risks Initiative, etc.
• community-building work for a local AI safety group (e.g., at MIT or Oxford)
• AI governance research
• or something part-time like participating in the EA Cambridge AGI Safety Fundamentals program and then facilitating for it

Personally, my estimate of the probability of doom is much lower than Eliezer's, but in any case, I think it's worthwhile to carefully consider how to maximize your positive impact on the world, whether that involves reducing existential risk from AI or not.

I'd second the recommendation for applying for career advising from 80,000 Hours or scheduling a call with AI Safety Support if you're open to working on AI safety.

I can't help with the object level determination, but I think you may be overrating both the balance and import of the second-order evidence.

As far as I can tell, Yudkowsky is a (?dramatically) pessimistic outlier among the class of "rationalist/rationalist-adjacent" SMEs in AI safety, and probably even more so relative to aggregate opinion without an LW-y filter applied (cf. [LW(p) · GW(p)]). My impression of the epistemic track-record is Yudkowsky has a tendency of staking out positions (both within and without AI) with striking levels of confidence but not commensurately-striking levels of accuracy. 

In essence, I doubt there's much epistemic reason to defer to Yudkowsky more (or much more) than folks like Carl Shulman, or Paul Christiano, nor maybe much more than "a random AI alignment researcher" or "a superforecaster making a guess after watching a few Rob Miles videos" (although these have a few implied premises around difficulty curves/ subject matter expertise being relatively uncorrelated to judgemental accuracy). 

I suggest ~all reasonable attempts at idealised aggregate wouldn't take a hand-brake turn to extreme pessimism on finding Yudkowsky is. My impression is the plurality LW view has shifted more from "pretty worried" to "pessimistic" (e.g. p(screwed) > 0.4) rather than agreement with Yudkowsky, but in any case I'd attribute large shifts in this aggregate mostly to Yudkowsky's cultural influence on the LW-community plus some degree of internet cabin fever (and selection) distorting collective judgement.

None of this is cause for complacency: even if p(screwed) isn't ~1, > 0.1 (or 0.001) is ample cause for concern, and resolution on values between (say) [0.1 0.9] is informative for many things (like personal career choice). I'm not sure whether you get more yield for marginal effort on object or second-order uncertainty (e.g. my impression is the 'LW cluster' trends towards pessimism, so adjudicating whether this cluster should be over/under weighted could be more informative than trying to get up to speed on ELK). I would guess, though, that whatever distils out of LW discourse in 1-2 months will be much more useful than what you'd get right now.

I'm sympathetic to the position you feel you're in. I'm sorry it's currently like that.

I think you should be quite convinced by the point you're taking out loans to study, and that the apparent plurality of the LessWrong commentariat is unlikely to be sufficient evidence to reach that level of convincement – just my feeling.

I'm hoping some more detailed arguments for doom will be posted in the near future and that will help many people reach their own conclusions not based on information cascades, etc.

Lastly, I do think people should be more "creative" in finding ways to boost log odds of survival. Direct research might make sense for some, but if you'd need to go back to the school for it, there are maybe other things you should brainstorm and consider.

Well, at least one of us thinks you're going to die and to maximize your chance to die with dignity you should quit your job, say bollocks to it all, and enjoy the sunshine while you still can!

Don't look at opinions, look for data and facts.  Speculations, opinions or beliefs cannot be the basis on which you take decisions or update your knowledge. It's better to know few things, but with high confidence. 

Ask yourself, which hard data points are there in favour of doom-soon? 

Note that an observer must always see that all previous prophets of existential doom were wrong. (Yay anthropics. I'm not sure if or how much this should change our reasoning though.)

That's certainly the way to bet!