Precise P(doom) isn't very important for prioritization or strategy

Speaking as someone who does work on prioritization, this is the opposite of my lived experience, which is that robust broadly credible values for this would be incredibly valuable, and I would happily accept them over billions of dollars for risk reduction and feel civilization's prospects substantially improved.

 These sorts of forecasts are critical to setting budget and impact threshold across cause areas [EA(p) · GW(p)], and even more crucially, to determining the signs of interventions, e.g. in arguments about whether to race for AGI with less concern about catastrophic unintended AI action, the relative magnitude of the downsides of unwelcome use of AGI by others vs accidental catastrophe is critical to how AI companies and governments will decide how much risk of accidental catastrophe they will take, how AI researchers decide whether to bother with advance preparations, how much they will be willing to delay deployment for safety testing, etc.

Holden Karnofsky discusses [EA · GW] this:

How difficult should we expect AI alignment to be? In this post from the Most Important Century series, I argue that this broad sort of question is of central strategic importance.

• If we had good arguments that alignment will be very hard and require “heroic coordination,” the EA funders and the EA community could focus on spreading these arguments and pushing for coordination/cooperation measures. I think a huge amount of talent and money could be well-used on persuasion alone, if we had a message here that we were confident ought to be spread far and wide.
• If we had good arguments that it won’t be, we could focus more on speeding/boosting the countries, labs and/or people that seem likely to make wise decisions about deploying transformative AI. I think a huge amount of talent and money could be directed toward speeding AI development in particular places.

My understanding is that humanity is like a person stuck in a car whose brakes have failed, plummeting down a steep road towards a cliff. Timelines are about figuring out whether we have 1 minute to solve our dilemma or 10 days. The difference is very relevant indeed to what actions we strategically attempt in our limited time. Here's my post about that: https://www.lesswrong.com/posts/wgcFStYwacRB8y3Yp/timelines-are-relevant-to-alignment-research-timelines-2-of [LW · GW] P(doom), given the situation, is primarily about two questions: will our strategic action succeed? Will we survive the crash? I agree that, given we have accurate timelines and thus make correct strategic choices about which actions to attempt, the probability of our actions suceeding isn't relevant. We must do our best. The probability that we will die if the car goes over the edge of the cliff is relevant only insofar as it motivates us to take action. We only need agreement that it is large enough to be worth worrying about, precision is irrelevant. Currently, lots of people who could potentially be taking helpful action are instead actively making the problem worse by working on capabilities. Talking about our reasoning for our personal estimates of p(doom) is useful if and only if it helps sway some potential safety researchers into working on safety, or some capabilities researchers into stopping work on capabilities.

Precision beyond order-of-magnitude probably doesn't matter.  But there's not much agreement on order-of-magnitude risks.  Is it 1% in the next 10 years or next 80?  Is it much over 1%, or closer to 0.1%.   And is that conditional on other risks not materializing?  Why would you give less than 1% to those other risks (my suspicion is you don't think civilization collapse or 90% reduction in population is existential, which is debatable on it's own).

And even if it IS the most important (but still very unlikely) risk, that doesn't make it the one with the highest-EV to work on or donate to.   You need to multiply by the amount of change you think you can have.