So if a UFO lands in your backyard and aliens ask if you if you want to go on a magical (but not particularly instrumental) space adventure with them, I think it's reasonable to very politely decline, and get back to work solving alignment.

I think I'd probably go for that, actually, if there isn't some specific reason to very strongly doubt it could possibly help? It seems somewhat more likely that I'll end up decisive via space adventure than by mundane means, even if there's no obvious way the space adventure will contribute.

This is different if you're already in a position where you're making substantial progress though.

Mm, I agree with the detailed scenarios outlined, but I think there are some anomalous worlds the correct policy in which is to switch from working on AI Alignment: worlds in which there's a shorter avenue to faithful cognitive augmentation/creating an utopia/such stuff. E. g., if we discover a very flexible magic system, then abandoning all your agency-research stuff and re-specializing towards figuring out how to design a Becomus Goddus spell in it may be the correct move. Or if the Matrix Lord comes down and explicitly sends you off on a heroic quest the reward for completing which will be control over the simulation.

Which also means you'd need to investigate the anomalous occurrences first, so that you may do your due diligence and check that they don't contain said shorter avenues (or existential threats more pressing than AGI).

Overall, I agree that there's a thing you need to single-mindedly pursue across all possible worlds: your CEV. Which, for most people, likely means an eutopia. And in this world, the shortest and most robust route there seems to be solving AGI Alignment via agency-foundations research. Sure. But it's not a universal constant.

(I mean, I suppose it kind of all does ultimately bottom out in you causing a humanity-aligned superintelligence to exist (so that it can do acausal trade for us). But the pathways there can look very different, and not necessarily most-accurately described as "working on solving alignment".)

Someone may think "Anomalous worlds imply the simulation-runners will save us from failing at alignment!"

My reply is: Why are they running a simulation where we have to solve alignment?

At a first pass, if we're in a simulation, it's probably for research, rather than e.g. a video game or utopia. (H/t an IRL friend for pointing this out).

Therefore, if we observe ourselves needing to solve AI alignment (and not having solved it yet), the simulation-runners potentially also need AI alignment to get solved. And if history is any guide, we should not rely on any such beings "saving us" before things cross a given threshold of badness.

(There are other caveats I can respond to about this, but please DM me about them if you think of them, since they may be infohazard-leaning and (thus) should not be commented publicly, pending more understanding.)