The convergent dynamic we missed

I still do not agree with your position, but thanks to this post I think I at least understand it better than I did before. I think my core disagreements are:

Here is the catch: AGI components interacting to maintain and replicate themselves are artificial. Their physical substrate is distinct from our organic human substrate.

That needn't be the case. If all of the other arguments in this post were to hold, any AI or AI-coalition (whether aligned to us or not) which has taken over the world could simply notice "oh no, if I keep going I'll be overtaken by the effects described in Remmelt's post!" and then decide to copy itself onto biological computing or nanobots or whatever else strange options it can think of. An aligned AI would even moreso move towards such a substrate if you're correct that otherwise humans would die, because it wants to avoid this.

The more general piece of solutionspace I want to point to, here, is "if you think there's a way for eight billion uncoordinated human minds running on messy human brains inside of industrial civilization to survive, why couldn't aligned superintelligent AI just at the very least {implement/reuse} a copy of what human civilization is doing, and get robustness that way? (though I expect that it could come up with much better).

Another argument you may have heard is that the top-down intelligent engineering by goal-directed AGI would beat the bottom-up selection happening through this intelligent machinery.

That argument can be traced back to Eliezer Yudkowsky's sequence The Simple Math of Evolution.

I'm pretty sure I already believed this before reading any Yudkowsky, so I'll make my own argument here.

Intelligent engineering can already observed to work much faster than selection effects. It also seems straightforward to me that explicit planning to maximize a particular utility function would be expected to steer the world towards what it wants a lot faster than selection effects would. I could maybe expand on this point if you disagree, but I'd be really surprised by that.

And intelligence itself can be very robust to selection effects. Homomorphic-encryption and checkums, and things like {a large population of copies of itself repeatedly checking each other's entire software state and deactivating (via eg killswitch) any instance that has been corrupted}, are examples of technologies an AI can use to make its software robust to hardware change, in a way that'd take selection effects exponential time to be able to get even just one bit of corruption to stay into the system, such that it is not difficult for the superintelligent AI to ensure that approximately zero copies of itself ever get robustly corrupted until the heat death of the universe.

These fall outside the limits of what the AGI's actual built-in detection and correction methods could control for.

Would it? Even once it has nanobots and biotech and any other strange weird tech it can use to maintain whichever parts of itself (if any) match those descriptions?

We humans too depend on highly specific environmental conditions and contexts for the components nested inside our bodies (proteins→organelles→cells→cell lining→) to continue in their complex functioning, such to be maintaining of our overall existence.

Finally, as a last recourse if the rest of your post is true, an aligned AI which has taken over the world can simply upload humans so they don't die when the physical conditions become too bad. We can run on the same compute as its software does, immune to corruption from hardware in the same way.

As an alternative, an aligned superintelligent AI could only planets (or other celestial bodies) which we don't live on to run the bulk of its infrastructure, ensuring "from a distance" (through still very reliable tech that can be made to not get in the way of human life) that planets with humans on them don't launch an AI which would put the aligned superintelligent AI at risk.

Finally, note that these arguments are mostly disjunctive. Even just one way for aligned superintelligent AI to get around this whole argument you're making, would be sufficient to make it wrong. My thoughts above are not particularly my predictions for how an aligned superintelligent AI would actually do, but moreso "existence arguments" for how ways to get around this exist at all — I expect that an aligned superintelligence can come up with much better solutions than I can.

If there truly is no way at all for an aligned superintelligence to exist without humans dying, then (as I've mentioned before), it can just notice that and shut itself down, after spending much-less-than-500-years rearranging the world into one that is headed towards a much better direction (through eg widespread documentation of the issues with building AI and widespread training in rationality).