so, my claim we should find the fact that the problem and the solution has a lot of research in common (studying powerful AI) is a weird interesting fact, and we should generally assume that the research that is involved in the problem won't particularly be helpful to research that is involved in the solution — for example, if FAS is made in a way that is pretty different from current AI or XAI.

This essay seemed straightforward until this last paragraph, because: the "problem and solution has alot of research in common" seems to directly contradict "research that is involved in the problem won't particularly be helpful to research that is involved in the solution".

I think I can predict what you intended based on " FAS is made in a way that is pretty different from current AI or XAI", but that's a low probability miracle at this point. The difference between FAS and XAI is just one of utility functions; both will need to be built on powerful efficient approximations of bayesian model-based planning agents, the structure of which is dictated by physics and ends up looking like advanced neural nets as that is simply what efficient approximate bayesian inference over circuit space entails.