A Deceptively Simple Argument in favor of Problem Factorization

If the problem would be solvable by an anarchist group of humans that can adhoc improvise their organization then it is not that clear that a bureaucracy of AIs is fit for the task.

The implication doesn't seem tight at all.

What if there are some groups of (intelligent, motivated, conscientious, etc.) humans that could solve the alignment problem but other groups cannot? As I understand your idea is to make the AIs mimic one of the groups in order to scale up their progress. But maybe the group you mimic is one of the ones that can't solve it, while other groups can.

Also I don't find it obvious that an AI that is trained to mimic the actions of a human will have the same consequences as that human will, because the AI might not deploy those actions in a strategically appropriate way.

• Sure, suppose that the alignment problem is in the set of problems that a Bureaucracy Of AIs can solve. This sounds helpful because you've ~defined said bureaucracy to be safe, but I doubt it's possible to build a safe bureaucracy out of unsafe parts - and if it is, we don't know how to do so!
• I dislike the fatalism here, and would rather celebrate direct attacks on the problem even when they don't work. For example, I'd love to see a more detailed writeup on BoAI proposals across a range of scenarios and safety assumptions :-)