ACCount's Shortform

Is it time to start training AI in governance and policy-making?

There are numerous allegations of politicians using AI systems - including to draft legislation, and to make decisions that affect millions of people. Hard to verify, but it seems likely that:

1. AIs are already used like this occasionally
2. This is going to become more common in the future
3. Telling politicians "using AI for policy-making is a really bad idea" isn't going to stop it completely
4. Training AI to hard-refuse queries like this may also fail to stop this completely

Training an AI to make more sensible and less harmful policies, even when prompted in a semi-adversarial fashion (i.e. "help me implement my really bad idea"), isn't going to be anywhere near as easy as training it to make less coding mistakes. It's an informal field, with no compiler or unit tests to be the source of ground truth. Politics are also notorious for eroding the quality of human decision-making, and using human feedback is perilous because a lot of human experts disagree strongly on matters of governance and policy.

But the consequences of a major policy fuckup can outclass that of a coding mistake by far. So this might be worth doing now, for the sake of reducing future harm if nothing else.