Design policy to be testable

Seems cut off.  But the theme seems to be a lack of Hansonian understanding that policy debate isn't about finding the best policies.  It's about power and appearing to help those who give you the power.  

In small groups and businesses, testable policies and practices rule - they care about the results, not (as much) about the public reaction.  In public and large-non-hierarchical groups, testable policy is a mistake, because you'll be blamed if the test shows that it's wrong.  The best policies are those that are hard to argue against, and that show you care about your constituents.  

This post seems to have cut off in the middle?

Level 1: “There’s a lion across the river.” = There’s a lion across the river.

Level 2: “There’s a lion across the river.” = I don’t want to go (or have other people go) across the river

Level 3: “There’s a lion across the river.” = I’m with the popular kids who are too cool to go across the river.

Level 4: “There’s a lion across the river.” = A firm stance against trans-river expansionism focus grouped well with undecided voters in my constituency.

Source: https://www.lesswrong.com/posts/qDmnyEMtJkE9Wrpau/simulacra-levels-and-their-interactions [LW · GW]

The optimization you are describing is strongly rooted in level 1. There's a clear image of a before/after state along with a feedback loop for correction. I presume this speaks to many people in the LW community, along with good project managers, engineers, etc., anyone really who really wants to change something in the world, whether it's big or small.

However, I think politicians and the bureaucrats that work for them (or vice versa?) are on levels 3 and 4. Having metrics and using them to score policies would only get in the way of signaling group affiliation or building consensus. This is just another way of saying that often, the real impact a decision has is of less importance than how the people making the decision look and act.

That said, I've found a similar technique mentioned in productivity circles. Perhaps it was in a book by Peter Drucker. The gist was: When making a decision, make to include criteria which tell you whether it's working or not.

All very interesting - there are things called impact assessments which are used for legislation to predict and justify impact - the Westminster Parliament does not insist on impact assessments for Bills - statutory instruments normally have them - the devolved legislatures normally have them - they carry from almost the kind of specific modelling you are talking about to something more generic and less helpful from your perspective.

On of the key goals of politicians is to avoid blame. If you would create policy is upfront criteria of how to measure how effective the policy is that opens up room for the policy maker to be blamed. 

It would be great to have more information about which policies work but the key problem is to think through the incentives of how you could create a system where that works. 

I think this will have a number of benefits:

Since you're proposing a policy here, I feel like someone needs to ask: how would you measure and test these benefits?

They seem plausible to me, but I could tell just-so stories where this goes wrong. Like:

• Some amount of leaving-things-unsaid turns out to be necessary for humans to cooperate.
• Policy is just too complicated. Prediction markets rarely have any idea whether a bill will succeed at its stated impact measures.
• A non partisan group of scientists isn't enough for us to break Goodhart's law. We can improve the things we measure, but the things we can't or don't measure turn to shit.

There's a bunch of criticisms here that politicians/politics don't work that way, you haven't considered incentives, and that may be true, but...

Everybody knows [LW · GW] feels relevant here, plus that thing Scott's described where (IIRC, something along the lines of) he goes from "those people seem crazy" over a few steps to "oh, they were right, but they're being unsophisticated about how they say it".

Like, maybe this is unworkable because politicians aren't actually trying to cause good outcomes. But even if that's the case, they still generally pretend they're trying to cause good outcomes. Making that pretense marginally less believable seems marginally good. And so it seems good to talk about "here's some ways politicians could more reliably cause good outcomes".

I like the general idea, policy that is drafted with clear statements about what it does, why it's doing things they way it's designed and what the desired outcomes are aimed for. It seems to be somewhat similar to cases where the courts have thrown laws back to the law makers because they are not clear enough for legal enforcement by the courts. However that is a really poor initial check as something first has to get to a court and then go in front of a judge that is willing to call BS on some legislation or regulation.

I think there is a lot of room for improving the legislative processes -- certainly in the USA where I am. But I also agree with those noting that in many ways such structures or process requirements are not really aligned with now politics and policy setting actually work. But it seems like that might be one of the shackles we might want Leviathan to have to wear.