Still Not in Charge

post by Zvi · 2021-02-09T16:00:01.474Z · LW · GW · 24 comments

Contents

24 comments

Epistemic Status: Speed premium, will hopefully flesh out more carefully in future

Previously: Why I Am Not in Charge

A brief update about my exchange with Scott from this past week.

After Scott Alexander wrote a post about why WebMD is terrible and it would be good but impossible to put me in charge of the CDC, which is super flattering and unsurprisingly blew up my inbox, I wrote a quick response post to expand on things, and give more color on my model of the dynamics involved in organizations like the CDC and FDA, my view of how people in my position can often get reliably ahead of such organizations, and what would happen if one tried to change them and get them to do the most useful things. That required tying things back to several key past posts, including Zeroing Out, Leaders of Men, Asymmetric Justice, Motive Ambiguity, and the sequences on Simulacra and Moral Mazes.

The disagreements between my model and Scott’s, and the places in which my communication attempts fell short, are broadly in two (closely linked) categories, which his Reddit response captured well and made clearer to me, as did other reactions on the same thread.

The first category is where claims about how perverse things are get rounded down and not seen. Scott is advocating here for what we might call the Utility Function Hypothesis (UFH). 

The second is (as Scott explicitly notes) the generalized form of the Efficient Market Hypothesis, which one might call the Efficient Action Hypothesis (EAH). 

UFH and EAH are linked.

If UFH is impactfully false, then EAH is also false. If you’re acting meaningfully distinctly from how you would act with a coherent utility function, you are leaving money on the table.

If UFH is effectively true (e.g. it is not impactfully false), then EAH is plausible, but can still either be true or false. EAH could still be false if either one could have a better model of the impact of decisions than those making the decisions (this could be better political sense, better physical-impact sense that then impacts the political calculus, or both). EAH could also still be false if the utility function being maximized doesn’t square up with what we’d mean by politically successful.

If the EAH is true, then UFH is effectively true as well. Anyone acting in a way that can’t be improved upon isn’t meaningfully different from how they’d be with a utility function.

If the EAH is false, then that’s strong evidence that UFH is also false (since EAH -> UFH), but UFH could still be true if the EAH is false due to our having superior information, especially superior information about physical impacts.

On the first disagreement, I think we can look at the decisions in detail and find evidence for who is right. The utility function hypothesis expects to find sensible trade-offs being made, with bad decisions only being made when good information was unavailable or political pressure was stronger than the physical stakes. We can ask ourselves if we see this pattern. The difficulty is that the political pressures are often invisible to us, or hard to measure in magnitude. But if it’s a matter of battling real interests, we should expect political pressure to generally push in favor of useful actions for particular interests, rather than for perversity in general. 

Another way of looking at the question is, do such folks seem to be goal optimizers or adaptation executors? To the extent that actions seem to reflect goals, including political ones, what kind of time horizon and discount rate seem to be in play? 

We can also ask whether making destructive decisions seems to correlate with political profit. The issue here is that both sides have competing hypotheses that explain this correlation. The UFH says this is because of trade-offs. The alternate hypothesis says that this happens because there has been selection for those who act as if they need to be careful not to be seen favoring the constructive over the destructive because it is constructive.  

What distinguishes these claims is that the UFH thinks that when the explicit political stakes are low then constructive decisions will dominate, whereas the alternate hypothesis thinks that there can be no explicit stakes at all and the destructive decisions still happen, because the mechanism causing them is still functioning. 

On the second disagreement, my claim is that if you could execute with an ordinary level of tactical political savvy and not make continuous unforced errors, except when it mattered you used my judgment to make constructive decisions and implement constructive policies, that this would have a good chance of working out, especially if applied to the FDA along with the CDC.

My claim is importantly not that putting me in as CDC director tomorrow would have a good chance of working out, which it most certainly wouldn’t because I wouldn’t have the ordinary skill in the art necessary to not have the whole thing blow up for unrelated reasons.

But again, I don’t need to be a better overall judge of what is politically savvy and be able to execute the entire job myself to point to a better path, I only need to make improvements on the margin, even if that margin is relatively wide and deep. I can claim the EAH is importantly false and point to big improvements without being able to reimplement the rest of the system.

In particular, don’t make me head of the FDA, that’s crazy, but do appoint my father Solomon Mowshowitz as head of the FDA, give us some rope, and watch what happens. Scott was talking about the director of the CDC, which I’d also accept, but I think you can have a lot more impact right now at the FDA. 

Why do I think a lot of politicians are leaving money on the ground for ‘no reason’, other than “Donald Trump spent four years as President of the United States’? 

First, my model is that politicians mostly aren’t goal maximizers with utility functions, they’re adaptation executors who have developed systems that have been shaped to seek power.  Those adaptations lead to political success. They’ve been heavily selected for those attributes by people looking for those attributes. One of the adaptations is to be visibly part of this selection process. Another is avoiding displaying competing loyalties, such as caring about money being on the ground enough to both see the money on the ground and then pick it up. 

Second, the politicians don’t directly know what would and wouldn’t work out, and have been selected for not thinking it is possible to know such things. To the extent they try to do things that would work out, they approximate this by having a mechanism that avoids being blamed for things within the next two weeks, which is then subject to back propagation by others. If you do something with bad consequences next year or next month, if it’s bad enough the hope is that other people notice this and get mad about it now, you notice that and so you choose differently. The advantage of this approach is that Avoid Blame is a Fully General Excuse for action (or even better, for inaction), so it doesn’t cause suspicion that you prefer constructive to destructive action, or think you can tell the difference. 

Third, this is all outside of the training data that the politicians learned on. Everyone involved is trained on data where the feedback loops are much longer, and the physical impacts are slow and far away. It hasn’t sunk in that this is a real emergency, and that in an emergency the rules are different. One can think of this as a battle between perception and reality, to see who can get inside whose OODA loop. People in mazes (and everyone involved here is in a maze) are used to making destructive decisions and then either not having the consequences rebound back at all, or being long gone before the physical consequences can catch up with them. Also, a lot of these learned behaviors go back to previous times without rapid social media feedback loops and other ways for us to distinguish constructive from destructive action, or identify lies or obvious nonsense. Back then there was more elite control over opinion and more reason to worry greatly about being a Very Serious Person who properly demanded their Sacrifices to the Gods, and consequences were less likely to back propagate into current blame and credit assignments. 

Fourth, Leaders of Men, but also imposter syndrome and the illusion of competence. Everyone is everywhere and always muddling through far more than anyone realizes. Always have been. They make dumb mistakes and unforced errors, they overlook giant opportunities, they improvise and act confident and knowledgable. How many political campaigns does one need to watch unforced error after unforced error, to say ‘how did we end up with all these horrible choices and no good ones?’ and to watch candidates be woefully unprepared time and time again, before one realizes that this is the normal state of affairs? We’re choosing the people who were in the right place at the right time with the skills that most impact ability to raise money and campaign, not the people who are the best at governance. There is a dramatic difference in effectiveness levels between different politicians and leaders, not only in government but also in other areas. You take what you can get. And again, we’re aiming at improving on the margin, and it would be pretty shocking if there weren’t large marginal improvements available that we could spot if we tried. 

Fifth, because they’re not properly modeling the shifts that occur when policy changes. The people who get to move elite opinion, and hence move ‘expert’ opinion, don’t realize this is within their power. Again, each time there was a fight over a shift from a destructive policy stance or claim to a constructive one in the pandemic, once the shift was made, most of the ‘experts’ saying otherwise fell in line immediately. At maximum, they nominally complained and then kept quiet. It’s almost like they’re not offering their expertise except insofar as they use it to back up what elites decided to tell us.  

It’s an interesting question to what extent that mechanism doesn’t work when the new decision is destructive, but again we have data on that, so think back and form your own opinion on who would push back how much on such questions. 

You could also respond that the constructive changes were chosen and timed exactly in order to be politically beneficial, and thus this isn’t a fair test. There’s certainly some selection effects but if you compare the results to a naive prior or to the prediction of the trade-off model, I think you’ll notice a big difference.

Sixth, because bandwidth is limited. Politicians aren’t looking on the sidewalk for the bill, so they don’t notice it and therefore don’t pick it up. When you are a powerful person there’s tons of things to do and no time to do them, whatever your combination of avoiding blame, executing adaptations, cashing in, gathering power and trying to do the most good as you see it. Everyone trying to contact you and give you ideas has an agenda of some form. Getting the good ideas even on the radar screen is hard even when you have a relatively competent and well-meaning group of people. 

Seventh, this is a known blind spot, where there is reliably not enough attention to satisfying the real needs of voters and giving them what they care about, and not doing this reliably loses people power, while satisfying such needs reliably gets rewarded. This is true for things that voters are right about, and also things voters are wrong or selfish about. 

Eighth, if a process filters out actions by making them unthinkable to anyone with the power to execute on them, partly by filtering who gets power and partly by getting those who seek power to self-modify, and thus such people never think seriously about them, or have lots of people whose job it is when they accidentally do think them to point out how unthinkable they are via what are usually essentially Beware Trivial Inconveniences [LW · GW] arguments, it’s hard to turn around and call not taking those actions evidence that those actions wouldn’t work if someone actually did them.

Lastly, because ‘there’s a good chance this would actually work out’ does not translate to free money on the ground. The political calculus is not ‘free money’ here, it’s ‘if this works sufficiently well, you reap large political benefits that outweigh your costs.’ You’d be betting on an alternate mechanism kicking in. Doing a different thing that isn’t as legible puts one very open to large amounts of blame via Asymmetric Justice, and inherently feels terrible for those trained on such data. None of this looks like a safe play, even if a lot of it on the margin is safe. Doing the full throttle version would definitely not be safe. 

In general, rather than look at this as a ‘all trading opportunities are bad or someone would have taken them already’ or ‘if the fence should have been taken down then the fence-removal experts would have already taken care of it,’ look at this as the “How are you f***ing me?” or Chesterson’s Fence question. Before you take down a fence you need to know why someone put it up. Before you do a trade, you need to know the reason why you have the opportunity to do this trade. We see politicians failing to do seemingly obviously correct things that would be physically beneficial to people and look like they would be popular and net them political capital, so we need an explanation for why they’re not acting on it.

We have plenty of interlocking explanations for why they’re not acting on it. That doesn’t mean that any given instance doesn’t have additional good explanations, including explanations that could plausibly carry the day. And some of the explanations given here are reasonable reasons to not do some of the things, including the pure ‘there are people who don’t want you to do that, and they pressure you, and that sucks and raises the cost of acting an amount it’s hard for us to measure.’  

As for the pure modesty argument, that I am not a political expert and thus shouldn’t expect to be able to predict what will win political capital, the response is in two parts.

First, I’m also not a medical or biological expert, yet here we are. I fully reject the idea that smart people can’t improve on the margin on ‘expert’ opinion, period. Welcome to 2021. Modesty shmodesty. 

Second, much of the difference is in our physical world models and not our political models. To fully model politics, one must fully understand the world. 

I don’t think this fully did justice to the questions involved. That will require several posts that I’ve been working on for a while in one form or another and are difficult to write. This did make writing those posts easier, so there is hope.

24 comments

Comments sorted by top scores.