Gears Level & Policy Level

post by abramdemski · 2017-11-24T07:17:51.525Z · LW · GW · 8 comments

Contents

  Issues with Inside vs Outside
  The Gears Level and the Policy Level
  Gears-Leves and Policy-Level Are Not Opposites
None
8 comments

Inside view vs outside view has been a fairly useful intuition-pump for rationality. However, the dichotomy has a lot of shortcomings. We've just gotten a whole sequence about failures of a cluster of practices called modest epistemology, which largely overlaps with what people call outside view. I'm not ready to stop championing what I think of as the outside view. However, I am ready for a name change. The term outside view doesn't exactly have a clear definition; or, to the extent that it does have one, it's "reference class forecasting", which is not what I want to point at. Reference class forecasting has its uses, but many problems have been noted.

I propose gears level & policy level. But, before I discuss why these are appropriate replacements, let's look at my motives for finding better terms.

Issues with Inside vs Outside

Problems with the concept of outside view as it currently exists:

• Reference class forecasting tends to imply stopping at base-rate reasoning, rather than starting at base-rate reasoning. I want a concept of outside view which helps overcome base-rate neglect, but which more obviously connotes combining an outside view with an inside view (by analogy to combining a prior probability with a likelihood function to get a posterior probability).
• Reference class forecasting lends itself to reference class tennis, IE, a game of choosing the reference class which best makes your point for you. (That's a link to the same article as the previous bullet point, since it originated the term, but this Stuart Armstrong article also discusses it. Paul Christiano discusses rules and ettiquete of reference class tennis, because of course he does.) Reference class tennis is both a pretty bad conversation to have, which makes reference class forecasting a poor choice for productive discussion, and a ponentially big source of bias if you do it to yourself. It's closely related to the worst argument in the world.
• Reference class forecasting is specified at the object level: you find a class fitting the prediction you want to make, and you check the statistics for things in that class to make your prediction. However, central examples of the usefulness of the outside view occur at the meta level. In examples of planning-fallacy correction, you don't just note how close you usually get to the deadline before finishing something. You compare it to how close to the deadline you usually expect to get. Why would you do that? To correct your inside view! As I mentioned before, the type of the outside view should be such that it begs combination with the inside view, rather than standing on its own.
• Outside view has the connotation of stepping back and ignoring some details. However, we'd like to be able to use all the information at our disposal -- so long as we can use it in the right way. Taking base rates into account can look like ignoring information: walking by the proverbial hundred-dollar bill on the gorund in times square, or preparing for a large flood despite there being none in living memory. However, while accounting for base rates does indeed tend to smooth out behavior and make it depend less on evidence, that's because we're working with more information, not less. A concept of outside view which connotes bringing in more information, rather than less, would be an improvement.

The existing notion of inside view is also problematic:

• The inside-view vs outside-view distinction does double duty as a descriptive dichotomy and a prescriptive technique. This is especially harmful in the case of inside view, which gets belittled as the naive thing you do before you learn to move to outside view. (We could similarly malign the outside view as what you have before you have a true inside-view understanding of a thing.) On the contrary, there are significant skills in forming a high-quality inside view. I primarily want to point at those, rather than the descriptive cluster.

The Gears Level and the Policy Level

Gears-level understanding is a term from CFAR, so you can't blame me for it. Well, I'm endorsing it, so I suppose you can blame me a little. In any case, I like the term, and I think it fits my purposes. Some features of gears-level reasoning:

• Dishing out probability mass precisely, so as to have the virtue of precision.
• Having the properties of a good explanation, along the lines of David Deutsch: being pinned down on all sides by the evidence, and providing understanding, not only predictive accuracy. (Contrast this concept with a big neural-net model which classifies images extremely well but is difficult to analyse.)
• Reasoning from first principles, rather than analogy.
• Making a prediction with a well-defined model, such that anyone who understood your model could calculate the same prediction independently.

The policy level is not a CFAR concept. It is similar to the CFAR concept of the strategic level, which I suspect is based on Nate Soares' Staring Into Regrets. In any case, here are some things which point in the right direction:

• Placing yourself as an instance of a class.
• Accounting for knock-on effects, including consistency effects. Choosing an action really is a lot like setting your future policy.
• What game theorists mean by policy: a function from observations to actions, which is (ideally) in equilibrium with the policies of all other agents. A good policy lets you coordinate sucessfully with yourself and with others. Choosing a policy illustrates the idea of choosing at the meta level: you aren't selecting an action, but rather, a function from situations to actions.
• Timeless decision theory / updateless decision theory / functional decision theory. Roughly, choosing a policy from behind a Rawlsian veil of ignorance. As I mentioned with accounting for base rates, it might seem from one perspective like this kind of reasoning is throwing information away; but actually, it is much more powerful. It allows you to set up arbitrary functions from information states to strategies. You are not actually throwing information away; you always have the option of responding to it as usual. You are gaining the option of ignoring it, or reacting to it in a different way, based on larger considerations.
• Cognitive reductions, in Jessica Taylor's sense (points five and six here). Taking the outside view should not entail giving up on having a gears-level model. The virtues of good models at the gears level are still virtues at the policy level. Rather, the policy level asks you to make a gears-level model of your own cognitive process. When you go to the policy level, you take your normal way of thinking and doing as an object. You think about the causes and effects of your normal ways of being.

Most of the existing ideas I can point to are about actions: game theory, decision theory, the planning fallacy. That's probably the worst problem with the terminology choice. Policy-level thinking has a very instrumental character, because it is about process. However, at its core, it is epistemic. Gears level thinking is the practice of good map-making. The output is a high-quality map. Policy-level thinking, on the other hand, is the theory of map-making. The output is a refined strategy for making maps.

The standard example with the planning fallacy illustrates this: although the goal is to improve planning, which sounds instrumental, the key is noticing the miscalibration of time estimates. The same trick works for any kind of mental miscalibration: if you know about it, you can adjust for it.

This is not just reference class forecasting, though. You don't adjust your time estimates for projects upward and stop there. The fact that you normally underestimate how long things will take makes you think about your model. "Hm, that's interesting. My plans almost never come out as stated, but I always believe in them when I'm making them." You shouldn't be satisfied with this state of affairs! You can slap on a correction factor and keep planning like you always have, but this is a sort of paradoxical mental state to maintain. If you do manage keep the disparity between your past predictions and actual events actively in mind, I think it's more natural to start considering which parts of your plans are most likely to go wrong.

If I had to spell it out in steps:

1. Notice that a thing is happening. In particular, notice that a thing is happening to you, or that you're doing a thing. This step is skipped in experiments on the planning fallacy; experimenters frame the situation. In some respects, though, it's the most important part; naming the situation as a situation is what lets you jump outside of it. This is what lets you go off-script, or be anti-sphexish.
2. Make a model of the input-output relations involved. Why did you say what you just said? Why did you think what you just thought? Why did you do what you just did? What are the typical effects of these thoughts, words, actions? This step is most similar to reference class forecasting. Figuring out the input-output relation is a combination of refining the reference class to be the most relevant one, and thinking of the base-rates of outcomes in the reference class.
3. Adjust your policy. Is there a systematic bias in what you're currently doing? Is there a risk you weren't accounting for? Is there an extra variable you could use to differentiate between two cases you were treating as the same? Chesterton-fencing your old strategy is important here. Be gentle with policy changes -- you don't want to make a bucket error or fall into a hufflepuff trap. If you notice resistence in yourself, be sure to leave a line of retreat by visualizing possible worlds. (Yes, I think all those links are actually relevant. No, you don't have to read them to get the point.)

I don't know quite what I can say here to convey the importance of this. There is a skill here; a very important skill, which can be done in a split second. It is the skill of going meta.

Gears-Leves and Policy-Level Are Not Opposites

The second-most confusing thing about my proposed terms is probably that they are not opposites of each other. They'd be snappier if they were; "inside view vs outside view" had a nice sound to it. On the other hand, I don't want the concepts to be opposed. I don't want a dichotomy that serves as a discriptive clustering of ways of thinking; I want to point at skills of thinking. As I mentioned, the virtuous features of gears-level thinking are still present when thinking at the policy level; unlike in reference class forecasting, the ideal is still to get a good causal model of what's going on (IE, a good causal model of what is producing systematic bias in your way of thinking).

The opposite of gears-level thinking is un-gears-like thinking: reasoning by analogy, loose verbal arguments, rules of thumb. Policy-level thinking will often be like this when you seek to make simple corrections for biases. But, remember, these are error models in the errors-vs-bugs dichotomy; real skill improvement relies on bug models (as studies in deliberate practice suggest).

The opposite of policy-level thinking? Stimulus-response; reinforcement learning; habit; scripted, sphexish behavior. This, too, has its place.

Still, like inside and outside view, gears and policy thinking are made to work together. Learning the principles of strong gears-level thinking helps you fill in the intricate structure of the universe. It allows you to get past social reasoning about who said what and what you were taught and whay you're supposed to think and believe, and instead, get at what's true. Policy-level thinking, on the other hand, helps you to not get lost in the details. It provides the rudder which can keep you moving in the right direction. It's better at cooperating with others, maintaining sanity before you figure out how it all adds up to normality, and optimizing your daily life.

Gears and policies both constitute moment-to-moment ways of looking at the world which can change the way you think. There's no simple place to go to learn the skillsets behind each of them, but if you've been around LessWrong long enough, I suspect you know what I'm gesturing at.

8 comments

Comments sorted by top scores.