Deliberate Practice for Decision Making

post by wattsd · 2012-10-04T03:19:25.460Z · LW · GW · Legacy · 3 comments

Contents

3 comments

Much of this material is sourced/summarized from Deliberate Performance: Accelerating Expertise in Natural Settings and “The Power of Intuition”. Both contain much more than what is written here. An earlier draft/version of this was posted as "Productive Use of Heuristics and Biases". Based on feedback in the comments, much has changed. Feedback is appreciated, as are any other ideas on how deliberate practice can be applied.

How does someone incorporate something like deliberate practice into a typical job and everyday life? Deliberate practice is meant to be challenging, and because of this, it is draining. Much of the expertise literature describes a limit of a few hours of deliberate practice per day. If you can sustain more than that, you probably aren’t doing it right.

While the amount you can put in per day is limited, it still takes many hours of this intense practice, so reaching expert performance in a domain takes anywhere from years to decades. Deliberate performance is a concept related to deliberate practice, but perhaps more effective and efficient. Rather than separating practice and performance, the idea is to overlap the two as much as possible. The primary aim is to accelerate the development of expertise, while also improving the productivity of practice. Since you are practicing using normal tasks, you don’t have to set as much time aside where you aren’t working.

Deliberate performance is readily applicable to decision making. You make decisions as you normally would but also record your expectations and thinking behind the decision. What do you think will happen and why? How do you feel about the decision? When you have feedback on how the decision turned out, you can go back to see what you were thinking and how well your expectations matched reality. The idea of recording this information has been recommended by Peter Drucker, and more recently by Daniel Kahneman (Kahneman sees it as a way to reduce hindsight bias).

The term deliberate performance comes from the paper Deliberate Performance: Accelerating Expertise in Natural Settings, where the concept is elaborated into four exercises:

Estimate
One of the simplest ways to do this is to estimate how long a task you have will take. Accurately predicting how long things will take is a valuable skill on its own, but in estimating it, you form a mental model of the task and where the obstacles lie. When your estimate differs from reality, you can go back and see where you went wrong.

Experiment
The simplest way to experiment is just to try something and see what happens. This can be helpful in early stages when you are exploring, but typically, a better way is explicitly state what you expect to occur. As in estimation, if the actual results aren’t what you expect, you can update your mental models. It is usually a good idea to make your experiments cheap, in the sense that you expect failure and change to occur. Steven Spear, in his studies of Toyota, described cheap experiments as preferring bolting to welding, clamping to bolting, taping to clamping, and holding to taping. The more rapidly you can experiment, the more rapidly you can get feedback and update your models.

Experimentation is sort of where the rubber meets the road in deliberate performance. Where the other exercises are more focused on thought, experimentation is more about action. You see how much your mental models match the real world.

Explain
Why did things happen as they did? Try to explain from the available evidence. Explanation gives you an opportunity to make sense of why things turned out the way they did. Whether or not things went as you expected, try to explain why and how it happened. You can also seek out explanations from others, but you should probably form your own first so you can compare.

Extrapolate
What do you know now that you can use to reason about something you don’t know or something that doesn’t yet exist? What do you already know that you can use to solve a new problem? Using induction in proofs is an example of this, as is analogical reasoning. Klein describes an example of the latter in “Sources of Power”. Engineers are trying to figure out how much parts of new airplane designs will cost, but the parts are still being developed. The engineers end up looking at existing parts for analogs. None of the analogs are perfect matches, so they extrapolate the characteristics of multiple parts to create a new “Frankenstein” part.

Feedback and Coaching
Unlike deliberate practice, where the role of a coach is more formal and explicit, deliberate performance has the more realistic expectation that you will not have a coach. Instead, you try to make the most of the feedback you can get. Rather than a coach pointing out mistakes and what you can improve, you find errors in your mental models and ask effective questions. The 4Es will likely prompt some questions, while also giving you some knowledge and experience to make sense of the answers. While the exercises probably sound simplistic, something important happens as you do them: you are forced to form mental models, compare them to reality, and adjust them as needed.

Another useful method for getting feedback is to predict or emulate what an expert would do. When things don’t go as expected, you can then explain what you did and your reasoning to the expert and get feedback. As you find and correct mistakes in your mental models, your “expert emulation” should improve. Seeking explanations to compare to your own is also an example of this.

Using the Exercises
The first example, keeping a decision journal and reflecting on how things turns out, demonstrates that the 4Es can be combined and work well together. In fact, that is probably the best way to use them, looping through them multiple times. Similar to the scientific method, start with an estimate or hypothesis. Then, experiment to see if your hypothesis is correct. Next, attempt to explain what you saw in the results of the experiment in relation to your hypothesis. You’d then finish the first loop by extrapolating from the explanation, which can provide a new hypothesis to test.

One other method that combines multiple exercises is to do what Klein calls a premortem. If you have some large project, start by imagining that what you are trying to do has been a complete failure, and you are now doing a post mortem to understand what went wrong. Assuming things have gone wrong, why did they go wrong? The reasoning behind this technique is that people don’t want something to fail (or look like they want it to fail), but assuming it already has failed reduces that bias. In a sense, this is similar to red teams but easier to implement and less resource intensive. This gives you a chance to extrapolate and estimate, then form an explanation for why you think things might go wrong.

A few more exercises specific to decision making
In “The Power of Intuition”, Klein advocates identifying decisions where problems have occurred, forming exercises based on those decisions, playing through the exercises, and then critiquing your decision making process. The idea is that the most difficult scenarios occur rarely enough that developing expertise in them is impossible without some intervention, they occur too rarely to gain significant experience.

First, identify a decision. When reviewing the decisions, note what makes it difficult, what kinds of errors are often made, how an expert might approach it differently than a novice, and how the decision can be practiced and how you can get feedback.

Identified decisions can then be turned into scenarios which can be repeatedly practiced (typically in groups). Start with describing the events that led to the decision. The players are then told what they are trying to achieve, the context, and the constraints. Try to include a visual representation whenever possible. Even if you don’t work through the exercise, forming it can be useful in and of itself.

After the exercise, critique the decision and the process used to make it. Start with a timeline and identify key judgments. For each of the key judgments, note why it was difficult, how you were interpreting the situation, what cues/patterns you should have been picking up, why you chose to do what you did, and what you would’ve done differently with the benefit of hindsight.

3 comments

Comments sorted by top scores.