“Necessary Claims”: A technique to structure complex decisions

post by fgrosseholz@yahoo.com · 2021-06-10T05:16:34.409Z · LW · GW · 0 comments

Contents

  In a nutshell
  Description of the technique, using an example
  “Recipe” to copy-paste if you want to give Necessary Claims a spin
None
No comments

Life often turns out to be too complicated to fit into a human brain. If you’ve ever tried to choose from fifty flavors of ice cream, or to decide whether to buy a house, you know what I mean.

Humans tend to make complex decisions based on simplified models called heuristics. Heuristics sometimes turn out to lack important factors, be biased, or hard to explain. Often, that’s fine - the stakes are low, you don’t have time and no one needs to understand your reasoning (e.g., choosing from fifty flavors of ice cream). “Necessary Claims'' is for the cases where a high risk of being incomplete, biased or hard to explain isn’t fine: Stakes are high, and you may need to involve others in your decision (e.g., buying a house). 

Breaking down a decision into necessary claims helps to not overlook important factors, and it gives you a story to tell. In the beginning, your story might sound like “we will probably decide this way, if x turns out to be true and y turns out to be false”. And later on, it will change to “we have decided this way, because we are highly confident that x is true and somewhat confident that y is false”.

Note that this technique formalizes a process that may seem obvious to some people. To me, it was hugely helpful to think about it explicitly and use it systematically, and I hope it will be to others, too. It also builds on several techniques taught by CFAR, although I am not officially affiliated with CFAR, this post reflects my personal experience and opinions and all mistakes are my own. I'm grateful to Elizabeth Garrett and Kyle Scott for feedback on an initial draft.

In a nutshell

To make a decision, you list all the necessary claims that would need to be true for you to decide one way. You also list your confidence that each claim is true. You then do analyses to support (or refute) those claims, until your aggregate confidence is high enough to decide.

Description of the technique, using an example

Say you and your partner are considering buying a house in the city you live in. Houses are expensive, so this is high-stakes, and you pool your money, so you want to decide together. You are aware that this decision will take up a few of your Saturdays, and decide to invest the time. First, you agree on the decision you want to make: “Should we buy a house here and now?” 

If you have trouble defining the decision at this step, you might consider Goal Factoring [LW · GW] (a CFAR technique that helps you find all the subgoals you may have behind an idea). You may want to decide between multiple options, but “Necessary Claims” works best if your question has the form “should we do xx”. If you have multiple options, go with the one that seems best, and keep the list of other options at hand for the next step - you will likely find some of your necessary claims taking the form of “the top option is really better than the others in a specific aspect”.

Now, you collect necessary claims by asking yourself (and your partner): “What claims would need to be true, so that buying a house here and now is a good idea?”

You might come up with the first claims rather quickly, for instance “(It needs to be true that) it will be cheaper to buy than rent if we stay in this city for another five years”. To find more necessary claims, it can help to go through a framework that fits the content of the decision. For instance, you might consider how buying a house influences your spare money vs. spare time, and come up with an additional necessary claim such as “(It needs to be true that) we will enjoy doing maintenance tasks in the garden and house so much that we want to do it in our spare time”. For other types of decisions, you may need other frameworks, such as EA’s famous important - tractable - neglected, or business frameworks such as Porter's Five Forces. Or you could perform a pre-mortem (also called “Murphyjitsu” at CFAR), asking yourself “If this turns out to be the worst (or best) decision I ever made, what could have happened?”. Another good way to come up with non-obvious but important necessary claims could be Double Crux, a CFAR technique that entails discussing the matter (with someone or with yourself) based on statements like “to believe that x is true, I would need to believe the following things: ...”. Double Crux can also help to get a sense of whether you have covered enough ground with your necessary claims to make a confident decision. You won't be able to find all the claims, and you definitely won't be able to conclusively confirm or refute them. But if you squint at the decision from a bunch of different angles, you could probably collect at least the top 5 claims. And if you squint at those claims from a bunch of different angles, you could probably get a decent sense of how to decide. Don’t be afraid to spend a significant part of your time budget on finding necessary claims and phrasing them precisely - getting the claims right can save you a lot of analysis. 

Once you have your set of claims, assign a level of confidence to each claim, asking “how sure are we already that this claim is true?”. For the mathematically savvy, you could use Bayes’ rule here - although for most real-world problems, confidence levels from “low” to “high” will likely do the job. You may also want to sort the claims by how important they are. Write claims and confidence levels on an overview page (for me, a spreadsheet works best) so you can update them in the next steps - this will be what your final output looks like.

Upon assigning confidence levels, keep asking yourself (and your partner): “What type of analysis could change our level of confidence, supporting or refuting this claim?” and collect a list of possible analyses. You will likely not want to do all of them; rather prioritize the ones that give a lot of confidence on important claims quickly over analyses that barely swing your confidence on less important claims, but take forever.

Now, you can assign different analyses on different claims to different people for parallel work. For instance, you might decide to calculate how many months of rent payments it would take to pay for an average house in the area, including interest, overhead costs of the purchase like taxes and fees, and maintenance costs. You could start with a rough estimate for things that are hard to find out, e.g., maintenance costs. If that part of the analysis ends up being a “swing factor” for your decision (e.g., the costs for rent look a bit higher than for the purchase, but you don’t know if they are high enough to exceed both purchase and maintenance cost), you could do a more in-depth evaluation (e.g., talk to a home owner and ask what their maintenance costs were in the last years). In parallel, your partner could do a different analysis - for instance, helping a friend out in the garden for a day to see whether you might like it enough to consider it a spare time activity.

Upon completing each analysis, you can go back and update your confidence levels on the overview of your necessary claims. Your confidence in your overall decision then is an aggregate of the confidence in individual necessary claims. The aggregate could be an average, or could follow a specific rule (e.g., all individual confidence levels must be larger than x), or you can just eyeball it - whatever works best for you (and your partner). Ultimately, the goal is an overview of necessary claims with confidence levels that allows you to tell a story of how you decided and why: “We have decided to buy a house here and now, because we are highly confident that it will be cheaper than renting. We are also somewhat confident that we will see maintenance work in the house and garden as a welcome spare time activity (...)”. 

Note that one of the most useful outcomes of this process is often a tweak in how you do the thing you decide to do. For instance, you may decide to buy rather than rent, but to buy a flat in a managed compound rather than an equally priced but larger house. You may have realized that cutting bushes in the garden isn’t your thing, so you’re willing to compromise a bit on space if you can have someone do the gardening for you.

“Recipe” to copy-paste if you want to give Necessary Claims a spin

• Write down the decision you want to make as a question of the form "Should we do x?"
  • Among multiple options, pick your top option based on current knowledge and keep a list of other options
  • Consider goal factoring to create options and refine the question
• Check: Is this decision one that benefits from a rather complex technique like “Necessary Claims”? I.e., are one or more of the below true?
  • High complexity (need for extra working memory)
  • High stakes (need for high confidence level)
  • To be made in a team (need for clear modules)
  • Involving different stakeholders (need for a narrative to explain the decision)
• Consider less time-consuming alternatives, such as a simple pro-con list, or maybe even deferring to authority instead of making the decision yourself
• Consider time-boxing your decision
• Ask yourself, and any team members: “What claims would need to be true for x to be a good idea?” and write down your necessary claims as statements
  • To find more necessary claims, go through a framework, or do a pre-mortem, or use Double Crux
  • Be aware that finding all necessary claims and phrasing them precisely can be the main effort in the decision, and an iterative process
• For each necessary claim, assign a prior level of confidence by asking “How confident are we already that this is true?”
  • Potentially rank necessary claims by importance
  • Keep that overview of necessary claims and confidence levels to iteratively update, e.g., in a spreadsheet shared with all team members
• For each necessary claim, define analyses by asking: “What type of analysis could change our level of confidence, supporting or refuting this claim?”
• Prioritize analyses, then start doing the highest priority analyses (potentially allocate to team members for parallel work)
• Define how much confidence you want to decide
• Upon completing each analysis, go back and update your confidence levels on the overview of your necessary claims, until your overall confidence is high enough to decide
• Please consider letting me (and/or people here) know about any improvements you make to the technique!

0 comments

Comments sorted by top scores.