Key Decision Analysis - a fundamental rationality technique

post by elityre · 2020-01-12T05:59:57.704Z · score: 46 (12 votes) · LW · GW · 10 comments

Contents

  The technique
  Some theory
  Personal experience
None
10 comments

The technique

This post is signal-boosting and recommending a strategy for improving your decision making that I picked up from the entrepreneur Ivan Mazour. He describes the process here, and publishes his own results every year on his blog.

In his words...

I believe that life is far too fast-paced for us to be able to make rational, carefully thought-out, decisions all the time. This completely contradicts my mathematical upbringing and training, but is something I have come to realise throughout my twenties. We need a way of keeping up with the constant barrage of decisions, even when the inevitable ‘decision fatigue’ sets in. The only way to do this, I find, is to act on instinct, but this only works if your instincts are correct. They cannot be correct all the time, of course, but if we can maximise the chance of making the right decision by instinct, then we have a strategy for coping with a complicated and highly productive life.
To sharpen my instincts, I keep a monthly journal of all key decisions which I make – decisions that could be truly life changing – and my instinctive reasons for why I made them. I go back only after exactly a year has passed, and I note down whether the decision was correct, and more importantly whether my instincts were right. At the end of the year, I go over all twelve months worth of notes, and search for any patterns amongst all of the right and wrong choices.
This is not a short-term strategy, as you can tell. In fact it takes exactly two years from the day you start following it, to the time that you can get some useful insights to sharpen your instincts. Keeping a diary of decisions has other uses, and there are many ways of getting an overview of your life prior to this, but it is only after the two years have passed that a genuine clear pattern presents itself.

Some theory

This accords with some abstract theory about human rationality. A perfect-Bayesian expected utility maximizer doesn't start out with an optimal policy. Rather, its strength is being able to learn from its experience (optimally), so that it converges towards the optimal policy.

Of course, humans have a number of limitations standing between us and perfect-decision-making-in-the-limit. Due to computational constraints, perfect Bayesian updating is out of reach. But among a number of limitations, the first and most fundamental consideration is "are you learning from your data at all?".

If the consequences of your decisions don't propagate back to the process(es) that you use to makes decisions, then that decision process isn't going to improve.

And I think that, by default, I mostly don't learn from my own experience, for a couple of reasons.

So I need some process, that involves writing things down, that allows me to intentionally implement back-propagation.

Personal experience

I only started logging my decisions a little more than year ago, and did the analysis for the end of 2018 this week, so I don't have that much personal experience to share. I'm sharing anyway, because it will be years until I have lots of experience with this technique.

That said,

Even given the issues I described above, I found the assessment activity to be extremely useful. There were some places where I was able to highlight "past Eli was flat-out wrong", and others where, having seen how things turned out, I could outline nuanced heuristics that take into account the right considerations in the right circumstances.

It also clearly affirmed two principles / hamming problems, that had occurred to me before, but hadn't really slapped my in the face. This was helpful for realizing that "my tendency to X is preemptively destroying most of the value I might create", which is an important thing to get to full conscious attention.


Good luck!




10 comments

Comments sorted by top scores.

comment by jacobjacob · 2020-01-12T09:52:59.852Z · score: 6 (3 votes) · LW(p) · GW(p)

I'm curious, could you share more details about what patterns you observed, and which heuristics you actually seemed to use?

comment by elityre · 2020-01-12T19:22:52.747Z · score: 4 (2 votes) · LW(p) · GW(p)

I think that's pretty personal to me.

comment by elityre · 2020-01-12T19:29:37.598Z · score: 5 (3 votes) · LW(p) · GW(p)

One thing I guess I could share:

I often make choices based on plans that I’m excited about at the time. But very frequently I don’t actually get very far with those plans before they peter out and falter / I move on to something else. When I am making choices about what to do, I should take my current plans with a grain of salt, because things actually change a lot from that initial vission.

comment by Kaj_Sotala · 2020-01-12T16:34:42.586Z · score: 5 (3 votes) · LW(p) · GW(p)

I find that the greatest challenge in starting to employ something like this, is learning to recognize the things that count as decisions to be recorded. To the extent that they are not too private, could you share more examples of the kinds of decisions that you have used this on?

comment by elityre · 2020-01-12T19:13:54.033Z · score: 11 (6 votes) · LW(p) · GW(p)

Here are some examples, though as I said, I think my own definition of a decision was too strict:

  • I went to that forecasting day that included Carl Shulman, Kajta, etc.
  • I didn’t do [multi week research project with some people].
  • I rejoined CFAR’s colloquium.
  • I decided to go to the mainline workshop in late February.
  • I bought a macbook air with a 512 GB hard drive and 18 GB of RAM. I returned it for a macbook pro with a 512 GB hard drive and 18 GB of RAM. This was $100 more expensive, but with a faster processor. For that reason I’m typing this on my old (often crashing) machine.
  • I opted not to attend the Bay NVC convergence facilitation training.
  • I returned my macbook pro to get a macbook air again, because of the better battery life.
  • I got on to Prague time the long way, by staying up late, sleeping all day, and then taking an evening plane, having a long travel day, then crashing, when I got to Europe.
  • I decided to come back from Europe early so that I could meet with Brienne and Duncan about instructor training, instead of hanging with FHI.
  • I didn't join the conversation about [topic] between [people].
  • I downloaded [that sketchy file].
  • I told [employer] that I could do about 10 to 12 hours of [category] work in October.
  • I bought access to AWC’s demonstrations of Focusing.
  • I stayed two extra days in Prague and then had a flight that left at 9:00 AM from Prague to Copenhagen, and then a connecting flight from Copenhagen to Oakland. Getting up really early to go to the airport didn’t suit me much since I had been waking up around 10:00. So I bought an $85 ticket to Copenhagen a day early, and stayed in the cheapest Hostel I could find.

I think at least one trigger for flagging decisions might be something like "I'm about to 'pull the trigger' on something." I have some amount of indecision, or conflictedness, and then I settle into one state or another.

comment by jacobjacob · 2020-01-12T20:40:15.490Z · score: 2 (1 votes) · LW(p) · GW(p)

For some reason seeing all this concreteness made me more excited/likely to try this technique.

comment by elityre · 2020-07-23T06:53:28.646Z · score: 2 (1 votes) · LW(p) · GW(p)

Cool!

comment by MoritzG · 2020-01-14T14:29:32.662Z · score: 1 (1 votes) · LW(p) · GW(p)

This reminds me of another issue. If you do make informed complicated decisions, the basis of these decisions might change over time. I struggle with that problem professionally. As an engineer I have to make complicated compromises/decisions. The trouble is that the situation changes all the time. The requirements and the means change. Without tracking why I made decisions there is no way to tell if those decisions still hold, because I do not even remember myself. The project becomes a zombie even before there are true legacy and hand-over issues. Usually decisions are incomprehensible later. We all know this and have though everyone else is an idiot, but often people had good reason to do it that way or lost track as described. Making changes to often reveals that there were reasons, but too late.

Privately you might find yourself in a place that you had reason to go into but those reasons went away without you noticing.

comment by Pattern · 2020-01-14T00:45:09.604Z · score: 1 (1 votes) · LW(p) · GW(p)
This was helpful for realizing that "my tendency to X is preemptively destroying most of the value might might create",

might might?

What was the tendency? (Or was this part just saying "noticing bad habits"?)


comment by elityre · 2020-01-14T01:50:41.304Z · score: 3 (2 votes) · LW(p) · GW(p)

I fixed the typo.

At least for the time being, I'm abstracting the specifics from most of the things I learned, because they are pretty personal.