Update more slowly!

post by Davidmanheim · 2020-07-13T07:10:50.164Z · LW · GW · 4 comments

My experience forecasting has led me to realize that a key mistake that is often made is updating on new data too quickly. This seems backwards, but I think that often the biggest reason that people both under- and over-react to evidence is that they don't consider the evidence clearly enough, and immediately start revising their model to account for the evidence, instead of actually thinking before updating.

Let's deconstruct how rapid updating is misleading with a simple notional example. Someone shows up with a coin and claims that she is psychic and can predict coinflips. You are skeptical, and challenge her to do so. She immediately takes out a coin and correctly predict heads 3 times in a row. You can update a few ways:

Notice that these possibilities which spring to mind are completely different cognitive strategies. Either you update to believe her more, you decide the evidence is insufficient, or you immediately seize on a new hypothesis that explains the evidence.

But these are all mentally lazy strategies. If you thought about the situation for longer, you could easily generate a half dozen additional hypotheses. Perhaps the she has trained herself to flip coins a certain way. Perhaps she simply lied and said the coin landed heads each time, and didn't really give you time to see it well when it didn't. Perhaps she is substituting the coin as it lands. Perhaps, perhaps, perhaps.

My advice, per the title, is to slow down. You might decide to be a good Bayesian, and preserve multiple hypotheses, updating marginally - but doing this means that you assume the correct hypothesis is in your prior set. There are a million hypotheses that can explain a given set of events, and the most cognitively available ones are those that allow you to be lazy.

Don't worry, take your time. If the issue matters enough to bother trying to update your model, taking five minutes to reflect is better than jumping the gun. And if you don't need to make any decisions, at least file everything away and decide that it's unclear instead of quickly responding with an overconfident "Aha, now I understand!" or worse, a "Eureka, I've solved it!"

Bayesian thinking gives you answers no faster than a rational accumulation of evidence can possibly allow, given the uncertainties that exist. Slow down. Marginally rationally updating doesn't give you confident answers quickly. It can't.
Trying to update faster isn't going to get you better answers now, it will get you worse answers more quickly.

Updating isn't a sprint to the answer, or even a race - it's a long-duration hike towards a still-unclear goal. If you imprudently start sprinting early because you think you see the goal, you're just going to hurt yourself, or get lost and never make it to the right destination. Take your time.

4 comments

Comments sorted by top scores.

comment by Bucky · 2020-07-13T10:43:50.855Z · LW(p) · GW(p)

An error that I’ve made more than once is to go through my existing hypotheses‘ likelihoods for the new evidence and update accordingly while managing not to realise that the likelihoods are low for all of the hypotheses, suggesting the true answer might be outside of my hypothesis space.

comment by Dagon · 2020-07-13T16:05:02.719Z · LW(p) · GW(p)

I don't exactly disagree, but I think this focuses on the wrong thing. It's not about slowing down in your calculations, nor about updating by less for each data point (what I initially thought "slow" meant). Calculate your updates as fast as you like, as long as you do it correctly. Bayes Theorem gives us a numerical approach to updates - if you're making bigger jumps than are justified by the evidence, that's an error. If you're making smaller jumps, that's also an error.

Likewise, assuming that the truth is in your hypothesis set is a mistake. You should include "other" in your modeling at this level. More importantly, you should recognize that coinflip outcomes provide _zero_ evidence about the relative likelihood between cheating and psychic (and "other"). Each successful prediction _does_ provide evidence against luck and toward those other possibilities.

Getting better at updating is a skill, and "go slow" is probably good advice when practicing (per the standard saying "Slow is smooth and smooth is fast"), but doesn't generalize. Time is a limited resource and you're spending valuable seconds here that you'd rather be doing something else.

Replies from: Davidmanheim
↑ comment by Davidmanheim · 2020-07-13T17:04:59.584Z · LW(p) · GW(p)

I should clarify that what I mean by "slow" was supposed to be in the cognitive / Kahneman sense. In most cases, as I said, "if you don't need to make any decisions, at least file everything away and decide that it's unclear." Instead, what I see people do is jump to updating / grabbing a hypothesis, acting "fast." The failure modes that this can cause will include over or under-updating, ignoring "missing" hypotheses and not thinking of alternative explanations, narrowing the hypotheses too early, not updating marginally, etc.

Given that, I think I agree on all of the substantive points. However, I will note that including an explicit "other" category is tricky, but critical. The problem is that given such a category, over time it's more plausible than anything else. It turns into the equivalent of "the witch down the road did it," which is super-parsimonious and can explain anything.

And slowness in the sense you thought I was talking about is equivalent to lowering the weight on evidence, or to having a higher weight on priors. Strength of priors and how much to weight evidence are good questions to discuss, and can be tricky, but weren't my point in this post.

comment by romeostevensit · 2020-07-13T18:56:12.516Z · LW(p) · GW(p)

Overfitting on novelty seems like it could be the epigraph of a generation.