"No evidence" as a Valley of Bad Rationality
post by adamzerner
Quick summary of Doctor, There are Two Kinds of “No Evidence”:
- Author has a relative with cancer. Relative is doing well after chemo and is going to a doctor to see if it's worth getting more chemo to kill the little extra bits of cancer that might be lingering.
- Doctor says that there is no evidence that getting more chemo does any good in these situations.
- Author says that this violates common sense.
- Doctor says that common sense doesn't matter, evidence does.
- Author asks whether "no evidence" means 1) a lot of studies showing that it doesn't do any good, or 2) not enough studies to conclusively say that it does good.
- Doctor didn't understand the difference.
Let me be clear about the mistake the doctor is making: he's focused on conclusive evidence. To him, if the evidence isn't conclusive, it doesn't count.
I think this doctor is stuck in a Valley of Bad Rationality. Here's what I mean:
- The average Joe doesn't know anything about t-tests and p-values, but the average Joe does know to update his beliefs incrementally. Lamar Jackson just had another 4 touchdown game? It's not conclusive, but it starts to point more in the direction of him winning the MVP.
- The average Joe doesn't know anything about formal statistical methods. He updates his beliefs in a hand-wavy, wishy-washy way.
- The doctor went to school to learn about these formal statistical methods. He learned that theorizing is error prone and that we need to base our beliefs on hard data. And he learned that if our p-value isn't less than 0.05, we can't reject the null hypothesis.
- You can argue that so far, the doctors education didn't move him forward. That it instead caused him to take a step backwards. Think about it: he's telling a patient with cancer to not even consider more chemo because there is "no evidence" that it will do "any" good. I think Average Joe could do better than that.
- But if the doctor continued his education and learned more about statistics, he'd learn that his intro class didn't paint a complete picture. He'd learn that you don't always have access to "conclusive" evidence, and that in these situations, sometimes you just have to work with what you have. He'd also learn that he was privileging the null hypothesis in a situation where it'd make sense to do the opposite. The null hypothesis of "more chemo has no effect" probably isn't true.
- Once the doctor receives this further education, it'd push him two steps forward.
- In the intro class, he took one step backwards. At that point he's in the Valley of Bad Rationality: education made him worse than where he started. But then when he received more education, he took two steps forward. It brought him out of this valley and further along than where he started.
I think that a lot of people are stuck in this same valley.
Comments sorted by top scores.
comment by Jay Molstad (jay-molstad) ·
2020-03-29T02:58:42.068Z · LW(p) · GW(p)
I've definitely seen this in the academic literature. And it's extra annoying if the study used a small sample; the p-values are going to be large simply because the study didn't collect much evidence.
OTOH, chemotherapy isn't a very good example because there are other factors at work:
Replies from: ricraz
- Chemotherapy has serious side effects. There are good reasons to be cautious in using extra.
- There are also not-as-good reasons to avoid using extra chemotherapy. Medical care is highly regulated and liability-prone (to varying extents in various areas). In the US, insurers are notoriously reluctant to pay for any treatment they consider unnecessary. Departing from standard practice is likely to be expensive.
↑ comment by Richard_Ngo (ricraz) ·
2020-03-30T13:22:12.438Z · LW(p) · GW(p)
I think the fact that chemotherapy isn't a very good example demonstrates a broader problem with this post: that maybe in general your beliefs will be more accurate if you stick with the null hypothesis until you have significant evidence otherwise. Doing so often protects you from confirmation bias, bias towards doing something, and the more general failure to imagine alternative possibilities. Sure, there are some cases where, on the inside view, you should update before the studies come in, but there are also plenty of cases where your inside view is just wrong.
comment by PatrickDFarley ·
2020-03-29T00:49:43.231Z · LW(p) · GW(p)
I like this, thanks for posting. I've noticed there's a contrarian thrill in declaring, "Actually there's no evidence for that" / "Actually that doesn't count as evidence."
Academics love it when some application of math/statistics allows them to say the opposite of what people expect. There's this sense that anything that contradicts "common sense" must be the enlightened way of thinking, rising above the "common," "ignorant" thinking of the masses (aka non-coastal America).
Replies from: andrew-jacob-sauer
comment by Nick_Tarleton ·
2020-03-30T17:33:47.963Z · LW(p) · GW(p)
Upvoted, but weighing in the other direction: Average Joe also updates on things he shouldn't, like marketing. I expect the doctor to have moved forward some in resistance to BS (though in practice, not as much as he would if he were consistently applying his education).
comment by Jacob Falkovich (Jacobian) ·
2020-03-30T14:19:11.706Z · LW(p) · GW(p)
I just thought of this in the context of this study on hydroxychloroquine in which 14/15 patients on the drug improved vs 13/15 patients treated with something else. To the average Joe, HCQ curing 14/15 people is an amazing positive result, and it's heartening to know that other antivirals are almost as good. To the galaxy-brained journalist, there's p>0.05 and so "the new study casts doubt on hydroxychloroquine effectiveness... a prime example of why Trump shouldn't be endorsing... actually isn't any more effective."Replies from: Nick_Tarleton
comment by Pattern ·
2020-03-29T05:47:26.292Z · LW(p) · GW(p)
Upon seeing the title, I guessed this piece was going to argue that people are often right without evidence. Instead the OP argued against believing something without evidence.
In the intro class, he took one step backwards. At that point he's in the Valley of Bad Rationality: education made him worse than where he started.
But is the doctor worse or better for it (even assuming that this story, second hand or third hand or more is accurate)? And how do we know?Replies from: orthonormal
↑ comment by orthonormal ·
2020-03-29T06:13:18.778Z · LW(p) · GW(p)
In general, it's good to check your intuitions against evidence where possible (so, seek out experiments and treat experimentally validated hypotheses as much stronger than intuitions).
The valley being described here is the idea that you should just discard your intuitions in favor of the null hypothesis, not just when experiments have failed to reject the null hypothesis (though even here, they could just be underpowered!), but when experiments haven't been done at all!
It's a generalized form of an isolated demand for rigor, where whatever gets defined as a null hypothesis gets a free pass, but anything else has to prove itself to a high standard. And that leads to really poor performance in domains where evidence is hard to come by (quickly enough), relative to trusting intuitive priors and weak evidence when that's all that's available.Replies from: Pattern
↑ comment by Pattern ·
2020-03-30T02:27:17.608Z · LW(p) · GW(p)
Having the reverse as the null hypothesis is also bad. Which is worse?Replies from: elriggs
↑ comment by elriggs ·
2020-04-01T21:20:41.390Z · LW(p) · GW(p)
Correct, favoring hypothesis H or NOT H simply because you label one "null hypothesis" are both bad. Equally bad when you don't have evidence either way.
In this case, intuition favors "more chemo should kill more cancer cells", and intuition counts as some evidence. The doctor ignores intuition (which is the only evidence we have here) and favors the opposite hypothesis because it's labeled "null hypothesis". Replies from: Pattern
↑ comment by Pattern ·
2020-04-01T23:14:58.750Z · LW(p) · GW(p)
I was suggesting that there might be ways of assigning the label of "null hypothesis".
X is good, more X is good. (intuition favors "more chemo should kill more cancer cells")
X has a cost, we go as far as the standards say, and stop there. (Chemo kills cells. This works on your cells, and cancer cells. Maybe chemo isn't like shooting someone - they aren't that likely to die as a result - but just as you wouldn't shoot someone to improve their health unless it was absolutely necessary, and no more, chemo should be treated the same way.) "Do no harm." (This may implicitly distinguish between action and inaction.)
comment by jmh ·
2020-03-30T22:45:17.665Z · LW(p) · GW(p)
Shouldn't the follow up to no evidence showing that it does any good be "Is there any evidence showing it does harm?"
Have you seen this before? Any thoughts on how it might inform on your examples?
I am not defending the arrogance of some doctors but I do wonder if you are truly giving the doctor in question here a full opportunity and might have biased the discussion by stating things in a way that did not allow a good discussion to ensue but perhaps setup a more adversarial framework.
I wonder how much a believe in the Hippocratic Oath might be at play here.