"No evidence" as a Valley of Bad Rationality

Well, we can say that 27/30 (90%) patients improved. With a very high level of confidence, we can say that this disease is less fatal than Ebola (which would have killed 26 or so).

Having the reverse as the null hypothesis is also bad. Which is worse?

Before responding, I think this is an opportunity for a productive and charitable back and forth, which I'd like to have with you! This also might be challenging, because there are already a bunch of threads to this argument. So I'll respond to a couple pieces of what you've said, and feel free to only respond to part of what I've said.

The original article seemed to have three points, or question-clusters.

1. Medical: For N chemo treatments, why not do N + 1 treatments? How do we determine the stopping point, or how should we?
2. Social: Was this oncologist a trustworthy authority figure? What's the best way to ensure we're making the best decisions for my relative's cancer treatment? How do we navigate these conversations and relationships, balancing the range of priorities and constraints of all involved?
3. Philosophical: Is absence of evidence evidence of absence? How do we reason under uncertainty, given what seems to us like common sense, our priors, and our extremely limited understanding of the causal mechanisms at play?

Here's my interpretation and critique of the author's implied answer to the medical question. I didn't do this in my top-level comment, but let's charitably grant that he was fully aware of the quiescent stem cell issue.

• The author seems to be implying that it's common sense to apply at least N + 1 treatments in this case, to kill any remaining proliferating cells. But this rule has no clear stopping point. Why not have the relative do chemo for the rest of their life? Charitably, the OP didn't intend for their relative to get chemo forever - they had some common-sense stopping point in mind. But where?
• The doctor's view is that "there was "no evidence" that additional chemo, after there are no signs of disease, did *any* additional good at all, and that the treatments therefore should have been stopped a long time ago and should certainly stop now." The doctor's preferred stopping point for chemo was "as soon as we see no signs of disease," but his actual stopping point in this case seems to be "quite a few rounds of chemo after we stopped seeing any signs of the disease."

So when the OP initiated this conversation, the cancer had been in remission for a while. So what's the "common-sense" criterion for a stopping point?

The doctor has one, and it also makes "common sense." If you can't see it, and you've been fighting it past the point of not being able to see it for a while, it's probably not there. We know that chemo is harming the body and quality of life of the patient, and will continue to do so until the treatment is stopped. We can also resume the chemo if the cancer re-emerges.

I think my charitable interpretation fo the doctor's perspective (a charity the OP did not grant the oncologist), is at the very least a reasonable argument. It's one that the OP made no effort to suss out, either in the conversation with the doctor or in reflection afterward.

This brings us to the social issue. My interpretation of the doctor is that he's dealing with a relative who is:

• Uninformed about the biomedical literature, by his own admission.
• Making one "common-sense" argument, without making an effort to see the "common sense" in the doctor's perspective.
• Not doing research prior to attending this meeting.
• Assessing the doctor's competence to make good decisions for the patient.

Now, I think that you and I agree that it is good and necessary for patients, or their advocates, to assess the competence of experts generally, including doctors. This is a tricky problem, and it's been written about at length in the rationalsphere.

My perspective is that when you do this, there's a tradeoff involved. On the one hand, if you do it well, you increase your chance of working with a competent expert. On the other hand, if you do it poorly, you may disrupt the formation of appropriate trust, and complicate the work of establishing relationships and sharing information.

In this case, I am arguing that the OP, by his own admission, did not do some of the common-sense things you'd do if you were trying to play the role of expert-vetter well. In this case, especially if you see yourself as a very rational, smart person, you'd make an effort to do some research in advance, and to understand the doctor's point of view. I agree that doctors, as you say, are not always up-to-date on the literature, and I've been on the receiving end of some bad care myself. However, a simple action you can take to safeguard against such cases, when you already know the diagnosis, is to find that literature yourself.

The OP seems not to have done that. He also wasn't asking about the doctor's up-to-dateness, but rather the doctor's willingness to address his point about absence of evidence. So I assert that the OP seems to have been playing the role of expert-vetter poorly. Furthermore, they perceived and portrayed themselves as doing it well, a sort of Dunning-Krueger effect.

They then use this as a dig against their doctor, and their post is now being used as a lesson in rationality. This seems concerning to me. Not having been present, I don't want to assume I really know what was going on. But it rubs me the wrong way, and doesn't seem like a central case of either instrumental or epistemological rationality in action.

I don't want to get into the philosophical level, because this isn't a main source of my objection to the Overcoming Bias post. So I will leave that to the side.

You don't have to persuade me that doctors are not always as well-informed as we'd like them to be. And certainly I already know that our medical system is deeply flawed, even broken. As I say, I'm on board with the idea that patients or their relatives should inform themselves, and have a collaborative role with their doctors.

I'm saying that I think the poster of the Overcoming Bias post comes across to me as having done a below-average job of occupying that role. That's just the perception I get from reading the post. Maybe I'd feel different if I'd been there in person to observe, but I can only go off the information given.