A simpler way to think about positive test bias

I don't think this is quite right, for reasons related to this post.

Sometimes a hypothesis can be "too strong" or "too weak". Sometimes hypotheses can just be different. You mention the 2-4-6 task and the soda task. In the soda task, Hermoine makes a prediction which is "too strong" in that it predicts anything spilled on the robe will vanish; but also "too weak" in that it predicts the soda will not vanish if spilled on the floor. Actually, I'm not even sure if that is right. What does "too strong" mean? What is a maximally strong or weak hypothesis? Is it based on the entropy of the hypothesis?

I think this mis-places the difficulty in following Eliezer's "twisty thinking" advice. The problem is that trying to disconfirm a hypothesis is not a specification of a computation you can just carry out. It sort of points in a direction; but, it relies on my ingenuity to picture the scenario where my hypothesis is false. What does this really mean? It means coming up with a second-best hypothesis and then finding a test which differentiates between the best and second best. Similarly, your "too strong" heuristic points in the direction of coming up with alternate hypotheses to test. But, I claim, it's not really about being "too strong".

What I would say instead is your test should differentiate between hypotheses (the best hypotheses you can think of; formally, your test should have maximal VIO). The bias is to test your cherished hypothesis against hypotheses which already have a fairly low probability (such as the null hypothesis, perhaps), rather than testing it against the most plausible alternatives.

Terminology request: Can we use the term "positive test bias" instead of "positive bias"?

"Positive bias" seems like bad jargon - it is not used by researchers, an unfamiliar listener would probably think that it had something to do with having an overly rosey view of things, and all of the results on the first page of Google except for those from LW use it to refer to an overly rosey view.

Whereas "positive test bias" is used by some researchers in the same sense that Eliezer used "positive bias", is only used in that sense on the first page of Google hits, is a more precise phrasing of the same idea, and is less likely to be taken by unfamiliar listeners as referring to an overly rosey view.

The term that is most commonly used by researchers is "confirmation bias", but as Eliezer noted in his original post this term gets used to talk about a cluster of related biases; some researchers recognize this and instead talk about "confirmatory biases". Singling out "positive test bias" with a separate label seems like a potentially good case of jargon proliferation - having more terms in order to talk more precisely about different related concepts - but calling it "positive bias" seems like a mistake.

What is that quote of Scott's... Something about how the sequences obsolete themselves. And that he remembers the sequences being full of all these great insights about difficult topics - but when he goes back and rereads them, it's all just so obvious.

You probably see where I'm going with this. It seems entirely possible that when you say "oh, it's easy, you just notice when you're making a hypothesis that might be too strong and then come up with a way to test it," you are in fact doing the complete content of that sequence post that seeemed insightful way back when, it's just that it's easy to you now.