The One Mistake Rule

You should be deeply embarrassed if your model outputs an obviously wrong or obviously time-inconsistent answer even in a hypothetical situation.

Suppose you have a particle accelerator that goes up to half the speed of light. You notice an effect whereby faster particles become harder to accelerate.

You curve fit this effect and get that $\frac{c}{\sqrt{c^2-v^2}}$. and $1+\frac{1}{2}\frac{v^2}{c^2}+\frac{1}{2}\frac{v^4}{c^4}$ both fit the data, well the first one fits the data slightly better. However, when you test your formula on the case of a particle travelling at twice the speed of light, you get back nonsensical imaginary numbers. Clearly the real formula must be the second one. (The real formula is actually the first one)

A good model will often give a nonsensical answer when asked a nonsensical question, and nonsensical questions don't always look nonsensical.

I think the need for this is relatively higher in an anti-inductive environment like trying to beat the market than in an inductive environment like materials science or disease modeling. In the former, you don't dare deploy until you're confident it's missing nothing; in the latter, what you have mid-iteration is still very useful, especially if you're making big time-sensitive decisions.

One obviously mistaken model that I got a lot of use out of during a stretch of Feb-Mar is the one where the cumulative number of coronavirus infections in a region doubles every n days (for some globally fixed, unknown value of n).

This model has ridiculous implications if you extend it forward for a few months, as well as various other flaws. I was aware of those ridiculous implications and some of those other flaws, and used it anyways for several days before trying to find less flawed models.

I'm glad that I did, since it helped me have a better grasp of the situation and be more prepared for what was coming. And I don't think it would've made much difference at the time if I'd learned more about SEIR models and so on.

It's unclear how examples like this are supposed to fit with the One Mistake Rule or the exceptions in the last paragraph.

I think you're making this argument a bit strongly. Now, I've written a number of posts arguing that most people are too dismissive of flaws in models that only occur in hypothetical or unrealistic situations, but I don't think perfection is realistic. It seems that a model with no flaws would have to approach infinite complexity in most cases. The only reason why this rule might work is eventually your model will become complex enough that you can't find the mistake. Additionally, you will be limited by the data you have. It's no good knowing that prediction X is wrong because you ignore factor F if you don't have data related to factor F.

What are good methods of determining when the model is giving a definitely wrong answer, and when your not-a-model is grossly overconfident and wrong?

All models go wrong somewhere , and they are still useful if you use them where they don't go wrong.

Thinking about Donald's and Dagon's comments I do think there is something that can be more fully acknowledged (though was also reasonably assumed I think).

I walked away from the OP, but those two comments further clarified for me my own thinking, the one strike rule is a good way to refine our maps when we know the territory we are mapping out.

However, that is not always the case so when making call on a strike we should make sure that we're actually in the territory or home plate and have not wondered somewhere else. I think that was supposed to be address as part of the discussion about was it a mistake or not.

I think this makes for a useful heuristic when modeling but clearly other heuristics should also be applied to ensure you do stay in the territory intended. So is there a good one for helping to keep one on the right sports field? Is that needed, if no why and if so when?