Right for the Wrong Reasons

Predictions are justified not by becoming a reality, but by the likelihood of their becoming a reality [1]. When this likelihood is hard to estimate, we can take their becoming a reality as weak evidence that the likelihood is high. But in the end, after counting all the evidence, it's really only the likelihood itself that matters.

If I predict [...] that I will win [...] and I in fact lose fourteen touches in a row, only to win by forfeit

If I place a bet on you to win and this happens, I'll happily collect my prize, but still feel that I put my money on the wrong athlete. My prior and the signal are rich enough for me to deduce that your victory, although factual, was unlikely. If I believed that you're likely to win, then my belief wasn't "true for the wrong reasons", it was simply false. If I believed that "you will win" (no probability qualifier), then in the many universes where you didn't I'm in Bayes Hell.

Conversely in the other example, your winning itself is again not the best evidence for its own likelihood. Your scoring 14 touches is. My belief that you're likely to win is true and justified for the right reasons: you're clearly the better athlete.

[1] Where likelihood is measured either given what I know, or what I could know, or what anybody could know - depending on why we're asking the question in the first place.

I'm confused about what your goal is in making the kind of predictions you're talking about. If, for example, you're betting money on the outcome of your upcoming fencing bout, your estimate of how likely you are to win should include all hypotheses about possible paths to victory, and any of those paths should qualify as a validation of your prediction. If the decision you want to make is about something other than the outcome of the bout, e.g. how well you fence, then you should make a prediction about that instead.

In other words, the fencing bout looks like a toy example and I don't understand what non-toy examples it's supposed to be illustrating.

In order to make better predictions, we must cast out those predictions that are right for the wrong reasons. While it may be tempting to award such efforts partial credit, this flies against the spirit of the truth.

Are you going to reward me for being wrong for the right reasons? If not I want to know who is skimming 'credit' off the top.

Lets abstract about this:

There are 2 unfair coins. One has P(heads)=1/3 and the other P(heads)=2/3. I take one of them, flip twice and it turns heads twice. Now I believe that the coin chosen was the one with P(heads)=2/3. In fact there are 4/5 likelihood of being so. I also believe that flipping again will turn heads again, mostly because I think that I choose the 2/3 heads coin (p=8/15). I also admit the possibility of getting heads but being wrong about the chosen coin, but this is much less likely (p=1/15). So I bet on heads. So I flip it again and it turns heads. I was right. But it turns out that the coin was the other one, the one with P(heads)=1/3 (which I found after a few hundreds flips). Would you say I was right for the wrong reasons? Well I was certainly surprised to find out I had the wrong coin. Does this apply for the Gettier problem?

Lets go back to the original problem to see that this abstraction is similar. Smith believes "the person who will get the job has ten coins in his pocket". And he does that mostly because he thinks Jones will get it and has ten coins. But if he is reasonable, he will also admit the possibility of he getting the job and also having ten coins, although with lower probability.

My point here is: at which probability the Gettier problem arises? Would it arises if in the coin problem P(heads) was different?

A simpler maxim is to pay attention to (fixing) cognitive errors at all times, without excuses. Correctness of a prediction is potentially useful data, but it's also an excuse for overlooking flaws in the prediction procedure.

Fixing a flaw in a procedure (behavior, skill, intuition) is a task that's separate from updating the prediction. Predictions need updating simply as a matter of recycling cached thoughts, which might be useful even if you are not fixing any particular reasoning error.

There is also a more subtle kind of cognitive errors where you can correctly solve a problem using the right systematic/conscious/formal method, but your intuition (maybe just one of the relevant intuitive mental models) is out of tune with this process. It's useful to have at your disposal a somewhat reliable intuitive guidance in thinking about a problem (it can be crucial in forming a plan for solving it), so when a discrepancy appears, it means that there is a bug either in the intuition or in the more formal procedure, which motivates looking into the discrepancy in more detail, and fixing the bug.

(The first thing I thought when I saw the title of this post in the sidebar, before even following the link, was “Gettier, schmettier, won't you people taboo ‘know’ already?” But anyway...)

(Gettier does not explain this behavior)

I assume he was going to write that Smith believed that he would get the job himself and had counted the coins in his own pocket but Jones got the job instead and happened to also have ten coins, and then some massive brain fart occurred.

Thanks for this post. Whatever problems the JTB definition of knowledge may have—the most obvious one of those to LWers probably being the treatment of "knowledge" as a binary condition—the Gettier problem has always struck me as being a truly ridiculous critique, for just the reasons you put forward here.

Seeing something like someone with 14 points lose on the last point should actually drive your probability closer to 1/2 of winning a 1v1. That is, you can't just discount it. You just have to take into account all the stuff you learned when, er, learning things.

Events are subsets of outcome space, not just single outcomes. If your prediction is merely "I will be declared the winner of this fencing match," and space aliens mind-control the judges so that they declare you the winner before you even arrive, you were correct, because that outcome is within the subset you predicted. If you didn't predict a specific mechanism, go ahead and laud yourself for predicting the correct event.

However, my own predictions (and, I suspect, those of most humans) are usually of a different character. They include some causal narrative. I think the tricky part is either removing that narrative from your prediction, or explicitly including what does and does not fall inside that narrative as separate predictions.

I strongly recommend taking a look at Epistemology and the Psychology of Human Judgment by Bishop and Trout (badger's review).

Here's a simple analysis of what's going on when we feel a prediction is right for the wrong reasons:

One function of the habit of making firm predictions is to test our models of the world and be able to give others strong evidence about how good our models are. What's happening when we are right for the wrong reasons is that our model is confirmed by the explicitly predicted outcome but then disconfirmed by some detail that wasn't described in the prediction. The subsequent disconfirmation might be strong enough to cancel out the initial confirmation, or stronger. Even if it only partially cancels out the initial confirmation, focusing only on the evidence given by the explicitly predicted outcome exaggerates the extent to which our model is confirmed by the evidence.

Your fencing example: based solely on disagreement about the the fencers' relative ability, two predictors make different predictions about whether I win or my opponent wins. I lose 14 touches, but my opponent is disqualified, etc. "My" predictor collects his money, but his model is not clearly vindicated. If we focus only on the explicitly predicted event, his model assigned it a greater probability than the other predictors' model, and so is confirmed by the explicitly predicted outcome. But conditional on me winning, "my" predictor's model assigns a (much?) lower probability (compared to the other predictors' model) to the total evidence that I won in the stated circumstances. So the rest of the total evidence disconfirms "my" predictor's model, likely strongly enough that his model is disconfirmed on net.

The problem seems to be that coarse-grained predictions only provide coarse-grained information about the accuracy of our models of the world. Since we (qua rationalists) are interested in predictions because they are potentially strong evidence about our models, we should try to make our explicit predictions more fine-grained, as you recommend (while considering the obvious costs).

I agree that "right for the wrong reasons" is an indictment of your epsitemic process: it says that you made a prediction that turned out correctly, but that actually you just got lucky. What is important for making future predictions is being able to pick the option that is most likely, since "being lucky" is not a repeatable strategy.

The moral for making better decisions is that we should not praise people who predict prima facie unlikely outcomes -- without presenting a strong rationale for doing so -- but who then happen to be correct. Amongst those who have made unusual but successful predictions we have to distinguish people who are reliably capable of insight from those who were just lucky. Pick your contrarians carefully.

There's a more complex case where your predictions are made for the "wrong" reasons, but they are still reliably correct. Say you have a disorder that makes you feel nauseous in proportion to the unlikeliness of an option, and you habitually avoid options that make you nauseous. In that case, it seems more that you've hit upon a useful heuristic than anything else. Gettier cases aren't really like this because they are usually more about luck than about reliable heuristics that aren't explicitly "rational"

I've been posting this around a lot lately (including on "Gettier in Zombie World"), still looking for a solid response.

I think Bayesian probability actually resolves Gettier problems, completely and (ironically, because Bayesian probability doesn't concern itself with this in the slightest) satisfyingly. Understanding that we only know likelihoods, not facts, is enough.

Situation: I know John had 10 coins in his pocket. I think he got the job. I don't know that Smith had 10 coins in his pocket. Do I "know" that the person who got the job had 10 coins in their pocket?

Classic Gettier Interpretation:

• Belief-Jones had 10 coins in his pocket
• Belief-Jones got the job
• • Conclusion-The person who got the job had 10 coins in their pocket

Bayesian Gettier Interpretation(Example numbers used for ease of intuition; minimal significant digits used for ease of calculation):

• Belief-It's likely (90%) John got the job
• Belief-It's likely (90%) John had 10 coins in his pocket
• • Conclusion-It's likely (81%) John got the job with 10 coins in his pocket

...and...

• Belief-It's unlikely (10%) someone other than John got the job
• Belief-It's possible (50%) a given person had 10 coins in their pocket
• • Conclusion-It's unlikely (5%) that someone other than John got the job with 10 coins in their pocket

...thus...

• Belief-It's likely (81%) John got the job with 10 coins in his pocket
• Belief-It's unlikely (5%) that someone other than John got the job with 10 coins in their pocket
• • Conclusion-It's likely (86%) that someone got the job with 10 coins in their pocket

The appropriate thing to do is apply (an estimate of) Bayes rule. You don't need to try to specify every possible outcome in advance; that is hopeless and a waste of effort. Rather, you extract the information that you got about what happened to create an improved prediction of what would have happened, and assign credit appropriately.

First, let's look at what we're trying to do. If you're trying to make good predictions, you want p(X | "X") to be as close to 1 as possible, where X is what happens, and "X" is what you say will happen.

If an unbiased observer initially would have predicted, say, p(you win at fencing) = 0.5, then initially the estimate of your accuracy for that statement would be 0.5; and after winning 14 touches in a row it would probably be somewhere around 0.999, which is nearly as good as it having been true (unless your accuracy is already in the 99.9%+ range, at which point this doesn't help refine the estimate of your accuracy).

So, you don't need to ask more precise questions. You do need to honestly evaluate in aborted trials whether there were dramatic shifts in the apparent probability of the outcome. When doing these things in real life, actually going through Bayesian mathematics is probably not worthwhile, but keeping the gist of it certainly is.

My prediction about the outcome of the fencing bout could better be expressed as multiple predictions, for instance "I will score more points than my opponent" and "I will win the bout."

What if lightning struck the building a day earlier and the match is called off? Of if you get arrested on the way to the match and thus you forfeit by failure to show up? Or if your opponent forfeits, which makes the first prediction wrong?

In those circumstances, there's no reason to modify your prediction for the next time. While they were wrong, none of the wrongness had anything to do with your fencing skill.

A better set of statement might be "I will not score less points than my opponent." and "I will not lose the bout on points.". Although those logically reduce to almost the same prediction.

In fact, that seems like a reasonable general principle. Predictions often should state what won't happen than what will. Stating what won't happen allow you to automatically exclude many random outliers that are outside the scope of the prediction you're trying to make.

If you state the prediction in such way that an 'act of God' means you are right, not wrong, then when they occur you don't have to try to justify why your wrongness doesn't really count. Granted, it doesn't mean you were right, either, but, mentally, it's probably safer to say 'I technically got that right, but it doesn't really count' than 'I technically got that wrong, but it doesn't really count.'.

Upvoted. This article makes an extremely important point.

I use correct/incorrect to refer to 'prediction and outcome coincided'/'made what turned out to be the most favourable choice'/etc.

I use right/wrong to refer to 'made the best prediction'/'chose the outcome that, as far as could be told, would be most favourable'/etc.

Smith was correct and wrong.

Although one might expect ambiguity to be a problem with these terms (e.g. 'right' becoming overloaded to the point of equivocation), in my experience it hasn't been one once they have been explained.

The thesis of the right/correct distinction is defying the data.

The antithesis is regret of rationality, i.e. predictably losing due to a flaw in a model. This is a hazard that arises from devotion to a theory or undervaluing data, which lead to insistence one is still right even as the defeats pile up.

As for the No True Fencing Victory thingy: that's simply insufficient correspondence between one's internal understanding (e.g. one visualises winning in a fifteen-point whitewash, specifies 'winning', and is saved from the jaws of defeat by a technicality.) Such cases are of being too imprecise or inaccurate. I generally lean very heavily towards 'a win is a win', because anything else often seems to stem from an unrealistic expectation of perfect specification.

To me it seem the problem here is simply trying to treat natural language sentences as real things when they are only an approximate abstraction, that breaks down in these kinds of edge cases.

There are no discrete "belief's" with "justifications", there are only a probability distribution over the configuration space of all possible histories of sensory input. And that's just another layer of abstraction really, but it's enough for now.

Here is what I think is a better example of the Gettier problem, and a subsequent reason the Gettier problem is flawed in its definitions of truths.

You are driving down the highway, passing what appear to be several dozen barns. Unknown to you, all but one of these barns is a stage prop cutout. You decide to stop at one of these barns and by luck it is the only real one. You now have a belief (which is that the barns you see are real), which is justified, and in this case, true. But it cannot be called knowledge. Why? Because the belief is imprecise and leaves room for vagaries. A belief should describe the fundamental mechanisms of the universe. i.e. the presence of light patterns in format X indicates structure Y, because light interacts in ways Z. In this case the belief about the barns is unjustified and untrue, because there is an additional way format X could be created, by structure Y2 and light interaction Z2 (the cutout). Discovery of the real barn is only weak evidence for the belief that format X indicates a real barn, as the discovery proves the possibility thereof, but does not eliminate the alternative (cutouts). Under this new definition of belief, a concept of the universe fundamental mechanisms, as opposed to informal correlations, only accurate and precise beliefs that allow prediction generation constitute knowledge.

But this is not how we think. And for very good reason. Typically a scenario in which all options appear identical to cursory examination, and in which detailed examination provides some conclusion about one option, it can be a huge waste of time and effort to generate all theoretically possible contradictory scenarios and test them, not to mention the possibility that you may not think of or be able to test all such options. So our brain takes a mental shortcut. Barns appear to be same? Check. Barn 10 is a 3d barn? Check. Therefore all barns are 3d. Though nothing was falsified, it is a useful informal deduction which only fails us in extreme circumstances such as the problem listed above. But there is a very good reason that we don't use such logic in scientific experimentation. When we have not repeatedly experienced a phenomenon and have no hard-set reason to believe a correlation indicates causation, falsification is all we can trust. We don't have the huge backdrop of everyday data to fall back upon. Oftentimes we have a hard time realizing this though, and make assumptions as if we have such a backdrop when we don't.

Scientific Method: Don't do that.