Non-communicable Evidence

I think this is what many people find confusing about Bayesian reasoning: the nods to subjectivity. Some people I've discussed it with often say things like "But it's still subjective!" or "You're pulling those numbers out of your ass!" Well, yes, maybe. But calibration games, immediate-feedback systems like the Good Judgement Project, and general awareness of priors are being shown as better than chance, and better than intuition, when the RESULTS are objectively measured.

If I had to speculate on why people react so negatively, I'd say it's because of the false dichotomy between "objective" and "subjective." Objective: numbers, math, computer programs. Subjective: fuzzy, things that aren't science. So, saying that something can be evidence in a rational approach AND be subjective...is confusing. They aren't thinking about weighting the evidence accordingly. They're annoyed that it's counted at all.

You shouldn't say, "Well if you can't provide any evidence, you shouldn't believe what you do." The problem with that statement is that it assumes that the person has "no evidence".

Another reason not to do that is that people often come up with fake reasons for believing something and convince themselves that it is the actual reason. The more socially unacceptable it is to fail to provide an adequate reason for a belief, the more this is encouraged.

Unfortunately I cannot communicate why I think Christianity is true; it's a gestalt thing - it just makes sense, it can't be any other way in the light of all the evidence.

-- Any number of quite successful CEOs, neurosurgeons, writers.

When you're thinking "in explicit mode" -- seriously trying to assess probabilities, considering the possibility that your own judgement might be in error, etc. -- you should (1) take account of your feelings and hunches and whatnot, but (2) not take them at face value. E.g., Douglas Crockford should think "This is only a gut feeling, but I've found that my gut feelings in this field are right much more often than they're wrong. So my gut feeling is good evidence."

This evidence is (at least in principle) communicable. In fact, it seems like to some extent it's been successfully communicated to you. If Douglas Crockford ever tells you of a gut feeling he has about something related to good Javascript programming style, you will take it seriously and may well modify how you write your code.

Now, if Crockford is operating in system-1 mode instead, just going with what his gut tells him, then indeed his evidence isn't communicable to you. I don't see why this is of more concern than the fact that when a cricket fielder is catching a ball, the information his eyes are feeding him that helps him catch the ball isn't communicable to you.

I wonder if it's worth separating out the track record across working by gut and by communicable-evidence channels? Like, if the last 30 predictions he made, he gave reasoning for 27 of them, and all of those were right but the other 3 were wrong, then that's a bad track record for this sort of thing.

Seems obvious, but it might be over-fitting, especially since 'trust me' cases might be rare.

You shouldn't say, "Well, if you can't provide any evidence, you shouldn't believe what you do."

There are at least two justifications I can think of for this.

1. A reductio: basically saying "since you do believe X, you must have some evidence, so see if you can figure out what it is". This can start a discussion on how not all evidence is definite/based on statistics etc.
2. If they don't have a good track record, then you're trying to influence them to drop the belief because of lack of evidence (this rarely works, unfortunately).

I treat conversations like this as a communication problem, since the information should be communicable if it's based in Douglas Crockford’s mind. I try to find what the intuition is based on, which helps i) send me in the right direction and ii) avoid double-counting the evidence if I find it independently.

To me, the labels “skill” or “intuition” mean that something is not well understood enough to be communicated objectively. A total understanding would include the ability to describe it as one or more clear-cut techniques or algorithms.

Do you go through an explicit updating procedure on a regular basis in conversations like that? I don't and I think most people don't.

Agree with this. Have had to learn to do this with some people I know who intuit a lot of their beliefs, but tend to be right a lot.

This seems like an informal extension of the no disagreement theroem to situations where the agent are not perfect rationalists.

The claim is not observable in any way and offers no testable predictions or anything that even remotely sounds like advice. It's unprovable because it doesn't talk about objective reality.