Secure Your Beliefs

Will Sawin:

If it's a high-status belief, due to correctness of the Titanic analogy, very few high-status people will ask that [what's the harm]. They are much more likely, instead, to criticize your argument that the belief is wrong.

Would that it were so! When it comes to really pervasive and established high-status delusional beliefs, with very few exceptions, what you'll get is at best a criticism whose content is far below the usual scholarly standards, and at worst just mindless sneering and moral indignation.

This holds both for those high-status false beliefs that are a matter of ideological orthodoxy and those that are a matter of venal interest. (The overlap between those is, of course, larger than the pure part of either category, and people have no problem coming up with honest rationalizations for their professional, ideological, and other interests.)

(In addition, the whole methodology of collecting individual examples works well for distributed mistakes like the passengers', but not for large, single mistakes like the captain's.)

In many cases, high-status delusional beliefs don't result in a single identifiable disaster, but rather in lots of widely distributed harm and suffering. (In this sense, the Titanic analogy breaks down.)

In these cases, however, a collection of touching human-interest stories will likely fail to strike the intended note among high-status readers, and will instead be dismissed as nefarious extremist propaganda.

If I go around saying "This belief is wrong! Hey everyone, did you know that this belief is wrong?" and it's a low-status belief, high-status people are likely to ask "What's the harm?"

Not really. It depends on the exact way the belief in question is perceived in high-status circles. In some cases, you'll win status points out of all proportion with the actual importance of the problem and without much scrutiny of the accuracy of your arguments (the phrase "raising awareness" comes to mind). In other cases, you won't register on high-status people's radar even if you have a solid case, simply because the issue doesn't happen to be a status-fertile cause. In yet other cases, it may happen that while a belief is low-status, it is also considered uncouth to attack it all-out; one is supposed to scoff at it in more subtle and oblique ways instead.

You don't compare harms and benefits of the same process. You compare properties of the alternative processes. Comparing harms with benefits of a single possible decision is a heuristic for comparing the decision with the default of doing nothing, but often there is no such default option.

(This remark is not relevant to the conversion, just a technical note on your usage of harms vs. benefits.)

I see. I certainly don't see anything wrong with condemning epistemic negligence in the abstract. I guess it can't hurt to make the point again.

This part put me on the wrong foot:

well-placed condemnation from people I respected changed my attitude toward gay marriage pretty quickly

Surely they were condemning your specific beliefs here.

Its entirely possible that most Americans should trust doctors more (and homeopaths and themselves less) while above average Americans should trust doctors less.

I absolutely agree that most smart people trust doctors too much. If I trusted doctors more, I'd have gotten a completely unnecessary total disc replacement at the ripe old age of 22. As it was, I trusted him too much- I'm ashamed that I actually considered it even though I understood all the reasons the next doctor used in his judgement of "total disc replacement!?!? He's insane!"

Starting on page 114 of his book about randomness, The Drunkard's Walk, Physicist Leonard Mlodinow tells a real-life story about being told by his doctor that the results of a blood test showed that there was a 999 out of 1,000 chance he had AIDS and would be "dead within a decade." Mlodinow (who did not have AIDS) uses this as an introduction to Bayes' Theorem. His doctor had made exactly the error described in An Intuitive Explanation of Bayes' Theorem.

Still, isn't a solemn vow to pursue truth better than no vow at all? Even if a burning itch to know is better... How DO you create a burning itch to know in someone who doesn't have one?

ho will have the courage to attack those with equal decisiveness and fervor?

The level does not have to be equal. It just has to be effective. A quiet word in one place can have more resonance than a rant in another.