Belief in Belief

Eliezer, my understanding is: "belief is to believe in something".

Whether you call it science fiction, heuristics, overcoming bias, history, a belief is a belief. You can't prove belief as it's self-subjective. You can't tell someone what they feel is wrong. Each individual has there equation when it comes to understanding the "dragon" within themself. If dragons can't be verified as they have never been verified based on history, why do people still feel the need to believe in dragons and continue to discuss the subject and be fascinated by it?

Just Curious Anna

Anna, this blog is too advanced for you and you should not be commenting on it. Go read The Simple Truth until you understand the relation between a map and the territory.

[EDIT: I deleted an additional comment from Anna in this thread.]

Oh great, now I'm going to think "There's no dragon in my garage" every time I open my garage door for the next week...

I enjoyed The Simple Truth, thanks for linking it.

[["If the pebbles didn't do anything," says Autrey, "our ISO 9000 process efficiency auditor would eliminate the procedure from our daily work."]]

This "ISO 9000" hypothesis has not been supported by direct observation, unfortunately...

From the post:

While I disagree with Dennett on some details and complications, I still think that Dennett's notion of belief in belief is the key insight necessary to understand the dragon-claimant. But we need a wider concept of belief, not limited to verbal sentences.

If you've read Dennett on beliefs, you'll appreciate that this "wider concept" based on behavior and predictability is really at the heart of things.

I think it is very difficult to attribute a belief in dragons to this "dragon-believer". Only a small subset of his actions - those involving verbal avowals - make sense if you attribute a belief in dragons to him. There is a conflict with the remainder of his beliefs, as can be seen when he nonchalantly enters his garage, or confabulates all sorts of reasons why his dragon can't be demonstrated.

But as you have shown, everything makes sense if you attribute a related, but slightly different belief, namely "I should avow a genuine, heartfelt belief in dragons". Perhaps we can say that this man (and the religious man, since this is the real point) doesn't just believe in belief, but they believe that they believe. He tries to make a second-order belief do the work of a first-order belief.

How does this compare with Popper's theory? In the instance above, it's clear that belief in belief doesn't make sense. But things may not be as clear. Won't an event with low probability look like the invisible dragon before it happens?

Anna, If you're talking about real dragons, the theory that made the most intuitive sense to me (I think I read it in an E.O. Wilson writing?) is that dragons are an amalgamation of things we've been naturally selected to biologically fear: snakes and birds of prey (I think rats may have also been part of the list). Dragons don't incorporate an element of them that looks like a handgun or a piping hot electric stove, probably because they're too new as threats for us to be naturally selected to fear things with those properties.

Eliezer, Very interesting post. I'll try to respond when I've had time to read it more closely and to digest it.

I want my wife to read this, but I don't think she'd believe it.

This post helps me understand some of the most infuriating phrases I ever hear (which the title immediately reminded me of): "it doesn't matter what you believe as long as you believe something", "everyone has to believe in something", "faith is a virtue", &c. It makes sense that if a person's second-order belief is stronger than their first-order belief, they would say things like that.

Was the reply to Anna serious? That's outrageous.

Shocked, it wasn't my first interaction with her.

I like Eliezer's essay on belief very much. I've been thinking about the role of belief in religion. (For the sake of full disclosure, my background is Calvinist.) I wonder why Christians say, "We believe in one God," as if that were a particularly strong assertion. Wouldn't it be stronger to say, "We know one God?" What is the difference between belief and knowledge? It seems to me that beliefs are usually based on no data. Most people who believe in a god do so in precisely the same way that they might believe in a dragon in the garage. People are comfortable saying that they know something only when they can refer to supporting data. Believers are valiantly clinging to concepts for which the data is absent. Most people who believe in a god do so in precisely the same way that they might believe in a dragon in the garage.

Regarding the dialogue between the dragon claimant and his challengers, why didn't the challengers simply ask the claimant, "Why do you say that there is an invisible, inaudible, non-respiriating, flour-permeable dragon in your garage?"

I've got a double garage... what if the dragon sneaks out one door while I'm coming in through the other door, then comes in behind me through the second door while I look for it outside the first door?? Dragons everywhere now!!

The parable was original with Antony Flew whose Theology and Falsification can be found here http://www.stephenjaygould.org/ctrl/flew_falsification.html

The parable your refer to by Sagan I think should be attributed to Antony Flew whose Theology and Falsification is available online. http://www.stephenjaygould.org/ctrl/flew_falsification.html

Trying to work out the biases of the new antispam filter. Frequency of comments from same individual in same thread ?

That lastr one got through, so let's try : Random malfunction ?

Anna: Whether you call it science fiction, heuristics, overcoming bias, history, a belief is a belief. You can't prove belief as it's self-subjective. --That only makes for more of a reason it should only be self affecting, too many people try to influance the actions of others bases on their dragons

You can't tell someone what they feel is wrong. --Yes I can. "There's a dragon in my bathroom"... (careful examination of the bathroom)... "No, there isn't, you're wrong."

Each individual has there equation when it comes to understanding the "dragon" within themself. --And it needs to be overcome with logic and reason.

If dragons can't be verified as they have never been verified based on history, why do people still feel the need to believe in dragons and continue to discuss the subject and be fascinated by it? --simply because they were raised with it, taught to dis-belive any evidence provided, and been shown that those who disagree with it are 'out to get them'

If there exists a conventional syntax for quoting another comment, I think a commenter should use that convention rather than invent his own syntax, don't you?

There is a distinction between Belief and Knowledge. We can believe things that are untru and disbelieve things that are true.

At least in the case of religious people who are actually convinced God exists, I think the difference between belief and knowledge is thus: Belief is when you think something is true but it's controversial. Knowledge is when you think someting is true and think everyone would agree with you.

I knew as soon as I read the first paragraph that the comments would start discussing religion, haha...

John Rozendaal, I never really thought about the "We know"/"We believe" distinction before. Seems to me like the church conditions people to think of their ideals as a belief, so that the thought of 'knowledge' wont creep into their minds and make them think. Thinking is the church's worst enemy.

I believe the difference between belief and knowledge stems from experience. Knowledge is accepted as proven data, whether from the experience of others or through your own direct experience. Beliefs are accepted as something proven to varying levels of personal satisfaction inferred directly from experience, dependent upon the person and the belief.

This makes the two rather interdependent in my view, for how can one know without believing that his knowledge is truth and how can one believe without knowing something to believe in?

In the case of modern religion, people take the belief others express as knowledge and use it as a basis for their own beliefs.

X: There's something that makes beliving and knowledge quite different, and that’s truth which isn’t inside one person head but out there, in reality. I’m sure that if we ask this man if he knows there is a dragon in the garage he will reply affirmatively, no doubt about it, but the truth is that there is no dragon and he just think he knows is in there. But the man doesn’t know anything, he believe a lie and he is making excuses to protect the lie, and one of those excuses is that he knows is in there, is not a belief.

I think this is one of humanity greatest weakness; the need to detach from reality and defend beliefs that are obviously wrong, I understand the psychological need to do so, but, in my opinion, still is a sign of weakness. As Eliezer said we should find joy in what is real.

"There's something that makes beliving and knowledge quite different, and that’s truth which isn’t inside one person head but out there, in reality."

Ehm, let me ask you this: Are you 100% sure that the sun will come up tomorrow?

All evidence points that way, yes. We have a fair idea of what is going on yes. But that's where the ball stops - we will never know with 100% certainity.

When we stop acknowledging that the science of tomorrow may produce evidence that will turn our whole world-view upside down, is when Science becomes Religion.

I'm not saying that we need to start taking mediums seriously and base our lifealtering decisions on numerology. I'm merely saying that the things you take seriously today, the things you'd base your lifealtering decisions on today may be falsified tomorrow, redeemed the week after, only to be shot down again with the latest research come this time next year.

The 'Truth' may be out there, but it needs to be approcached empirically with a clear understanding of the fact that even repeated measurements of the same thing will only ever give us circumstancial evidence that may be influenced by our abilities to measure and reason. I don't think we ever posess true knowledge. Instead we have beliefs that can or cannot stand up to empirical scrutiny. Beliefs that must still be challenged on a regular basis, and acknowledged for what they are.

Does the idea that it is a good thing to subject our beliefs (and even our belief in belief) to logical and analytical scrutiny count as belief in itself or is it so justifiable as to count as knowledge? If so, what is the justification?

I've always thought that the idea of "believing in" things was very curious. This is a very thought-provoking article. Every time I engage a debate about this subject (the relevance or usefulness of beliefs) someone is sure to say something about beliefs existing for the benefit of the believer. My feeling is that with most beliefs and with most believers, there is an internal acknowledgement of the falsifiablity of their belief which is outweighed by the perception that some benefit is derived from the belief. What I interpret from this is that most believers subtley admit their own practice of belief in belief. I also feel that even the idea of whether or not one believes in believing in belief can enter the mind of the mundane thinker at such an admission.

The real question is not "is there a dragon?", but "why is it having sex with my car?"

Chelo: "I don't think we ever posess true knowledge."

I KNOW I went to Tesco's this morning. Am I wrong? Discuss!

Main post "The claimant must have an accurate model of the situation somewhere in his mind, because he can anticipate, in advance, exactly which experimental results he'll need to excuse."

I know this is a bit of a side issue, but how do you justify this claim from the example given? You don't need such a model to give the answers he gives. Surely you once engaged in late-night undergraduate pseudo-intellectual discussions where you held an ultimately untenable viewpoint but still fended off such questions on the fly?

Perhaps though this is just a problem arising from the rather simplistic metaphor. A dragonista can postulate a dragon and then, as in your example, refute all challenges by simply denying all interactions with the real world, although then of course he's not really saying anything at all. The religionist has a much more difficult trick to perform. He cannot take the dragonista's line as his god must interact in some way with the world to have any meaning. He is faced with having to reconcile the interactions he needs from his god (e.g. responses to prayer) with the apparent absence of physical evidence for them. This DOES require the building of the consistent framework you propose, so that he can fend off new challenges without falling into a trap which concedes the non-existence of his god. The convolutions exhibited by fundamentalist Christians when trying to construct such a reconciliation between what they need to believe and the contrary evolutionary evidence are a better example of this.

"Those who find this confusing may find it helpful to study mathematical logic, which trains one to make very sharp distinctions between the proposition P, a proof of P, and a proof that P is provable"

This is a bit of a side question, but wouldn't a proof that P is provable be a proof of P? In fact, it sounds like a particularly elegant form of proof.

Blarg... okay this one is tripping me up. There are two parts to this comment. The first part is quasi-math; the other is not. It is very much a brain dump and I have not edited it thoroughly.

EDIT: I think I managed to get it cleared up and responded with a second comment. FYI.

Let B(X) mean belief in X where belief is defined as a predictor of reality so that reality contains event X. Using "There is a dragon in my garage" as X we get:

• B("There is a dragon in my garage.")
• B("There is not a dragon in my garage.")

I think it is okay to write the latter as:

• B(~X) where X is "There is a dragon in my garage."

So far okay and both can be verified. The problem comes when X is "There is an unverifiable dragon in my garage."

• B("There is an unverifiable dragon in my garage.")
• B("There is not an unverifiable dragon in my garage.")

Both of these are unverifiable, but the latter is okay because it matches reality? As in, we see no unverifiable dragon so the ~X is... what, the default? This confuses me. Perhaps my notation is wrong. Is it better to write:

• B(X)
• ~B(X)

If B(X) is belief in X, B(~X) != ~B(X). This way we can throw out the unverifiable belief without creating a second unverifiable belief. All of this makes sense to me. Am I still on track with the intent of the post? This implies that B(X) and B(~X) are equally unverifiable when X is unverifiable.

Next is belief in belief:

• B(B(X))

Of which I think you are arguing that B(B(X)) does not imply B(X). But are you also saying that B(X) implies B(B(X))? And this is how people can continue to believe in something unverifiable?

I feel like I am drifting far away from the purpose of this post. Where did I misstep?

Here is my second attempt, this time with no math:

Carl Sagan used this parable to illustrate the classic moral that poor hypotheses need to do fast footwork to avoid falsification. But I tell this parable to make a different point: The claimant must have an accurate model of the situation somewhere in his mind, because he can anticipate, in advance, exactly which experimental results he'll need to excuse.

Would there be any experimental results that he wouldn't need to excuse? Is there some form of invisiodragonometer that beeps when he goes into his garage? Would the scenario change any if the subject was genuinely surprised when no sounds of breathing were heard and the oxygen levels remained the same and still offered up excuses of inaudible and non-breathing? How would the typical believer in atoms defend their existence if we wandered into the garage and complained about no breathing sounds?

I can think of simple answers to all of these questions, but it makes me think less of the usefulness of your conclusion. When I think of unverifiable beliefs I think of examples where people will spend their whole life looking for physical proof and are constantly disappointed when they do not find it. These people don't have an accurate model of the situation in their mind. The example of invisible dragons still applies to these people while your claim that they dodge in advance does not seem to apply.

So... again, I feel like I am missing some key point here.

[I]f you believe in belief, you cannot admit to yourself that you only believe in belief, because it is virtuous to believe, not to believe in belief, and so if you only believe in belief, instead of believing, you are not virtuous. Nobody will admit to themselves, "I don't believe the Ultimate Cosmic Sky is blue and green, but I believe I ought to believe it" - not unless they are unusually capable of acknowledging their own lack of virtue. People don't believe in belief in belief, they just believe in belief.

I can think of examples where someone fully admits that they believe it would be better to believe X but as hard as they try and as much as they want to, they cannot. These people are often guilt ridden and have horrible, conflicting desires, but it doesn't take much imagination to think of someone who simply states the belief in belief X without emotion but admits to not believing X. At least, I can hear myself saying these words given the right circumstances.

Believing in belief of belief seems like something else entirely unrelated to dragons in garages or unverifiable beleifs. This, again, makes me feel as if I am missing a crucial piece of understanding throughout all of this. If I had to potshot at the missing pieces I would aim toward the definitions of belief. Specifically, what you are calling beliefs aside from predictors of reality. (And even there, I do not know if I have a correct translation.)

I do not know if you have any desire to discuss this subject with me. Perhaps someone else who knows the material is willing? I sincerely apologize if these types of responses are frustrating. This is how I ask for help. If there is a better way to ask I am all ears.

I also liked this essay very much. Many times it has occurred to me that "There are people who may claim to believe in Y, but deep down they know just as well as everyone else that Y is not true."

I would add that our conversation with the hypothetical dragon claimant is not likely to get to the point where you are discussing bags of flour. Because by that point he will probably be extremely angry with you and one way or another the conversation will be concluded.

As the old joke goes, "Shut up, she explained."

Actually I think it was Steve Sailer who pointed out that with any highly controversial issue, there is likely to be some truth floating around which people do not want to face. Arguably it's the same concept.

I also liked this essay very much. Many times it has occurred to me that "There are people who may claim to believe in Y, but deep down they know just as well as everyone else that Y is not true."

I would add that our conversation with the hypothetical dragon claimant is not likely to get to the point where you are discussing bags of flour. Because by that point he will probably be extremely angry with you and one way or another the conversation will be concluded.

As the old joke goes, "Shut up, she explained."

Actually I think it was Steve Sailer who pointed out that with any highly controversial issue, there is likely to be some truth floating around which people do not want to face. Arguably it's the same concept.

Hi, I'm new to this blog. I haven't read all the essays and whatnot, so if you read something I say that you think is bull, I'd like it if you linked me to the essay.

Anyway... Regarding the dragon and the garage scenario: what would the dude say if I were to ask "how do you know the Dragon is there if it's impossible to know if he's there?"

Might belief in belief occasionally be valuable when overcoming bias? It would be better to correct my beliefs, but sometimes those beliefs come from bias. I might be convinced in my head that standing on the glass floor of an airplane and looking down is totally safe - this specially-modified-for-cool-views airplane has flown hundreds of flights - yet in my heart deeply believe that if I step onto it I will fall through. I might then choose to "believe in the belief that it is safe to take a step", while all my instinctual reactions are based on a false model. The cognitive dissonance is due to my inability to integrate something so foreign to the evolutionary environment into my belief structure.

Belief in disbelief:

One of our neighbors in Tisvilde once fixed a horseshoe over the door to his house. When a mutual acquaintance asked him, 'But are you really superstitious? Do you honestly believe that this horseshoe will bring you luck?' he replied, 'Of course not; but they say it helps even if you don't believe it.'

— Niels Bohr

(Note: This is often retold with Bohr himself as the one with the horseshoe, but this quote appears to be the authentic one.)

I wonder how common that is, believing that you don't believe something but acting in a way that implies more belief in it than you acknowledge. One other example I experienced recently: For whatever reason, my mom had a homeopathic cold remedy lying around. (I think a friend gave it to her.) She and I both had colds recently, so she suggested I try some of it. The thing is, she gives full assent to my explanations of why homeopathy is both experimentally falsified and physical nonsense; she even appeared to believe me when I looked at the ingredients and dilution factors and determined that the bottle essentially contained water, sugar, and purple food colouring. But even after that, she still said we may as well try it because it couldn't hurt. True, it couldn't hurt... but "it can't hurt" doesn't sound like really understanding that the bottle you're holding consists of water, sugar, and purple.

Another instance may be former theists who still act in some ways as though they believe in God (an interesting mirror image of current theists who don't act as though they really believe what they profess to believe), though in my experience many of them consider it to be bad habit they're trying to break, so I'd be less inclined to call it belief in [dis]belief, I'd take that as something more akin to akrasia.

Wow. So, I'm basically brand new to this site. I've never taken a logic class and I've never read extensively on the subjects discussed here. So if I say something unbearably unsophisticated or naive, please direct me somewhere useful. But I do have a couple comments/questions about this post and some of the replies.

I don't think it's fair to completely discount prayer. When I was a young child, I asked my grandmother why I should bother praying, when God supposedly loved everyone the same and people praying for much more important things didn't get what they wanted all the time.

She told me that the idea is not to pray for things to happen or not happen. If I pray for my basketball team to win our game (or for my son to get well, or to win the lottery, or whatever) then based on how I interpret the results of my prayer I would be holding God accountable for me getting or not getting what I wanted. The point of praying, as she explained it, was to develop a relationship with God so I would be able to handle whatever situation I found myself in with grace. Even though we often structure our prayers as requests for things to happen, the important thing to keep in mind was how Jesus prayed in the garden before he was crucified. Even though he was scared of what was going to happen to him and he didn't want to go through with it, his prayer was "your will, not mine". He didn't pray for things to go his way, although he acknowledged in his prayers that he did have certain things that he wanted. The point of the prayer was not to avoid trials or fix their outcome, but to communicate with God for the strength and courage to hold fast to faith through trials.

Now, I'm certainly not citing my grandmother as a religious or theological expert. But that explanation made sense to me at the time, partially because I think you could probably that it would have the same benefit for people regardless of whether or not there was actually a God to correspond to the prayers, which jives well with how I believe in God.

Maybe I'm misunderstanding the post, but I think I have something like believing that I ought to believe in God, although I've always phrased it as choosing to believe in God. Even though I was raised Catholic, I never felt like I really "believed" it. For as long as I can remember, the idea of "belief" has made me incredible uncomfortable. Every time a TV show character asked "didn't you ever just believe something" I would cringe and wonder how anyone could possibly find such an experience valid when anyone else could have an alternate experience.

Secretly, I'm glad that I've never felt any kind of religious conviction. If I did, then I would have to prize my subjective experience over someone else's subjective experience. I'm quite aware that there are a multitude of people that have had very profound experiences that make them believe in one doctrine or another to the exclusion of all others, and that's something I can't really understand. Knowing that other people exist that feel equal conviction about different ideas of God with the same objective evidence makes it impossible for me to have any sort of belief in a specific God or scripture, at least at the level of someone who believes with enough conviction not to be perfectly comfortable with the idea that I'm wrong.

That said, I consider myself Catholic. I don't agree with all the doctrine and I don't think I could honestly say I think my religion is correct and other religions are wrong in any way that corresponds to an objective reality. But I choose to believe in this religion because what I do really believe deep down is that there is some higher order that gives meaningfulness to human life.

I consider it to be rather like the way I love my family- I don't objectively think that my family is the best family in the world, the particular subset of people most deserving of my love and affection. But they're my family, and I'll have no other. I can love them while still acknowledging that your love for your family is just as real as mine. Just because they're different experiences doesn't make them more or less valid- and just because it isn't tangible or falsifiable doesn't make it any less potent. Even so, I'm always curious if I'm really an atheist, or maybe an agnostic, since I don't really believe it beyond my conscious choice to believe it (and a bit of emotional attachment to my personal history with this specific religion).

Whew. That was a lot of words. Anyways, I'm sure that I've got plenty of logical and rational flaws and holes. Like I said, I'm basically brand new to all the ideas presented here, so I'm going to try and thrash my way through them and see what beliefs I still hold at the end.

Surprised not to find Pascal's wager linked to this discussion since he faced the same crisis of belief. It's well known he chose to believe because of the enormous (inf?) rewards if that turned out to be right, so he was arguably hedging his bets.

It's less well known that he understood it (coerced belief for expediency's sake) to be something that would be obvious to omniscient God, so it wasn't enough to choose to believe, but rather he actually Had To. To this end he hoped that practice would make perfect and I think died worrying about it. this is described in the Wikipedia article in an evasive third person, but a philosophy podcast I heard attributed the dilemma of insincere belief to Pascal directly.

Fun stuff.

Very interesting. I have transhumanist beliefs that I claim to hold. My actions imply that I believe that I believe, if I understand this properly.

A prime example would be how I tend to my health. There are simple rational steps I can take to increase my odds of living long enough to hit pay dirt. I take okay care of myself, but could do better. Much better.

Cryonics may be another example. More research is required on my part, but a non-zero last stab is arguably better than nothing. I am not enrolled. It feels a bit like Pascal’s Wager to me. Perhaps it is a more valid form of the argument, though. Hoping for a scientific miracle seems essentially different than hoping for a magical miracle. Scientific miracles abound. Artificial hearts, cochlear implants, understanding our origins, providing succor to imbalanced minds, the list goes on. Magical miracles… not so much.

Heck, I could stop forgetting to floss daily! (There seem to be strong correllations between gum disease and heart disease).

I anticipate as if there will be no radical life extension available within my life time, but I will argue for the possibility and even likelihood. Do I have this correct as a type of belief in belief?

This is fascinating. When I look back at the thought patterns of my younger self, I can see so much of this belief-in-belief. Despite being raised religious, I came to an agnostic conclusion at around age ten, and it terrified me, because I very much wanted to believe. To my mind, people with faith had a far greater sense of morality than those without, and I didn't want to fall into that latter category.

So I proceeded as if I believed, and eventually came to make justifications along the lines of 'ritual X accomplishes outcome Y' where Y was something psychologically valuable, for example a sense of community. That made X a good idea even if I didn't truly believe the theology involved.

When I was first told I had a second-order relationship to my belief, I was very insulted. It was as if I'd defined a good person as a religious one, and by challenging my belief that person was challenging my intrinsic worth (despite the fact that they were an atheist themselves and clearly thought nothing of the sort.) Cognitive distortion at its finest.

It took a profound shift in my thinking about the role of religion in morality before I could accept that it was alright not to believe. The rest followed nicely.

As for Santa Claus? I pretended I believed (despite knowing the absolute impossibility of it being true) for three whole years. The idea that there was such a great conspiracy that every adult seemed to be complicit in really worried me, at six years old, and made me afraid to speak the truth. I knew they wanted me to believe, so I let them think I did.

If I ever have children of my own, needless to say, Santa will be introduced as an enjoyable fiction and nothing more.

It doesn't seem to me that this post actually makes any coherent argument. It spends a fair amount of words using seemingly metaphysical terms without actually saying anything. But that's not even the important thing.

Is this post supposed to increase my happiness or lifespan, or even that of someone else?

As if the human brain only had enough disk space to represent one belief at a time!

If you think of belief as something like "representing the world as being a certain way", then a belief in Real Beliefs might have followed from profound ignorance of neursicence. But there are plenty of other ways of getting there. For instance, if one thinks of belief as expalaning actions, then the Real Belief is the one attested by action. Someone who does not giove to charity does ot Really Believe in chaity even if they say they do. Actions either occur or do not: one cannot perform contradictory actions , so one cannot entertain contradictory Real Beliefs. For reasons that have nothing to do with the number of neurons in the brain.

Yet again, a Philosophers are Idiots claim turns out to be poorly founded.

This post taught me a lot, but now "There is no invisible dragon in my garage" will be popping into my head whenever I see a garage.

I noticed that I was confused by your dragon analogy. 1) Why did this guy believe in this dragon when there was absolutely no evidence that it exists? 2) Why do I find the analogy so satisfying, when its premise is so absurd.

Observation 1) Religious people have evidence:

The thing about religion is that a given religion's effects on people tend to be predictable. When Christians tell you to accept Jesus into your heart, some of the less effective missionaries talk about heaven, but the better ones talk about positive changes to their emotional states. Often, they will imply that those positive life changes will happen for you if you join, and as a prediction that tends to be a very good one.

As a rationalist, I know the emotional benefits of paying attention when something nice happens, and I recognize that feeling gratitude boosts my altruism. I know I can get high on hypoxia if I ever want to see visions or speak in tongues. I know that spending at least an hour every week building ethical responses into my cached behavior is a good practice for keeping positive people in my life. I recognize the historical edifice of morality that allowed us to build the society we currently live in. This whole suite of tools is built into religion, and the means of achieving the benefits it provides is non-obvious enough that a mystical explanation makes sense. Questioning those beliefs without that additional knowledge means you lose access to the benefits of the beliefs.

Observation 2) We expect people to discard falsifiable parts of their beliefs without discarding all of that belief.

The dragon analogy is nice and uncomplicated. There are no benefits to believing in the dragon, so the person in the analogy can make no predictions with it. I've never seen that happen in the real world. Usually religious people have tested their beliefs, and found that the predictions they've made come true. The fact that those beliefs can't predict things in certain areas doesn't change the fact that they do work in others, and most people don't expect generality from their beliefs. When that guy says that the dragon is permeable to flour, that isn't him making an excuse for the lack of a dragon. That's him indicating a section of reality where he doesn't use the dragon to inform his decisions. Religious people don't apply their belief in their dragon in categories where believing has not provided them with positive results. Disproved hypotheses don't disprove the belief, but rather disprove the belief for that category of experience. And that's pretty normal. The fact that I don't know everything, and the fact that I can be right about some things and wrong about others means that I pretty much have to be categorizing my knowledge.

Thinking about this article has lead me to the conclusion that "belief in belief" is more accurately visualized as compartmentalization of belief, that it's common to everyone, and that it indicates that a belief that I have is providing the right answer for the wrong reasons. I predict that if I train myself to react to predicting that the world will behave strangely in order to not violate my hypothesis by saying out loud "this belief is not fully general" I will find that more often than not that this statement will be correct.

"Belief in belief" exists as a phenomenon but is neither necessary nor sufficient to explain the claims of the Dragonist (if I may name his espoused metaphysics thus) in Sagan's parable.

My most recent encounter with someone who believed in belief was someone who did not in addition believe. He had believed once, but he lost his faith (in this case, in God, not dragons) and he wished he could have it back. He believed in belief--that it was a good thing--but alas, he did not believe.

In the above article, Eliezer (if I may so call him) was invoking the concept of belief in belief to explain something--that is, it was a hypothesis of a sort. The phenomenon in question was this Dragonist who claimed to believe but gave some evidence that he did not in that he rejected the most obvious consequences of a dragon being in the garage. Our hypothesis was that he didn't really believe but thought he should and was, in effect, trying to convince himself and others that it was so but (in the case of himself) not so overtly that he'd have to admit to himself he wasn't how he hoped he'd be. If our hypothesis were true, what would we anticipate? If we confronted this guy, that he'd break down and admit he lack of belief? Someone whose belief system runs to invisible dragons is too crazy to let that happen so easily. Maybe what we anticipate is that given sufficient anti-psychotic meds and associated treatments and time, he would recant? What if he didn't? Would we so believe in our hypothesis that we would have faith that given infinite time (say, the amount of time necessary to search all the integers until we identified the last twin prime or the first perfect number that didn't end in 6 or 8) he would recant in principle. Worse still, maybe he would recant to get us off his back but continue to believe in secret.

In short, since our Dragonist's subjective mental state is invisible to us, even were we to sprinkle flour over his head, we are ultimately forced to rely on faith that belief in belief is what is behind this phenomenon.

My main take-away: There is a difference between conscious and subconscious. If you accuse sb with "You do not believe X" then you will get denial because he consciously believes it. The problem is that he subconsicously does not believe it and thus comes up with excuses in advance.

The rationalist virtue of empiricism...

I'm not disagreeing with any of the content above, but a note about terminology--

LessWrong keeps using the word "rationalism" to mean something like "reason" or possibly even "scientific methodology". In philosophy, however, "rationalism" is not allied to "empiricism", but diametrically opposed to it. What we call science was a gradual development, over a few centuries, of methodologies that harnessed the powers both of rationalism and empiricism, which had previously been thought to be incompatible.

But if you talk to a modernist or post-modernist today, when they use the term "rational", they mean old-school Greek, Platonic-Aristotelian rationalism. They, like us, think so much in this old Greek way that they may use the term "reason" when they mean "Aristotelian logic". All post-modernism is based on the assumption that scientific methodology is essentially the combination of Platonic essences, Aristotelian physics, and Aristotelian logic, which is rationalism. They are completely ignorant of what science is and how it works. But this is partly our fault, because they hear us talking about science and using the term "rationality" as if science were rationalism!

(Inb4 somebody says Plato was a rationalist and Aristotle was an empiricist: Really, really not. Aristotle couldn't measure things, and very likely couldn't do arithmetic. In any case the most important Aristotelian writings to post-modernists are the Physics, which aren't empirical in the slightest. No time to go into it here, though.)

Went back to re-read some Lacan and Zizek after this, with regards to Dennett's 'belief in belief.' Very similar to the 'displaced belief' they talk about. The common example they give is Santa Claus: children probably don't believe it but they say they do for the presents, because they understand that the adults expect them to believe, etc. The parents don't believe it but they continue the ruse for the benefit of the children, other people's children, or whatever they tell themselves. Thus people often *do* admit to themselves that they don't believe but they say "but nonetheless other people believe." They displace the belief onto someone else, and they continue going through the motions - and the 'belief' functions anyway. Even if nobody actually believes, they believe by proxy by trusting the apparent belief of those around them. Emperor's New Clothes comes to mind also.

" No, this invisibility business is a symptom of something much worse. "

Indeed. If only it *were* as simple as all that... There often is some fundamental Thing preventing people from realization of the truth and then acting in accordance with that truth though. Often times their entire worldview would be shattered, and they just Can't Have That - it is ideological, in other words. Others know something is charlatanism but they are the charlatan benefiting so they'll keep making up reasons for why there really is a dragon in their garage (maybe they are selling magical dragon breath for $100/jar). Others use false beliefs merely as a way to signal propaganda and attract followers - they know it's fakery but they don't care about debating in good faith to begin with.

Anyway I'm slowly making my way through these after a re-read of your HP fanfic. Just wanted to say that even if "what do I know and how do I know it" is the only thing my brain can hold on to it's already been well worth it (although I did pound Bayes' Theorem in there, too). Thanks!

I agree with gfarb that the claimant does not necessarily believe in a belief; taking the parable literally, it is far more likely that the claimant suffers from a delusional disorder and actually genuinely believes that there is a dragon in his garage. As to why he anticipates that no one else will be able to detect the dragon, that would most likely be explained by his past experiences, where other people denied his experiences and failed to find evidence for them.

The recent conception of the hostile telepaths problem [LW · GW] goes a long way towards explaining why people believe in belief in the first place.