The Puzzle of Faith and Belief

its certainly a lot less pleasant to have all of the imperfections picked apart than to have the effort and signal appreciated.

A lot less pleasant and a lot more useful.

This post comes across as a direct opposite of your usual interesting, clear and insightful entries. It feels to me as if it was written by a novice to this site who is also bad at writing. Normally I'd simply downvote and move on, but you are neither new, nor a poor writer, so I am hoping that somewhere inside this post there is a worthwhile point. However, it failed to materialize in the comments I checked, hence my suggestion to rewrite (and maybe run by someone in your local LW crowd, before posting again -- I hear you know a lot of high-profile regulars who would be happy to oblige).

So, the really important thing is to find right things to believe in? As you are saying,

follow the signal of "what/who makes the most sense to put my faith into?"

Any reason why that shouldn't be Jesus?

Again, Brienne's post suggested doing what you imagine a more rational version of yourself would do. I've used this technique to some benefit. It serves as a testable and useful way of demonstrating bias, though I still can't tell if it supports your thesis. She posted before you did. And she put the technique in the title and conclusion of her post, rather than making us guess what the Hell she was talking about!

all four of those words are pointing at roughly the same concept

Very VERY roughly. Too roughly for any use, I think. In particular, these words are not interchangeable -- to say that deciding to get out of bed in the morning is a matter of faith is a misuse of the language.

The different people who use the different words generally think that their way of thinking is superior

That's a non sequitur.

illustrate the "everyone is biased" theory

That theory is generally accepted on LW and has people like Kahneman popularizing it in the wide world :-)

The solution I'm alluding to is to be more open minded about what the possibilities actually are.

That's a platitude and a part of all self-help advice since times immemorial. So what else is new?

men of science tend to over-extrapolate. Ie: that your microwave works means certain things, which are more probable to relate to other certain things. However, you can take these chains of logic out very far to where they become very flimsy

In other words: Physics is highly reliable. You believe in the standard scientific explanation of physics. This creates a feeling of great confidence in "what you believe"... and then you are prone to apply this confidence mistakenly to everything that seems to belong to the literary genre of science. -- Even if the scientific field is not as reliable as physics. Or if you are not an expert in the given field, so regardless of the reliability of the field itself, your understanding of what the field says is unreliable.

I know a few people like this... who have a degree in computer science, are good at maths, have read a few popular science books on physics... which makes them believe they are "experts on science" in general... and then they produce laughable simplifications of psychology, and crackpot theories of evolution. Everything they say follows "logically" from their long and convoluted thought chains. Everything you say, even if it is standard science 101, they dismiss as not sufficiently Popper-approved.

I don't even know the question. The OP's comparison with religious faith serves no clear purpose; the whole post seems more like an advertisement than an immediately useful suggestion. Compare and contrast this post.

Taking what you said one piece at a time:

In the world of science, I can reason by the results. My microwave oven works. What is the chance it would work, if we got physics wrong?

So, I assume the reason you’re asking this is because you assume that belief in physics and mystical beliefs are incompatible. This is a false assumption.

As one of my favorite examples of this being false, I happen to be friends with someone who is a Tibetan grand master of Reiki, who is also a quantum physicist, with a Ph.D from Oxford. She is obviously extremely spiritual, identifies as a believer in the laws of physics, and knows what the laws of physics are in far more detail and with greater understanding than almost anyone who may read this blog.

I believe the base rate of "a random machine doing seemingly miraculous things" is pretty low, otherwise we would be surrounded by magical machines built on theories often incompatible with the official physics. And I mean, magical machines that would work as obviously and reliably as my microwave oven does, or as my mobile phone does... not just something supposedly providing some invisble and hard-to-measure effects.

Okay, so this is more elaboration based on the first assumption made, which I already addressed.

Now my personal life, and my everyday beliefs, that seems like a different kind of game. I see people with different beliefs, having not significantly worse or better results than myself. (A colleague of mine told me recently that he heard that the theory of evolution was disproved. Doesn't have any impact on his programming skills, which is what he gets paid for. But a better example would be some idea outside of science.) I don't have this kind of feedback for the correctness of my ideas. Thus it would be incorrect to put the same degree of faith in them.

Okay, so this is said to contrast the initial statement, again, doesn’t need a response now.

Unfortunately, I have no mind-reading abilities, so I don't know what the obviously successful people believe in. I can listen to what they tell me, but there are problems with this.

Perhaps time to start asking? :)

First, people compartmentalize (and that's the charitable approach; sometimes they also just plainly lie), so what they tell me they believe may not be the same thing they actually believe or alieve. (For example, reading the books by Kiyosaki will not give me the recipe for how to be as rich as Kiyosaki. The true secret of Kiyosaki is more likely something like: Just pretend to know the secret of being rich, and let other people pay you for whatever soundbites you have for them. It's not like someone would ever do a double-blind study to verify your teachings.)

Agreed that the true secret may be different than that given. Agreed that people also sometimes compartmentalize. True of everyone whether a rationalist or not. Gathering data and finding ways to test for truth and compartmentalization seems like a good idea.

Second, there could be a selection bias; even if most of the successful people believe the same thing, there may be even more unsuccessful people believing the very same thing. For example, "follow your passion" or "just buy a lottery ticket" may make a few people incredibly rich, and yet, it may be a poor strategy on average. But we will only hear the stories of the winners. "Yeah, I used to be a chicken like you, but then I decided to follow my gut, and played a few rounds of the Russian roulette, and look where I am now! If you are so smart, why aren't you as rich as me?”

True. Also, a strategy could be good, but not the only ingredient necessary.

Funny religious story:

There’s a big flood, and a priest is stuck on top of a roof as the water is gradually covering up the ground and the buildings. He is praying reverently. Eventually the water is up pretty high on the roofline, and a boat comes by.

They invite the priest to come with them. The priest says "no no no, God will save me!"

This happens a couple more times as the water gets higher and higher, and he eventually drowns.

When he eventually meets God, he asks “Why didn’t you save me? I still had work to do!”

God responds: “I sent you three boats, why didn’t you get on one?!”

One moral of this religious story is that you need to take opportunity when it comes to you, not assume its going to happen through magic.

Another aspect I find important is that it can take many different elements to get the results you want. In the reality of the story described, it requires praying + action. Praying alone was enough to get the opportunity, but only praying didn’t do anything at all for him.

Hence what I was getting at in my previous post about how over-simplification is really not useful and only leads to false confidence.

Actually, the point of my response was to illustrate that to say "all of these things are faith" is an incorrectly simplifying assumption. I did deliberately choose an absurd example of faith, not to attribute it to you, but to show the difference between one thing which you did explicitly claim is faith - trust in people - and another thing which would have to be an example of blind faith - belief in leprechauns. If you acknowledge that there is a real difference between the two, it would seem that I have misinterpreted your thesis.

tl;dr: No, the subject of the site is wider than that.

Long version: IIRC, EY originally conceived of rationality as comprising two relatively distinct domains: epistemic rationality, the art and science of ensuring the map reflects the territory, and instrumental rationality, the art and science of making decisions and taking actions that constrain the future state of the universe according to one's goals. Around the time of the fork of CFAR off of SIAI-that-was, EY had expanded his conception of rationality to include a third domain: human rationality, the art and science of coping with ape-brain.

In my view, these three domains have core subject matter and interfacial subject matter: the core of epistemic rationality is Bayesian epistemology; the core of instrumental rationality is expected utility optimization; the core of human rationality is Thinking, Fast and Slow and construal level theory. At the interface of epistemic and instrumental rationality sit topics like explore/exploit trade-offs and value-of-information calculations; at the interface of epistemic rationality and human rationality sit topics like belief vs. alief, heuristics and biases, and practical techniques for updating on and responding to new information in ways large and small; at the interface of instrumental rationality and human rationality sit topics like goal factoring/funging and habit formation; and right at the intersection of all three, I would locate techniques like implementing tight feedback loops.

You can call it habit, or you can call it faith

This would be a very non-central example of faith.