People don't have beliefs any more than they have goals: Beliefs As Body Language

I think this is an interesting and useful view, if applied judiciously. In particular, it will always tend to be most relevant for crony beliefs--beliefs that affect the belief-holder's life mainly through other people's opinions of them, like much of politics and some of religion. When it comes to close-up stuff that can cause benefit or harm directly, you will find that most people really do have a model of the world. When you ask someone whether so-and-so would make a good president, the answer is often a signal about their cultural affiliations. Ask them which is the fastest way to get to where they work, and the answer reflects what they've learned about rush-hour traffic patterns. Ask people if they believe in God, and the answer is a signal. Ask them if they believe pre-marital sex is ever acceptable, and the answer you get is a lot more practical.

It's also worth unpacking the us-vs-them terminology you employ here. Many of us may tend to be more literal than the average person (especially those who fall on the spectrum) but in my experience we are still prone to this same behavior. In most cases, there's nothing wrong with that. Understanding the difference can help us avoid trying to cooperatively world-model with people who are just expressing social beliefs, and can also help us recognize world-modeling when we see it, so that we can reduce our tendency to make snap judgements about people on the basis of the beliefs they express.

This is rude and unnecessary. It comes across as "we real people have real beliefs and goals; those fake people do not have real beliefs and goals."

The truth is that beliefs and goals are vague generalizations that apply imperfectly to human beings, including to you. It is true that they apply better in some instances and to some people, but perfectly to none.

Maybe the best strategy for you is to keep feeling a bit superior to "non-nerds", but learn enough manners and forethought to ensure that your conversations with them are pleasant. Reading Jane Austen might help.

[General pushback in the opposite direction and providing an alternate view.]

Counterclaim 1: It's less of a status / posture thing. Most people just aren't thinking most of the time, but totally have the ability to form beliefs if pressed. Thinking of them as lazy evaluators might make more sense. "Smart" people are just those who have more metacognitive activity and are thus asking themselves questions and answering them with less need for external prompts from the environment.

Counterclaim 2: Yes, I think this model might be useful in providing better explanations for why "normal" people do things. But I also think that it can limit the way that "smart" people interact with "normal" people.

Models I myself tend to use try to focus on answering the question, "How can I take actions that improve this person's worldview / life trajectory?" which might involve using the concept that they don't have well-formed beliefs to inform how I move forward, but it certainly doesn't just end with noting that they're being mindless and writing them off as hopeless.

I guess I'm just worried that these sorts of models become an excuse for "smart" people to not even try when it comes to communicating "complex" ideas to "normal" people. I think there's something good that happens on both sides when your focus is on bridging inferential gaps and less on just modeling the other party as some sort of mindless adaptation executor.

I mean, that's part of why we end up with ontologies that refer to objects that only phenomenologically, right? Because it turns out that we get all sorts of cool additional functions when we start looking a little deeper.

I agree with you that other categories than beliefs and goals are often useful for understanding people, especially if you want to understand their experiences, but I worry that you may be advising throwing the baby out with the bathwater.

Likely none of us literally have metaphysically basic beliefs, preferences, or goals. But, that being said, they seem a robust ontology for understanding human behavior and that's why some of the greatest eliminative thinkers in the world--economists--use them to build their models. They even seem to work for modeling the behavior of animals and inorganic, reflective processes that we cannot examine the experiences of.

Beliefs, goals, preferences, etc. have their place in ontology; I think that place is just often not, as you notice, in understanding experiences of those who do not organize their own thoughts in those terms.

I agree with your model, but without the nerd-exception.

The lack of nerd focus on epistemology and meta-ethics implies that nerds don't have beliefs either.

They do have pressures to appear rational. Either external (peer pressure) or internal (intelligence/rationality being part of the core identity because of reasons).

The same model you mention has been useful for me in understanding why nerdy people don't actually care about the epistemic soundness of their argument, and only about sounding rational. It made me understand why many were angered when I pointed the lack of sound definition of the words used or the use of countless fallacies : it's perceived as an attack against their rationality.

This has been said before, but you are very clear and concise here, others before you were not as much.

Especially this "nerds as an exception" explains a lot. Everything becomes quite clear to me with this "exception clause".

It is possible that on some occasions I function as a non-nerd, but mostly not. I do have a world model in my head and I do talk according to it. Which is quite a freakish behavior. Even more so, if your worldview isn't a very boring one.

Quite enlightening post, thank you.

There is a common mistake in modeling humans, to think that they are simple. Assuming that "human chose a goal X" implies "human will take actions that optimally reach X" would be silly. Likewise assuming that humans can accurately observe their own internal state is silly. Humans have a series of flaws and limitations that obscure the simple abstractions of goal and belief. However, saying that goals and beliefs do not exist is a bit much. They are still useful in many cases and for many people.

By the way, it sounds a little like you're referring to so some particular set of beliefs. I think naming them explicitly would add clarity.

"What do you think about X" is re-interpreted and answered as though you had said "What do good, high-status groups (that you can plausibly be a part of) think about X?"

I don't think that's a good model. From my own work with changing beliefs I have the impression that many people have a hard time changing foundational beliefs that they learned before the age of 6 even when all the good high-status people with whom they want to associate don't value that belief.

I agree that "values" are useful way describe human behaviour, but to think that they actually exist inside human brain is sort of mind projection fallacy.

Most people will near halt if we directly ask them about their final goals: they don't know.

Psychologists deduce "human values" from a) human actions b ) claims of the person about his preferences. Actions and claims are often misaligned.

It all surely makes value alignment with AI more difficult: if humans don't have exact values, what should be aligned with?

This is a remarkably good post with which I couldn't have more fully resonated. Thank you!

I think it is important to note that there are probably some ways in which this is adaptive. Us nerds probably spend far too much time thinking and trying to be consistent when it offers us very little benefit. It's also better socially in order to be more flexible - people don't like people who follow the rules too strictly as they are more likely to dob them in. It also much it much easier to appear sincere, but also come up with an excuse for avoiding your prior commitments.

I believe that your distinctions instead of clarifying the concept of belief it has the opposite effect. Belief as a concept can signify:

1. An embodied but not articulated attitude. In this sense a belief is only known when it is acted out.
2. An articulated statement meant to describe the nature of a thing or to propose a certain course of action.

Although as a rationalist you tend to articulate your held beliefs and justify them within the rationalist methodology that does not qualify you, as far as I can see, to proclaim yourself the 'one true believer'. In addition an utterance of articulated belief does not constitute a guaranty that you will act accordingly. Only real action can prove whether you truly believe what you say. For this reason we can even conceive cases that a non-articulated but acted upon belief may be more 'real' than an articulated but not acted upon one.

Apart from this matter of consistency and sincerity we can examine the epistemological status of a belief that is derived by the rationalist methodology in contrast to a belief arrived by means of personal experience, cultural transmission, indoctrination etc. I have to admit that I am sceptical of the statement that you have a comprehensive model of the world based on Bayesian rationality and that you use it in every day life in all occasions. I would suggest an experiment. Try to articulate your full belief system by creating an actual graph. If your whole being is rationally articulated you should be able to achieve it and then observe for consistency of the graph with your behaviour. I, for once, have tried and realised that the majority of my beliefs (in the broad sense of the term) are embodied or/and unconscious with the conscious part (the intellect) constantly observing, analysing, articulating and feeding back.

But how do you know it applies to some people and not others? Post-hoc?

You're talking about stated beliefs, right? Because when it comes to revealed beliefs (those that affect utility maximization), "nerds" don't seem to have the upper hand. This reminds me of the fact that people who don't believe in evolution have more children :-)

I enjoy your writing style and want to see more of it. That said, I'm not sure the "us vs them" way of thinking is healthy in the long run.

Try talking to a "non-nerdy" person about something they know well - ask a doctor about medicine, ask a driver about driving, etc - and you'll discover that they do have beliefs, quite strongly held ones, and will express them even at the cost of social points. Whereas if you ask them about something they don't know as well, like economics, a remarkable thing will happen: they will reply without trying to sound smart! I think that's admirable behavior and we would all do well to imitate it. If you aren't a specialist in some area, thinking too much about it is most likely a mistake, no matter how "nerdy" you are. And even the "nerdiest" of us can only be specialists in a handful of areas.