Reason as memetic immune disorder

Another reason converts are more zealous than people who grew up with a religion is that conversion is a voluntary act, whereas being born into a religious family is not. Converting to a religion late in life is a radical move, one that generally requires a certain amount of zeal and motivation to begin with, so converts are pre-selected to be zealous.

I've come at this from a similar angle that is, I think, different in the details; and that is rationality as a failure of compartmentalization - the attempt to take everything you hear seriously.

Michael Vassar, again, has a similar angle which is different in the details: nerds result from failing to learn the nonverbal rules of adulthood that are different from the verbal rules.

On the whole a very good post. But here --

The reason I bring this up is that intelligent people sometimes do things more stupid than stupid people are capable of. (For example, quite recently, several respected geneticists declared that there was no such thing as race - an idea that not even the dimmest kid I knew back in Detroit would have fallen for.)

-- you misunderstand the position that you're criticizing. The claim of the geneticists is not that race does not exist, but rather that it doesn't map to the joints at which geneticists, qua geneticists, find it particularly useful to carve reality. But when trying to understand the social world, within which your kid in Detroit is steeped, Race is certainly a useful way to carve reality. And this is all that people mean when they say that Race is a social concept, not a genetic one.

Rather than refusing to try to be consistent in my own beliefs, I find it far more useful to notice what kinds of beliefs most people don't really take seriously enough to be clear about what they mean, to bother to follow through with the most simple sorts of implications, and so on.

IIRC, the position of the Catholic Church is that the death and resurrection of Jesus fulfilled the Covenant and freed humans from the obligation to live according to the Jewish law of the Old Testament. In other words, sometimes the blind spots are explicitly acknowledged and handwaved away instead of being overlooked.

So, there is a hidden component in levels of belief: together with stated level of certainty, bland "truthiness" of a statement, there is also a procedural perspective, with the statement applying with different power in different contexts. This more nuanced level of belief is harder to see and harder to influence: take "belief in belief" as a special case; on one hand there is certainty, on the other it refuses to speak of the real world.

Compartmentalization seems to be the default method for managing "quoted" beliefs: instead of keeping track of what evidence there is for what, just start directly believing everything, but in narrow contexts. If the facts check out, collections of new pieces of knowledge pass coherence checks and gain influence. Insanity remains in the quarantine indefinitely, and even if within its crib it calls the shots, it is a mistake to interpret it as accepted by the person as a whole. When an aspect of most people is insane, it is so by design, part of the never-ending process of reevaluation.

This mechanism is also probably what's responsible for people not even caring to distinguish positive assertions from the negative ones. The natural mode is to just amass impressions of facts, by adherence to each other rather than in their original forms, with levels of certainty simply reporting how well the new statement fits in.

I have a theory that "radical Islam" is not native Islam, but Westernized Islam. Over half of 75 Muslim terrorists studied by Bergen & Pandey 2005 in the New York Times had gone to a Western college. (Only 9% had attended madrassas.) A very small percentage of all Muslims have received a Western college education. When someone lives all their life in a Muslim country, they're not likely to be hit with the urge to travel abroad and blow something up. But when someone from an Islamic nation goes to Europe for college, and comes back with Enlightenment ideas about reason and seeking logical closure over beliefs, and applies them to the Koran, then you have troubles. They have lost their cultural immunity.

Another relevant fact is that, for most of Islam's history, Islamic nations were militarily equal or superior to anyone that they were likely to come into contact with. Islam was a religion founded by conquerers, not by the conquered, and being in a position of profound weakness compared to Western (Christian/Jewish/secular) civilization is something that's simply never happened to them before. Radical Islam could very well be simply the Islam of the fourteenth century faithfully reproduced in the modern era, and the fact that it tends to involve suicide bombings instead of conquering armies is a matter of circumstance rather than ideology. I suspect that, if the Christianity of the fourteenth century, or the Judaism of the first century, were to be faithfully reproduced today, it would be equally horrifying.

Related: Nerds Are Nuts

Phil,

Well written and thought provoking. Reading this, I was reminded of a Douglas Adams essay/speech abut Balinese rice farmers and the way their religion is highly suitable to growing rice. The gods that they cite as reasons for this or that aren't necessarily real and some of the practices may actually be useless, but the end product is a very successful harvest. You might ask a rice farmer why he decided to plant this plant here. His answer could involve some custom that if the moon does this and the chickens do that, I need to put a plant here. That's obviously silly, but it doesn't mean the plant shouldn't be there. The customs and beliefs are the basis for how they do things and how they do things is good for growing rice.

I went back and reread the essay and noticed that I remembered it a little wrong. I also noticed that this isn't some interesting overlap between what you and he are thinking about. What you call the memetic immune system he calls an "artificial god". Actually, I think that your concept is a subset of the artificial god. You seem to assume his position of the artificial god and use it to construct this immune system idea. I think that you would enjoy the piece: http://www.biota.org/people/douglasadams/

"The conservatism of a religion - it's orthodoxy - is the inert coagulum of a once highly reactive sap." -Eric Hoffer, the True Believer

Love your post: religion as virulent namb-shub. See also Snow Crash by Stephenson.

a lot of the rules in the Bible are impossible, illegal, or embarassing to follow nowadays; like wearing tassels, tying your money to yourself, stoning adulterers, not eating fruit from a tree less than 5 years old, and not touching anything that a menstruating woman has touched;

I take it, the author doesn't know many Orthodox Jews..?

The mechanism isn't the same as for diseases[. . .] I suspect that, over time, individual selection favors those who are less zealous. The point is that a culture develops antibodies for the particular religions it co-exists with - attitudes and practices that make them less virulent.

Sometimes, reason [. . .] lets us see the blind spots, but not the dangerous memes. Either of these ways, reason can lead an individual to be unbalanced, no longer adapted to their memetic environment, and free to follow previously-dormant memes through to their logical conclusions.

This also has another side. If individual or cultural selection favors those who don't try to actually do what a religion or cultural norm tells them to do, what happens next depends on which variables are held constant. If the culture is constrained to hold constant the religion or cultural norms, then the resulting selection will cause the culture to develop blind spots, and also develop an unspoken (because unspeakable) but viciously enforced meta-norm of not seeing the blind spots. But if the culture is constrained to hold opposite meta-norms constant, such as a norm of seeing the blind spots or a norm of actually doing what one's religion or cultural norms tell one do do, then the resulting selection will act against the dangerous memes instead. This would make the culture safer for truth-seeking, and make the dividends of truth-seeking easier to pursue.

This sounds like Burke for the 21st Century,

"prejudices and prescriptions and presumptions are the instruments which the wisdom of the species employs to safeguard man against his own passions and appetites."

I suppose this can also explain why new cults, from Born-again Christians to the Scientologists and extreme environmentalists, seem so much more harmful than the boring old Church of England and the like. How fast do we think these counter-beliefs can arise?

Great post, thanks, upvoted.

So most any value-core will go evil if allowed to unfold to its logical conclusions. This sounds correct to me, and also it sounds just like the motivation for FAI. Now your argument that humans solve this problem by balanced deterrence among value-cores (as opposed to weighing them together in one utility function) sounds to me like a novel intuition applicable to FAI. We have some researchers on the topic here, maybe they could speak up?

Not only does this work when someone is trying to follow his religion rationality, it also works when someone is trying to follow rationality rationally.

In other words, not only can becoming more rational lead you to discard the cultural antibodies to religion without discarding the religion, becoming more rational can lead you to discard the cultural antibodies to all sorts of crazy ideas without discarding the crazy ideas. It works for non-religious crazy ideas as well as for religious ones. It even works when the crazy ideas are themselves "rational".

(Of course, if you're perfectly rational you would have no crazy ideas, unlike being perfectly religious, where you'd have lots of them, but human beings aren't perfect. And imperfectly-but-more than-you-were-before rational can lead to crazy ideas.)

I'll leave it as an exercise to the reader to decide which ideas that are popular here and are considered crazy by average people fall into this category--I'm sure everyone has a different list.

Christianity was pacifist at the start, as it arose in a conquered people. When the Romans adopted it, it didn't make them any more militaristic than they already were.

But conversely, Christianity became a lot more militaristic when it became the state religion. Listen e.g. to Dan Carlin's Hardcore History podcast Thor's Angels (free as of 01/2014; 4h long).

I have a theory that "radical Islam" is not native Islam, but Westernized Islam.

-> What about e.g. the fatwa over Salman Rushdie's Satanic Verses, or blasphemy laws, or whatever? This theory doesn't seem consistent with already known facts.

My guess is that his name is "A. J. Jacobs" and not "A. J. Acobs"

I'm also reminded of a talk I attended by one of the Dalai Lama's assistants.  This was not slick, Westernized Buddhism; this was saffron-robed fresh-off-the-plane-from-Tibet Buddhism.  He spoke about his beliefs, and then took questions.  People began asking him about some of the implications of his belief that life, love, feelings, and the universe as a whole are inherently bad and undesirable.  He had great difficulty comprehending the questions - not because of his English, I think; but because the notion of taking a belief expressed in one context, and applying it in another, seemed completely new to him.  To him, knowledge came in units; each unit of knowledge was a story with a conclusion and a specific application.  (No wonder they think understanding Buddhism takes decades.)  He seemed not to have the idea that these units could interact; that you could take an idea from one setting, and explore its implications in completely different settings.  This may have been an extreme form of cultural immunity.

Really?! That's fascinating.

Does anyone have any cite-able examples of the above?

As a Christian who is pretty familiar with the history of Christianity (less so with Islam, and embarrassingly ignorant as to Buddhist thought), I would suggest that perhaps the point on adult converts being radical needs some nuance.

From a Christian perspective, the AJ Jacobs experiment is intended to make any religion look idiotic, due to a very woodenly literal interpretation of what it means to follow the commands of the old and new testaments.

Although there may be some adult converts who do such actions, this seems pretty abnormal, and although adult converts may be marked by more sincerity, claiming they are marked by being more radical in a viral sense seems entirely unsubstantiated.  

Examples:

• Francis Collins, current NIH director & sequencer of the human genome: he converted to Christianity as an adult doctor, and neatly integrated religious metaphysics with an impeccably scientific worldview.  I'm not aware of him attempting to wear tassels or stone people.  
• Augustine of Hippo:  Adult convert from Mannicheaism (a classical-era dualistic religion) to Christianity.  Went from leading a sexual-pleasure-driven life to being one of the best-known philosophers of human history (regardless of how you view his work).  Certainly a drastic change in life direction, but his ideas were anything but extreme, except for perhaps unusually egalitarian for his era.

Feedback aside, your point about cultural blind spots is a good reminder. :) Thank you.

Interesting. So I guess the idea is we have immunities that 'wrap' memes so toxic that our incomplete rationality might be subverted by them and instead ensure that those ideas simply don't interact with the rest of it? So that neither is rational thinking allowed to attack them, but neither are they 'allowed' to extend their influence?

I've found being very careful when you find yourself deviating from everyone around you to be excellent advice, and I do so whenever I can - the first time I deviate in a certain way, after which it gets filed under confirmed ways in which I act differently. That seems to keep the cost of doing so manageable even for the quite abnormal.

I agree with everything you say, but you vascillate between somewhat contradictory positions: that the default is to have disconnected beliefs; or that the default is to have particular "antibodies" preventing action on particular "beliefs." Could you elaborate on this?

I do agree that both are important phenomena. I think the default is disconnected beliefs. I'm not clear on the prevalence and role of "antibodies." Maybe they're just for over-verbal nerds infected with the Enlightenment. But I think they're more general.

The mechanism isn't the same as for diseases, which can't be too virulent or they kill their hosts. Religions don't generally kill their hosts.

Don't drink the Kool-Aid ;)

Prevalent communicable diseases usually don't kill their hosts, because those that do (like Ebola) tend not to spread, and thus are not prevalent.

There have been multiple recorded instances of cults ending in mass suicide, as well as a gamut of other harms to adherents. Those that don't implode from their virulence may spread and survive, eventually becoming old enough to graduate and be called "religions" (i.e. prevalent cults).

Sounds like exactly the same mechanism to me.

This article seems relevant: "Clever sillies: Why high IQ people tend to be deficient in common sense."

The author argues that high IQ people solve problems by using abstract reasoning instead of evolved common sense. Moreover, general intelligence is mainly useful for solving evolutionarily novel problems, and can actually be a hindrance for problems which were a regular part of the evolutionary environment (for example, social situations). Hence, when facing problems where humans have evolved behavioral responses, smart people who apply abstract reasoning and override common sense often end up doing silly things.

I have read something very similar to this someplace else before reading this article (on a side-note. This is the very first article I ever read completely on Less Wrong, and had I not contracted H1N1 at the end of September I would have joined Less Wrong at that time).

I too have read A Year Living Biblically. Mostly so that I would have ready material should I ever have to talk to my Evangelical Aunt and Uncle who are busy preparing a huge number of people in Texas for the Rapture and Second Coming of Christ (Hoo boy?!?).

I seem to recall in the article that I read about Toxic Memes (I am almost positive that is the Title of the article) in second and third world countries that they too mentioned that we have developed memetic anti-bodies in the west to things such as pornography and free-speech, where people living, say in Muslim countries do not have the cultural antibodies that are necessary for defense against the memes in question. This causes a pathological relationship with the meme, resulting in things like blaming a woman for why a man would rape her - to him, it was the pornography's fault for suggesting to him that women are all sex objects.

Obviously, this is wrong, yet those who have been infected by a toxic meme are not going to understand why their understanding of the situation is wrong, any more than we would not understand why an extraterrestrial culture might thought it is wrong for humanity to be playing with their hyperdimensional fecal matter (from another post I made where I discuss that it might be the case that some alien race has feces that is capable of opening up wormholes in space-time). The simple fact of the matter is that the memetic resistance is non-existent.

Duh! It was an article by Jamais Cascio (Toxic Memes. Strangely, it is also a supplement for a game published by a friend of mine in Texas)

Your point about educated fundamentalists has been made elsewhere.

Its possible that some of the geneticists merely think it is good to perform (to steal Hopefully Anonymous' favorite word) such a belief, perhaps for Straussian reasons. Hence the popularity of the replacement term among many scientists "population".

I am many years late to the conversation, but this is fascinating. I think this is a good framework for explaining why taking moral theories to their logical extremes often yield unpalatable conclusions at odds with our moral intuitions - because they are overpowering our illogical cultural antibodies. It's still a good read after 10+ years :) 

I am skeptical of any epistemics that conflate memetic survival with truth, even weakly.  Mostly because acting on certain beliefs can destroy evidence for other beliefs.  Partly because I can think of no reason that all truths should intersect with anthropocentrism.  An example of the former might be the destruction of native american agricultural and hunting techniques by the destruction of the environment.  An example of the latter might be, more contentiously, natalism vs antinatalism.  If antinatalism is true it still loses memetically simply because it self-selects itself away.  So I don't think your heuristics are robust enough, and probably we need people to probe the boundaries of reason from time to time.  It would be good to reduce the social costs of this, but that's different from epistemically consolidating around a small bundle of high equity beliefs and never risking anything on outlandish but potentially important beliefs.  That sort of epistemic conservativism seems both exploitable to me, in the long run, and also a kind of death spiral.

That's the point. In (protestant) Christianity, the old law was a standard that humans could never follow. But the old law had to be paid in blood. So God became human himself, so that he could pay in blood a law that only he could live up to.

It sounds like the author accidentally LARPed as an orthodox Jew.

The trouble is that these antibodies are not logical.  On the contrary; these antibodies are often highly illogical. They are the blind spots that let us live with a dangerous meme without being impelled to action by it.

That is a brilliant point. I also loved you description of the Buddhist monk taking questions from a Western audience. The image of incompatible knowledge blocks is a great one, that actually makes a lot of sense of how various ideologically conditioned people are able to functionally operate. 

The example that comes up for me is animal suffering. I believe that torturing animals in factory farms will one day be regarded as a moral evil on bar with war, slavery, etc. I while I refrain from meat, I have a blindspot for eggs and milk, and still a bigger blind spot for other's eating meat. If I didn't have the latter blindspot, I wouldn't be able to function in society. I don't go around consciously thinking of factory carnivores as moral monsters. Maybe if I were more rationally driven I would think this way, and that might be a very bad thing. 

Maybe the true judo move is to learn how to include the practical rationality of when to compartmentalize in the rational calculus. Of course, we might not have such fine grained control over these unconscious aspects of our cognition. 

I have a theory that "radical Islam" is not native Islam, but Westernized Islam.

Depends on what do you call "radical Islam", but I think that a bit of study of Islam's early history should disabuse you of that notion.

I think your main point - that selective application of rationality could be dangerous - is true. But the question then is how often is it dangerous? And in what way should we apply rationality? Should we not apply rationality because it could be dangerous? I think the article would have been much better if these questions were brought up, and addressed.

I get the sense that applying rationality is usually more good than bad. Although I don't really know enough about radical religions to say if it's true for them too.

You said, "The history of religions sometimes resembles the history of viruses. Judaism and Islam were both highly virulent when they first broke out, driving the first generations of their people to conquer (Islam) or just slaughter (Judaism) everyone around them for the sin of not being them. "

I am not familiar with that history of early Judaism. Can you cite any references I can read about it? (I do admit I have not read the entire old testament, perhaps it's in there?) By the way, I have heard that Roman Catholics are actively discouraged from reading either testament directly.

It is because of the potential in posts like this that I wish that Less Wrong had an edit queue, or that the wiki were used as an edit queue. Do you have plans to write a longer version?

Considering religious belief as virus suggests a larger pattern expressed in the physical world as a whole, does it not? Do our beliefs reflect that pattern or is our perception of the pattern inate?

Regarding religious belief as viral suggests a larger pattern expressed in the physical world as a whole, does it not? Do our beliefs reflect that pattern or is our perception of the pattern inate to our humanity?