Every Cause Wants To Be A Cult

"Resist against being human" is an interesting choice of words. Surely, most people would not see that as a goal worth pursuing.

Surely, most people would not see that as a goal worth pursuing.

If you're going to concern yourself with popularity contests, you might as well abandon this entire field of endeavor right now. What "most people see" is utterly irrelevant.

I wonder if a Randian will pop up to deny any assertions of cultishness.

I liked your bit about science. I get tired of people saying "Science is a religion too" or some variant thereof, whether from Christians or global warming skeptics like Arnold Kling.

This is a great post. Especially since it applies just as much to the cause of "overcoming bias."

whose rallying cry was "Rationality! Reason! Objective reality!"

Not to disagree with your main point (I've seen cultishness even in mathematics, where we really do have objective reality), but aren't those cults whose banner is Rationality in a better position than those who aren't? They may be just as cultish on the inside, but they have publicly accepted a standard that makes then vulnerable to criticism they cannot just dismiss. Wouldn't that make them a bit more honest?

Same point: are we more honest at overcoming bias, because we have a type of discourse that leaves us vulnerable to arguments of bias in ways we can't ignore - or do we just become more skilled at rationalising?

TGGP: Not to defend global warming denialism, but is that the entirety of your evidence that Kling is a denier? Because when I read that, I strongly got the impression that he was not rallying against people who believe that anthropogenic global warming is occuring, but rather against people preaching dogmatic versions of a nuanced science. I didn't get the impression that he was saying that global warming scientists are preaching a religion, but that global warming activists are, and I think that's completely reasonable. I mean, there's a difference between calling out science and calling out activists: One thing to note is that Al Gore has been on the global warming beat since BEFORE there was a scientific consensus about it (at least as far as he tells it). I don't want to go off on Al Gore too much, but that's certainly the sign of a dogmatist (that is, believing something to be true before the world's experts on the subject had come to a consensus about it). There were basically two dogmas on the issue, and if you picked randomly you'd have a 50% chance of being vindicated.

And I hate to have to reiterate this, but I'm afraid to be lumped in with global warming deniers because I am defending someone who is perceived to be one, but I do NOT find the denialist position compelling. I do however think that Kling makes a good point there (and a similar point to the one in this blog post, I might add) that it is important to convey how you know what you know. It might be reading too far into it, but I would say that that circles back to the point made in this post about cultishness: it's easy to say that something "good" like trying to prevent climate disasters isn't going to have those cultish aspects of attempting to suppress dissent and form in-group mentality, and it it is important (if you are interested in overcoming your biases) to work against this by quantifying how big of a cultish presence you have in your "good" cause.

The I found the main points of the article interesting and fairly convincing but you seem to over-correct for correspondence bias when you say "If the allegations about Wikipedia are true, they're explained by ordinary human nature, not by extraordinary human nature". Even if normal human behaviour leads to cultishness, why assume that individual psychological quirks didn't have a relevant effect in a specific case?

TGGP, Kling wasn't calling science a religion, he was calling the anti-global-warming movement a religion, which strikes me as true regardless of whether the skeptics are right about the science (I think they're not).

So can we learn to recognize the sound of a "cult cooler", cooling down the cultishness, and distinguish it from a fake recording of such? Or at least invent a cultometer, so we can check our cultempature?

I singled out Kling because I could remember several occasions where he used the phrase "the religion of global warming" or something like it and just linked to that post after some quick googling. Perhaps my memory is bad and he hasn't used it that many times though.

In the same sense that every thermal differential wants to equalize itself, and every computer program wants to become a collection of ad-hoc patches, every Cause wants to be a cult. It's a high-entropy state into which the system trends, an attractor in human psychology.

For this to be strictly true, there would have to be more cultish microstates, more possible cultish groups, than sane ones. Do you think that's actually the case?

Aren't those cults whose banner is Rationality in a better position than those who aren't? They may be just as cultish on the inside, but they have publicly accepted a standard that makes then vulnerable to criticism they cannot just dismiss.

They can still dismiss it by redefining "rationality" to exclude the methods the attacker is using.

"Resist against being human" is an interesting choice of words. Surely, most people would not see that as a goal worth pursuing.

Nominull nailed it on the head, Eliezer. What are the human qualities worth amplifying, and what are the human qualities worth suppressing?

For myself, "cultishness" is definitely a human group failure mode.

To others, maybe "cultishness" is a comfortable state of being, like relaxing in a warm bath. (Partake in the vivid imagery of a group of nude people in a drowsy state soaking in their collective body-temperature urea solution...)

I assert that the choice of what elements of humanity are worthy, and what are unworthy, is completely personal and subjective. I would be interested in seeing the argument for the differentiation being objective. Is there an objective criteria for what elements of humanity are worthy, and what are unworthy?

A different point: You really demonstrate the value of blogging and independently developing a stable of ideas, and then being able to reference those ideas with terminology backed up by a hyperlink. I am constantly rereading your posts as you link back to them, and it is interesting and profitable.

We really need to figure out how to create more cultishness. If you could build a cult around known science, which happily describes everything in human experience, and spread it, you'd do more good in the world than "rationality" or "overcoming bias" ever could.

"So can we learn to recognize the sound of a "cult cooler", cooling down the cultishness, and distinguish it from a fake recording of such? Or at least invent a cultometer, so we can check our cultempature?"

We could just ask our perfect Baysian leaders. They know all and understand all.

Eliezer's reminder that even rationalists are human, and so are subject to human failings such as turning a community into a cult, is welcome. But it's a big mistake to dismiss explanations such as "Perhaps only people with innately totalitarian tendencies would try to become the world's authority on everything." There is a huge degree of heterogeneity across people in every relevant metric, including a tendency toward totalitarianism. I can't imagine that anyone disputes this. And if the selection process for being in a certain position tends to advantage people with those tendencies, so that they are selected into them, that might well explain a large part of how people in those positions behave.

Or at least invent a cultometer, so we can check our cultempature?

It's a bad sign if we develop identifiable cliques. Because of general attitudes it stands to reason agreements and disagreements won't be randomly distributed, but ideally we shouldn't "agree" or "disagree" with others because we agreed or disagreed with them in the past. It probably wouldn't be too hard to develop some sort of voting software that measured cliquishness if there's a demand for it.

Of course, the real disaster would be if people start saying things like "Eliezer is always right". Nobody is always right.

"Of course, the real disaster would be if people start saying things like "Eliezer is always right". Nobody is always right.

Posted by: George Weinberg | December 12, 2007 at 03:49 PM "

George,

That understates the risk, since self-identified rationalists familiar with the literature would concoct much better rationalizations. For instance, someone might say that "Nick Bostrom is very intelligent, actively works to overcome biases, and seems to have been relatively successful at it. Since almost all top academics are not immersed in the heuristics and biases literature and committed to Overcoming Bias, in a sustained dispute between top academics and Nick Bostrom we should expect the latter to be right much more often than a random high quality academic dissenter," but then treat this as license to accept Bostrom's positions on an improbable number of independent disagreements.

And even if Bostrom is always right, repeating back what Bostrom says may not mean that you have acquired any of Bostrom's beliefs. Works great for ingroup identification though.

I strongly endorse this post. I've actually watched it happen: Groups dedicated to secularism or the Singularity or even rationality itself can degrade into cliques, evolve into tribes, and then ultimately become as much cults as their greatest foes.

Cade Metz at The Register

It's interesting to stumble across old references to authors whose names you only recognize now, but didn't at the time. Cade Metz, huh? I wonder what he's been up to lately!

I feel confused about the distinction between ingroup behavior and cult-like behavior. It makes sense to me that groups would, as a default, regress towards ingroup behavior without a force pulling them away from it. But it doesn't seem accurate to say that they will naturally move towards cult-like behavior.

Maybe cult-like behavior is similar to ingroup behavior. We only label things as cults when the behavior gets extreme enough/far enough along the spectrum. And so, maybe it is accurate to say that groups naturally move towards cult-like behavior.

But even so, the equilibrium point surely isn't anything close to an "actual" cult. Ingroup behavior can certainly be very powerful, but rarely powerful enough to cause you to given away all your property in anticipation of the saucers landing. 

Relevant post given the situation with Leverage.

Seems relevant in the wake of the FTX scandal. I've seen people blaming effective altruism for the scandal, as if it FTX's fraudulent practices prove that the philosophy of giving to charities that demonstrably do the most good is flawed. Even if the entire EA movement is cult-like and misguided, that doesn't mean that the principle it's based on is wrong. I think the modern EA movement is misguided to some extent, but only because they have misjudged which causes are the most effective, and this shouldn't stop anyone else from donating to causes that they believe are more effective.