Cult impressions of Less Wrong/Singularity Institute

AAAAARRRGH! I am sick to death of this damned topic. It has been done to death.

I have become fully convinced that even bringing it up is actively harmful. It reminds me of a discussion on IRC, about how painstakingly and meticulously Eliezer idiot-proofed the sequences, and it didn't work because people still manage to be idiots about it. It's because of the Death Spirals and the Cult Attractor sequence that people bring the stupid "LW is a cult hur hur" meme, which would be great dramatic irony if you were reading a fictional version of the history of Less Wrong, since it's exactly what Eliezer was trying to combat by writing it. Does anyone else see this? Is anyone else bothered by:

Eliezer: Please, learn what turns good ideas into cults, and avoid it!
Barely-aware public: Huh, wah? Cults? Cults! Less Wrong is a cult!

&

Eliezer: Do not worship a hero! Do not trust!
Rationalwiki et al: LW is a personality cult around Eliezer because of so-and-so.

Really, am I the only one seeing the problem with this?

People thinking about this topic just seem to instantaneously fail basic sanity checks. I find it hard to believe that people even know what they're saying when they parrot out "LW looks kinda culty to me" or whatever. It's like people only want to convey pure connotation. Remember sneaking in connotations, and how you're not supposed to do that? How about, instead of saying "LW is a cult", "LW is bad for its members"? This is an actual message, one that speaks negatively of LW but contains more information than negative affective valence. Speaking of which, one of the primary indicators of culthood is being unresponsive or dismissal of criticism. People regularly accuse LW of this, which is outright batshit. XiXiDu regularly posts SIAI criticism, and it always gets upvoted, no matter how wrong. Not to mention all the other posts (more) disagreeing with claims in what are usually called the Sequences, all highly upvoted by Less Wrong members.

The more people at Less Wrong naively wax speculatively on how the community appears from the outside, throwing around vague negative-affective-valence words and phrases like "cult" and "telling people exactly how they should be", the worse this community will be perceived, and the worse this community will be. I reiterate: I am sick to death of people playing color politics on "whether LW is a cult" without doing any of making the discussion precise and explicit rather than vague and implicit, taking into account that dissent is not only tolerated but encouraged here, remembering that their brains instantly mark "cult" as being associated to wherever it's seen, and any of a million other factors. The "million other factors" is, I admit, a poor excuse, but I am out of breath and emotionally exhausted; forgive the laziness.

Everything that should have needed to be said about this has been said in the Cult Attractor sequence, and, from the Less Wrong wiki FAQ:

We have a general community policy of not pretending to be open-minded on long-settled issues for the sake of not offending people. If we spent our time debating the basics, we would never get to the advanced stuff at all. Yes, some of the results that fall out of these basics sound weird if you haven't seen the reasoning behind them, but there's nothing in the laws of physics that prevents reality from sounding weird.

Talking about this all the time makes it worse, and worse every time someone talks about it.

What the bleeding fuck.

The top autocompletes for "Less Wrong" are

• sequences
• harry potter
• meetups

These are my (logged-in) Google results for searching "Less Wrong_X" for each letter of the alphabet (some duplicates appear):

• akrasia
• amanda knox
• atheism
• australia
• blog
• bayes
• basilisk
• bayes theorem
• cryonics
• charity
• cult
• discussion
• definition
• decoherence
• decision theory
• epub
• evolutionary psychology
• eliezer yudkowsky
• evidence
• free will
• fanfiction
• fanfic
• fiction
• gender
• games
• goals
• growing up is hard
• harry potter
• harry potter and the methods of rationality
• how to be happy
• hindsight bias
• irc
• inferential distance
• iq
• illusion of transparency
• joint configurations
• joy in the merely real
• kindle
• amanda knox
• lyrics
• luminosity
• lost purposes
• leave a line of retreat
• meetup
• mobi
• meditation
• methods of rationality
• newcomb's problem
• nyc
• nootropics
• neural categories
• optimal employment
• overcoming bias
• open thread
• outside the laboratory
• procrastination
• pdf
• polyamory
• podcast
• quantum physics
• quotes
• quantum mechanics
• rationality quotes
• rationality quotes
• rationalwiki
• reading list
• rationality
• sequences
• survey
• survey results
• sequences pdf
• twitter
• textbooks
• three worlds collide
• toronto
• ugh fields
• universal fire
• value is fragile
• village idiot
• wiki
• wikipedia
• words
• what is evidence
• yudkowsky
• yvain
• your strength as a rationalist
• your rationality is my business
• zombies
• zombies the movie

The autocomplete bit doesn't seem to be too big a problem for Less Wrong.

However, it is one of the immediate autocompletes for "Singularity Institute." What pages do you get on the first page of results if you search "singularity institute cult"? I see the wikipedia page, the SI website, Michael Anissimov's blog, RationalWiki, Less Wrong posts about cultishness and death spirals, Lukeprog's blog, a Forbes article mention of "cargo-cult enthusiasm," and at the bottom a blog post making a case against SI and other transhumanist organizations.

Google's autocomplete has a problem, which has produced controversy in other contexts: when people want to know whether X is trustworthy, the most informative search they can make is "X scam". Generally speaking, they'll find no results and that will be reassuring. Unfortunately, Google remembers those searches, and presents them later as suggestions - implying that there might be results behind the query. Once the "X scam" link starts showing up in the autocomplete, people who weren't really suspicious of X click on it, so it stays there.

Eliezer addressed this in part with his "Death Spiral" essay, but there are some features to LW/SI that are strongly correlated with cultishness, other than the ones that Eliezer mentioned such as fanaticism and following the leader:

• Having a house where core members live together.
• Asking followers to completely adjust their thinking processes to include new essential concepts, terminologies, and so on to the lowest level of understanding reality.
• Claiming that only if you carry out said mental adjustment can you really understand the most important parts of the organization's philosophy.
• Asking for money for a charity, particularly one which does not quite have the conventional goals of a charity, and claiming that one should really be donating a much larger percentage of one's income than most people donate to charity.
• Presenting an apocalyptic scenario including extreme bad and good possibilities, and claiming to be the best positioned to deal with it.
• [Added] Demand you leave any (other) religion.

Sorry if this seems over-the-top. I support SI. These points have been mentioned, but has anyone suggested how to deal with them? Simply ignoring the problem does not seem to be the solution; nor does loudly denying the charges; nor changing one's approach just for appearances.

Did anyone reading this initially get the impression that Less Wrong was cultish when they first discovered it?

I only discovered LW about a week ago, and I got the "cult" impression strongly at first, but decided to stick around anyway because I am having fun talking to you guys, and am learning a lot. The cult impression faded once I carefully read articles and threads on here and realized that they really are rational, well argued concepts rather than blindly followed dogma. However, it takes time and effort to realize this, and I suspect that the initial appearance of a cult would turn many people off from putting out that time and effort.

For a newcomer expecting discussions about practical ways to overcome bias and think rationally, the focus on things like transhumanism and singularity development seem very weird- those appear to be pseudo-religous ideas with no obvious connection to rationality or daily life.

AI and transhumanism are very interesting, but are distinct concepts from rationality. I suggest moving singularity and AI specific articles to a different site, and removing the singularity institute and FHI links from the navigation bar.

There's also the problem of having a clearly defined leader, with strong controversial opinions which are treated like gospel. I would expect a community which discusses rationality to be more of an open debate/discussion between peers without any philosophical leaders that everybody agrees with. I don't see any easy solution here, because Eliezer Yudkowsky's reputation here is well earned- he actually is exceptionally brilliant and rational.

I would also like to see more articles on how to avoid bias, and apply bayesian methods to immediate present day problems and decision making. How can we avoid bias and correctly interpret data from scientific experiments, and then apply this knowledge to make good choices about things such as improving our own health?

Here's what an outsider might see:

"doomsday beliefs" (something "bad" may happen eschatologically, and we must work to prevent this): check

a gospel (The Sequences): check

vigorous assertions of untestable claims (Everett interpretation): check

a charismatic leader extracting a living from his followers: check

is sometimes called a cult: check

This is enough to make up a lot of minds, regardless of any additional distinctions you may want to make, sadly.

I'm here for only a couple of months, and I didn't have any impression of cultishness. I saw only a circle of friends doing a thing together, and very enthusiastic about it.

What I also did see (and still do) is specific people just sometimes being slightly crazy, in a nice way. As in: Eliezer's threatment of MWI. Or way too serious fear of weird acausal dangers that fall out of currently best decision theories.
Note: this impression is not because of craziness of the ideas, but because of taking them too seriously too early. However, the relevant posts always have sane critical comments, heavily upvoted.

I'm slightly more alarmed by posts like How would you stop Moore's Law?. I mean, seriously thinking of AI dangers is good. Seriously considering nuking Intel's fabs in order to stop the dangers is... not good.

Speaking for myself, I know of at least four people who know of Less Wrong/SI but are not enthusiasts, possibly due to atmosphere issues.

An acquaintance of mine attends Less Wrong meetups and describes most of his friends as being Less Wrongers, but doesn't read Less Wrong and privately holds reservations about the entire singularity thing, saying that we can't hope to say much about the future more than 10 years in advance. He told me that one of his coworkers is also skeptical of the singularity.

A math student/coder I met at an entrepreneurship event told me Less Wrong had good ideas but was "too pretentious".

I was interviewing for an internship once, and the interviewer and I realized we had a mutual acquaintance who was a Less Wronger and SI donor. He asked me if I was part of that entire group, and I said yes. His attitude was a bit derisive.

Defending oneself from the cult accusation just makes it worse. Did you write a long excuse why you are not a cult? Well, that's exactly what a cult would do, isn't it?

To be accused is to be convicted, because the allegation is unfalsifiable.

Trying to explain something is drawing more attention to the topic, from which people will notice only the keywords. The more complex explanation you make, especially if it requires reading some of your articles, the worse it gets.

The best way to win is to avoid the topic.

Unfortunately, someone else can bring this topic and be persistent enough to make it visible. (Did it really happen on a sufficient scale, or are we just creating it by our own imagination?) Then, the best way is to make some short (not necessarily rational, but cached-thought convincing) answer and then avoid the topic. For example: "So, what exactly is that evil thing people on LW did? Downvote someone's forum post? Seriously, guys, you need to get some life."

And now, everybody stop worrying and get some life. ;-)

It could also help to make the site seem a bit less serious. For example put more emphasis on the instrumental rationality on the front page. People discussing best diet habits don't seem like a doomsday cult, right?

The Sequences could be recommended somewhat differently, for example: "In this forum we sometimes discuss some complicated topics. To make the discussion more efficient and avoid endlessly repeating the same arguments about statistics, evolution, quantum mechanics, et cetera, it is recommended to read the Sequences." Not like 'you have to do this', but rather like 'read the FAQ, please'. Also in discussion, instead of "read the Sequences" it is better to recommend one specific sequence, or one article.

Relax, be friendly. But don't hesitate to downvote a stupid post, even if the downvotee threatens to accuse you of whatever.

Some things that might be problematic:

We use the latest insights from cognitive science, social psychology, probability theory, and decision theory to improve our understanding of how the world works and what we can do to achieve our goals.

I don't think we actually do that. Insights, sure, but latest insights? Also, it's mostly cognitive science and social psychology. The insights from probability and decision theory are more in the style of the simple math of everything.

Want to know if your doctor's diagnosis is correct? It helps to understand Bayes' Theorem.

This might sound weird to someone who hasn't already read the classic example about doctors not being able to calculate conditional probabilities. Like we believe Bayes theorem magically grants us medical knowledge or something.

[the link to rationality boot-camp]

I'm not a native speaker of English so I can't really tell, but I recall people complaining that the name 'boot-camp' is super creepy.

On the about page:

Introduce yourself to the community here.

That's not cultish-sounding but it's unnecessarily imperative. Introduction thread is optional.

Disclaimer: My partner and I casually refer to LW meetups (which I attend and she does not) as "the cult".

That said, if someone asked me if LW (or SIAI) "was a cult", I think my ideal response might be something like this:

"No, it's not; at least not in the sense I think you mean. What's bad about cults is not that they're weird. It's that they motivate people to do bad things, like lock kids in chain-lockers, shun their friends and families, or kill themselves). The badness of being a cult is not being weird; it's doing harmful things — and, secondarily, in coming up with excuses for why the cult gets to do those harmful things. Less Wrong is weird, but not harmful, so I don't think it is a cult in the sense you mean — at least not at the moment.

"That said, we do recognize that "every cause wants to be a cult", that human group behavior does sometimes tend toward cultish, and that just because a group says 'Rationality' on the label does not mean it contains good thinking. Hoping that we're special and that the normal rules of human behavior don't apply to us, would be a really bad idea. It seems that staying self-critical, understanding how cults happen and why, and consciously taking steps to avoid making in-group excuses for bad behavior or bad thinking, is a pretty good strategy for avoiding becoming a cult."

Did anyone reading this initially get the impression that Less Wrong was cultish when they first discovered it?

What do you mean, "initially" ? I am still getting that impression ! For example, just count the number of times Eliezer (who appears to only have a single name, like Prince or Jesus) is mentioned in the other comments on this post. And he's usually mentioned in the context of, "As Eliezer says...", as though the mere fact that it is Eliezer who says these things was enough.

The obvious counter-argument to the above is, "I like the things Eliezer says because they make sense, not because I worship him personally", but... well... that's what one would expect a cultist to say, no ?

Less Wrongers also seem to have their own vocabulary ("taboo that term or risk becoming mind-killed, which would be un-Bayesian"). We spend a lot of time worrying about doomsday events that most people would consider science-fictional (at best). We also cultivate a vaguely menacing air of superiority, as we talk about uplifting the ignorant masses by spreading our doctrine of rationality. As far as warning signs go, we've got it covered...

There doesn't seem to be anyone arguing seriously that Less Wrong is a cult, but we do give some newcomers that impression.

The LW FAQ says: >

Why do you all agree on so much? Am I joining a cult?

We have a general community policy of not pretending to be open-minded on long-settled issues for the sake of not offending people. If we spent our time debating the basics, we would never get to the advanced stuff at all. Yes, some of the results that fall out of these basics sound weird if you haven't seen the reasoning behind them, but there's nothing in the laws of physics that prevents reality from sounding weird.

I suspect that putting a more human face on the front page, rather than just linky text, would help.

Perhaps something like a YouTube video version of the FAQ, featuring two (ideally personable) people talking about what Less Wrong is and is not, and how to get started on it. For some people, seeing is believing. It is one thing to tell them there are lots of different posters here and we're not fanatics; but that doesn't have the same impact as watching an actual human with body cues talking.

I don't believe LW is a cult, but I can see where intelligent, critical thinking people might get that impression. I also think that there may be elitist and clannish tendencies within LW that are detrimental in ways that could stand to be (regularly) examined. Vigilance against irrational bias is the whole point here, right? Shouldn't that be embraced on the group level as much as on an individual one?

Part of the problem as I see it is that LW can't decide if it's a philosophy/science or a cultural movement.

For instance, as already mentioned, there's a great deal of jargon, and there's a general attitude of impatience for anyone not thoroughly versed in the established concepts and terminology. Philosophies and sciences also have this problem, but the widely accepted and respected philosophical and scientific theories have proven themselves to the world (and weren't taken very seriously until they did). I personally believe there's a lot of substance to the ideas here, but LW hasn't delivered anything dramatic to the world at large. Until it does so it may remain, in the eyes of outsiders, as some kind of hybrid of Scientology and Objectivism - an insular group of people with a special language, a revered spokesperson, and who claim to have "the answers".

If, however, LW is supposed to be a cultural movement, then I'm sorry, but ”ur doin it wrong". Cultural movements gain momentum by being inclusive and organic, and by creating a forum for people to express themselves without fear of judgment. Movements are bottom up, and LW often gives the impression of being top down.

I'm not saying that a choice has to be made or even can be made, merely that there are conflicting currents here. I don't know if I have any great suggestions. I guess the one thing I can say is that while I've observed (am observing) a lot of debate and self-examination internally, there's still a strong outward impression of having found “the answers”. Perhaps if this community presented itself a little more as a forum for the active practice of critical thinking, and a little less as the authoritative source for an established methodology for critical thinking.

And if that doesn't work, we could always try bus ads.

In general, I think we could stand more community effort being put into optimizing our about page, which you can do now here.

Thank you for this.

(In light of my other comment, I should emphasize that I really mean that. It is not sarcasm or any other kind of irony.)

I have seen this problem afflict other intellectually-driven communities, and believe me, it is a very hard problem to shake. Be grateful we aren't getting media attention. The adage, "All press is good press", has definitely been proven wrong.

The word "cult" never makes discussions like these easier. When people call LW cultish, they are mostly just expressing that they're creeped out by various aspects of the community - some perceived groupthink, say. Rather than trying to decide whether LW satisfies some normative definition of the word "cult," it may be more productive to simply inquire as to why these people are getting creeped out. (As other commenters have already been doing.)

We can fix this stuff easily, since they're both wiki pages, but I thought they were examples worth discussing.

Somebody please do so. Those examples are just obviously bad.

I got a distinct cultish vibe when I joined, but only from the far-out parts of the site, like UFAI, but not from the "modern rationality" discussions. When I raised the issue on #lesswrong, the reaction from most regulars was not very reassuring: somewhat negative and more emotional than rational. The same happened when I commented here. That's why I am looking forward to the separate rationality site, without the added untestable and useless to me EY's idiosyncrasies, such as the singularity, the UFAI and the MWI.

As I see it Cult =Clique + Weird Ideas

I think the weird ideas are an integral part of LessWrong, and any attempt to disguise them with a fluffy introduction would be counterproductive.

What about Cliquishness? I think that the problem here is that any internet forum tends to become a clique. To take part you need to read through lots of posts, so it requires quite a commitment. Then there is always some indication of your status within the group - Karma score in this case.

My advice would be to link to some non-internet things. Why not have the FHI news feed and links to a few relevant books on Amazon in the column on the right?

"Twelve Virtues of Rationality" has always seemed really culty to me. I've never read it, which may be part of the reason. It just comes across as telling people exactly how they should be, and what they should value.

Also, I've never liked that quote about the Sequences. I agree with it, what I've read of the sequences (and it would be wrong to not count HPMOR in this) is by far the most important work I've ever read. But that doesn't mean that's what we should advertise to people.

Ironically, I suspect the "cultlike" problem is that LessWrong/SI's key claims lack falsifiability.

Friendly AI? In the far future.

Self-improvement? All mental self-improvement is suspected of being a cult, unless it trains a skill outsiders are confident they can measure.

If I have a program for teaching people math, outsiders feel they know how they can check my claims - either my graduates are good at math or not.

But if I have a program for "putting you in touch with your inner goddess", how are people going to check my claims? For all outsiders know I'm making people feel good, or feel good about me, without actually making them meaningfully better.

Unfortunately, the external falsifiability of LW/SI's merits is more like the second case than the first. Especially, I suspect, for people who aren't already big fans of mathematics, information theory, probability, and potential AI.

Organization claims to improve a skill anyone can easily check = school. Organization claims to improve a quality that outsiders don't even know how to measure = cult.

If and when LW/SI can headline more easily falsifiable claims, it will be less cultlike.

I don't know if this is an immediately solvable problem, outside of developing other aspects of LW/SI that are more obviously useful/impressive to outsiders, and/or developing a generation of LW/SI fans who are indeed "winners" as rationalists ideally would be.

you'll find that people are searching for "less wrong cult" and "singularity institute cult" with some frequency.

Maybe a substantial number of people are searching for the posts about cultishness.

so I shouldn't refer people to death spirals and baby eating right away?

I think the biggest reason Less Wrong seems like a cult is because there's very little self-skepticism; people seem remarkably confident that their idiosyncratic views must be correct (if the rest of the world disagrees, that's just because they're all dumb). There's very little attempt to provide any "outside" evidence that this confidence is correctly-placed (e.g. by subjecting these idiosyncratic views to serious falsification tests).

Instead, when someone points this out, Eliezer fumes "do you know what pluralistic ignorance is, and Asch's conformity experiment? ... your smart friends and favorite SF writers are not remotely close to the rationality standards of Less Wrong".

What's especially amusing is that EY is able to keep this stuff up by systematically ignoring every bit of his own advice: telling people to take the outside view and then taking the inside one, telling people to look into the dark while he studiously avoids it, emphasizing the importance of AI safety while he embarks on an extremely dangerous way of building AI -- you can do this with pretty much every entry in the sequences.

These are the sorts of things that make me think LessWrong is most interesting as a study in psychoceramics.

It's like Greeks trying to do physics by pure reasoning. They got atoms right because of salt crystallizing,

Obviously, observeing salt is not prure reasoning. Very little philsophy is pure reasoning, the salient distinction is between informal, everyday observation and deliberately arranged experiements.

It's a rather unavoidable side-effect of claiming that you know the optimal way to fulfill one's utility function, especially if that claim sounds highly unusual (Unite the human species in making a friendly AI that will create Utopia). There are many groups that make such claims, and either one or none of them can be right. Most people(Who haven't already bought into a different philosophy of life) think it's the later, and thus tend not to take someone seriously when they make extraordinary claims.

Until recognition of the Singularity's imminence and need for attention enters mainstream scientific thought, the people most likely to join us (Scientifically-Literate Atheists and Truth-Lovers) will not seriously consider our claims. I haven't read nearly as much about the nonexistence of Zues as I have about the nonexistence of Yaweh, because the number of intelligent people who believe in Zues is insignificant compared to the number of educated Christians. So when 99% of the developed world isn't focusing on friendly-AI-theory, it was difficult for me to come to the conclusion that Richard Dawkins and Stephen Hawking and Stephen Fry were all ignorant of one of the most important things on the planet. A few months ago I gave no more thought to cryonics than to cryptozoology, and without MoR I doubt anything would have changed.

There doesn't seem to be anyone arguing seriously that Less Wrong is a cult

Well, there's the folks at RationalWiki.

Long time lurker, I think LW is not capable enough as a social unit to handle it's topic and I currently view that participating in LW is not a good way to efficiently drive it's goals.

In order to reach a (hostile) audience one needs to speak the language. However ambient ways of carrying out discussion are often intermingling status / identity / politics with epistemology. In order to forward a position that biased / faith / economy based thinking are not epistemologically efficient tools one needs to make at least the initial steps in this twisted up "insane troll logic" . The end product is to reject the premise the whole argument stands on but it will never be produced if the thinking doesn't get started. In making it public and praising this kind of transitioning of modes of thinking, a lot of the machinery temporarily required to play the drama out gets reinforced into a kind of bedrock. It complicates matters people are simultaneously in need of completing a particular step while others need to dispel them. Thus there is a tendency to fixate on a "development step" relevant to the majority and becoming hostile to everything else.

I don't see the need to profess stances on things if the relevant anticipations work correctly. Coding the keys of insights on a single namespace and honing them to work against a static context makes applying and discussing them in other contexts needlessly complex. If someone knows a bias / heuristic / concept by some other name and that makes a LW participant not recognize or fail to apply things that they have learnt the password for, LW has managed to isolate insights from their usual and most needed application area.

Things that "hardcore" pursuits find valuable are passed "as is" or "as finalized by AwesomeDude42". This is faith based "cause they say so". Hooked by the "quality of the merchandise" this communal activity is more of a distribution system of those closed packages of tools rather than an epistemic engine in it's own right. I think that even school should be a place of learning rather than a place to receive data about what others have learned.

Because there is a caliber difference not all members can follow or participate in the production of the "good stuff" they wait to be distributed right out of the oven. Doing a passive "level up" handbook in the form of sequences still leaves a big "you must be this tall to participate in this facet of this community". There is no escaping the cognitive work of the individual but LW functions more as a price rather than the workbench.

The activity of LW is limited in a content-independent way by social structure in areas that it wishes to be more. This is not the optimal venue of thinking, but that shouldn't come as a big surprise.

Do you know anyone who might fall into this category, i.e. someone who was exposed to Less Wrong but failed to become an enthusiast, potentially due to atmosphere issues?

Yes. I know a couple of people with whom I share interest in Artificial Intelligence (this is my primary focus in loading Less Wrong web pages) who communicated to me that they did not like the site's atmosphere. Atmosphere is not exactly the word they used. One person thought the cryonics was a deal breaker. (If you read the piece in the New York Times Sunday Magazine about Robin Hanson and his wife you will get a good idea of the global consensus distaste for the topic.) The other person was not so specific although it was clear they were turned off completely even if they couldn't or wouldn't explain how.

Is it possible that our culture might be different if these folks were hanging around and contributing? Presumably they are disproportionately represented among certain personality types.

It is obvious that the culture here would be different if the more controversial or unpopular topics were downplayed enough not to discourage people who don't find the atmosphere convivial.

If so, can you suggest any easy steps we could take?

Here is what I have personally heard or read in comments that people find most bothersome: cryonics, polyamory, pick up artistry, density of jargon, demographic homogeneity (highly educated white males). Any steps to water that down beyond those already taken (pick up artistry is regularly criticized and Bell Curve racial IQ discussion has been all but tabooed) would not be easy to implement quickly and would have consequences beyond making for a more inclusive atmosphere.

I am not in agreement with the suitability of the word cult to characterize this issue accurately. I did the google test you describe and was surprised to see cult pop up so fast, but when I think cult I think Hare Krishnas, I think Charles Manson, I think David Koresh; I don't think Singularity Institute, and I don't think about a number of the organizations on Rick Ross' pages. Rick Ross is a man whose business makes money by promoting fear of cults. The last time I looked he had Landmark Education listed as a cult; this might be true with an extremely loose definition of the word but they haven't killed anybody yet to the best of my knowledge. I have taken a couple of courses from them and the multi-level marketing vibe is irksome but they have some excellent (and rational!) content in their courses. The last time I looked Ross did not have the Ordo Templi Orientis listed as a cult. When I was a member of that organization there were around a couple of thousand dues paying members in the United States, so I presume the OTO cult (this word is far more appropriately applied to them than Landmark) is too small for him to spend resources on.

The poster who replied that he and his wife refer to his Less Wrong activity as his cult membership is understandable to me in a light and humorous manner; I would be surprised if they really classify Less Wrong with Scientology and Charles Manson.

What paper or text should I read to convince me y'all want to get to know reality? That's a sincere question, but I don't know how to say more without being rude (which I know you don't mind).

Put another way: What do you think Harry Potter (of HPMOR) would think of the publications from the Singularity Institute? (I mean, I have my answer).

I got a possible proto-small religion feeling from SL4 discussions with Eliezer and SI folk back in the day. Any possible cultishness feeling was with a small c, that is not harmful to the participants accept for their bank balance, as in the use of the word cult in cult following. There isn't a good word for this type of organization, which is why it gets lumped in with Cults.

Less wrong is better than SL4 for this feeling anyway.

Well, it's nice to know at least you guys see it. Yes, that was one of my reactions. I started reading some of the sequences (which really aren't put at a level that the mass public, or, I'd hazard to say, though not with certainty, even people whose IQs don't fall one standard deviation above the mean or higher can easily understand). I liked them, though I didn't fully understand them, and have referred people to them. However, at the time I was looking into a job and did some kind of search through the website. Anyways, I encountered a post with a person who was asking for advice on a job...I can't find it now, but from what I remember (this has been a long time, the memory is greatly degraded, but I think what little I remember may be actually more insightful in this case than a faithful representation of the actual post) the poster talked about divorce and doing a job they hated and the like to be able to donate more to charity, and how that was an acceptable though not valued trade-off. And, though I actually agree to a point, that did raise HUGE red flags in my mind for cult...particularly when combined with the many messages that seem to be embedded here about donating to the LW non-profits. I fled after reading that and stayed away for a long time. I dunno if it helps or not, but figured I'd share.

Also, I just Googled "Less Wrong" and all I did was added a space and Google auto-suggested cult. So things seem to have worsened since this was published.

Related.

The c-word is too strong for what LW actually is. But "rational" is not a complete descriptor either.

It is neither rational nor irrational to embrace cryonics. It may be rational to conclude that someone who wants to live forever and believes body death is the end of his life will embrace cryonics and life extension technologies.

It is neither rational nor irrational to vaunt current human values over any other. It is most likely that current human values are a snapshot in the evolution of humans, and as such are an approximate optimum in a natural selection sense for an environment that existed 10,000 years ago. The idea that "we" lose if we change our values seems more rooted in who "we" decide "we" are. Presumably in the past a human was more likely to have a narrower definition of "we" to include only a few hundred or a few thousand culture-mates. As time has gone on, "we" has grown to cover nationalities, individual races, pan-national, pan-race, for most people. Most Americans don't identify American with a particular race or national background, and many of us don't even require being born within the US or of US parents to be part of "we." Why wouldn't we extend our concept of "we" to include mammals, or all life that evolved on earth, or even all intelligences that evolved or were created on earth? Why would we necessarily identify a non-earth intelligence as "they" and not "we" as in "we intelligences can stick together and do a better job exploting the inanimate universe."

Rationality is a tool, not an answer. Having certain value decisions vaunted over others restricts LessWrong to being a community that uses rationality rather than a community of rationalists or a community serving all who use rationality. It is what Buffett calls "an unforced error."

Let the downvotes begin! To be clear, I don't WANT to be downvoted, but my history on this site suggests to me that I might be.

Edit: Nevermind, my point was poorly thought out and hastily formulated.

This comment will be heavy with jargon, to convey complex ideas with the minimum required words. That is what jargon is for, after all. The post's long enough even with this shortening.

Less Wrong inspires a feeling of wonder.

To see humans working seriously to advance the robot rebellion is inspiring. To become better, overcome the programming laid in by Azathoth and actually improve our future.

The audacity to challenge death itself, to reach for the stars, is breathtaking. The piercing insight in many of the works here is startling. And the gift of being able to find joy in the merely real again is priceless. It doesn't hurt that it's spearheaded by an extremely intelligent and honest person who's powers of written communication are among the greatest of his generation.

And that sense of awe and wonder makes people flinch. Especially people who have been trained to be wary of that sort of shit. Who've seen the glassed-over eyes of their fundamentalist families and the dazed ramblings of hippies named Storm. As much as HJPEV has tried to train himself to never flinch away from the truth, to never let his brain lie to him, MANY of us have been trained just as strongly to always flinch away from awe and wonder produced by charismatic people. In fact, if we had a "don't let your brain lie to you" instinct as strong as our "don't let awe and wonder seduce you into idiocy" instinct we'd be half way to being good rationalists already.

And honestly, that instinct is a good one. It saves us from insanity 98% of the time. But it'll occasionally result in a woo/cult-warning where one could genuinely and legitimately feel wonder and awe. I don't blame people for trusting their instincts and avoiding the site. And it'll mean we forever get people saying "I dunno what it is, but that Less Wrong site feels kinda cultish to me."

We're open, we're transparent, we are a positive force in the lives of our members. We've got nothing to fear, and that's why occasional accusations of cultishness will never stick. We've just got to learn to live with the vibe and realize that those who stick around long enough to look deeper will bear out that we're not.

It's nice to still have that awe and wonder somewhere. I wouldn't ever want to give that up just so a larger percentage of the skeptic community accepts us. That feeling is integral to this site, giving it up would kill LW for me.