Why Our Kind Can't Cooperate

In this community, agreeing with a poster such as yourself signals me as sycophantic and weak-minded; disagreement signals my independence and courage. There's also a sense that "there are leaders and followers in this world, and obviously just getting behind the program is no task for so great a mind as mine".

However, that's not the only reason I might hesitate to post my agreement; I might prefer only to post when I have something to add, which would more usually be disagreement. Since I don't only vote up things I agree with, perhaps I should start hacking on the feature that allows you to say "6 members marked their broad agreement with this point (click for list of members)".

The nice thing about karma/voting sites like this one is that they provide an efficient and socially acceptable mechanism for signaling agreement: just hit the upmod button. Nobody wants to read or listen to page after page of "me too"; forcing people to tolerate this would be bad enough to negate the advantage of making agreement visible. Voting accomplishes the same visibility without the irritating side-effects.

I must admit, I think I do find myself going into Vulcan mode when posting on LW. I find myself censoring very simple social cues -- expressions of gratitude, agreement, emotion -- because I imagine them being taken for noise. I think I'm going to make an effort to snap myself out of this.

Two thoughts.

1. In any relationship where I have influence, I expect to get more of what I model.

For example, in a community where I have influence, I expect demonstrating explicit support to push community norms towards explicit support, and demonstrating criticism to push norms towards criticism.

This creates the admittedly frustrating situation where, if a community is too critical and insufficiently supportive, it is counterproductive for me to criticize that. That just models criticism, which gets me more criticism; the more compelling and powerful my criticism, the more criticism I'll get in return.

If a community is too critical and insufficiently supportive, I do better to model agreement as visibly and as consistently as I can, and to avoid modeling criticism. For example, to criticize people privately and support them publicly.

1. In any relationship where I have influence, I expect to get more of what I reward.

If a community is too critical and insufficiently supportive, I do well to be actively on the lookout for others' supportive contributions and to reward them (for example: by praising them, by calling other people's attention to them, and/or by paying attention to them myself). I similarly do well to withhold those rewards from critical contributions.

Heh, it seems like this post has primed me for agreement, and I upvoted a lot more comments than I usually do. And it looks like many others did this as well -- look at the upvote counts! I was reading and voting with Kibitzer on, and was surprised to see the numbers.

(Have I just lowered my status by signaling that I'm susceptible to priming?)

This article seems to model rational discourse as a cybernetic system made of two opposite actions that need to be balanced:

• Agreement / support of shared actions
• Disagreement / criticism

Agreement and disagreement are not basic elements of a statement about base reality, they're contextual facts about the relation of your belief to others' beliefs. Is "the sky is blue" agreement or dissent? Depends on what other people are saying. If they're saying it's blue, it's agreement. If they're saying it's green, it's dissent. Someone might disagree with someone by supporting an action, or agree with a criticism of what was previously a shared story. When you have a specific belief about the world, that belief is not made of disagreement or agreement with others, it's made of constrained conditional anticipations about your observations [LW · GW].

This error seems likely related to using a synagogue fundraiser as the central case of a shared commitment of resources, rather than something like an assurance contract! There's a very obvious antirational motive for synagogue fundraisers not to welcome criticism - God is made up, and a community organized around the things its members would genuinely like to do together wouldn’t need to invoke fictitious justifications. Rational coordination should be structurally superior, not just the same old methods but for a better cause.

Insofar as there's something to be rescued from this post, it's that establishing common knowledge of well-known facts is underrated, because it helps with coordination to turn mutual knowledge into common knowledge so everyone can rely on everyone else in the community acting on that info. But that also recommends blurting out, "The emperor's naked!".

There's also the problem that sometimes people say stuff that's off-topic and not helpful enough to be worth it - but compressing the complexity of that problem down to managing the level of agreement vs criticism is substituting an easier but unhelpful task in place of a more difficult but important one.

Many points that are both new and good. Like prase, and like a selection of other fine LW-ers with whom I hope to be agreeing soon, I think your post is awesome :)

One root of the agreement/disagreement asymmetry is perhaps that many of us aspiring rationalists are intellectual show-offs, and we want our points to show everyone how smart we are. Status feels zero-sum, as though one gains smart-points from poking holes in others' claims and loses smart-points from affirming others' good ideas. Maybe we should brainstorm some schemas for expressing agreement while adding intellectual content and showing our own smarts, like "I think your point on slide 14 is awesome. And I bet it can be extended to new context __", or "I love the analogy you made on page 5; now that I read it, I see how to take my own research farther..."

Related: maybe we feel self-conscious about speaking if we don't have anything "new" to add to the conversation, and we don't notice "I, too, agree" as something new. One approach here would be to voice, not just agreement, but the analysis that's going into each individual's agreement, e.g. "I agree; that sounds just like my own experience trying to get an atheists club started", or "I'm adopting these beliefs now, because I trust Eliezer's judgment here, but I have little confirming evidence of my own, so don't double-count my agreement as new evidence". Voicing the causal structure of our agreement would:

• Give us practice seeing how others navigate evidence and Aumann-type issues;
• Expose us to others' evidence;
• Guard against information cascades (assuming honesty in those participating);
• Let us affirm our identities as smart rationalists, while we express agreement. :)

“If I agree, why should I bother saying it? Doesn’t my silence signal agreement enough?”

That’s been my non-verbal reasoning for years now! Not just here: everywhere. People have been telling me, with various degrees of success, that I never even speak except to argue. To those who have been successful in getting through to me, I would respond with, “Maybe it sounds like I’m arguing, but you’re WRONG. I’m not arguing!”

Until I read this post, I wasn’t even aware that I was doing it. Yikes!

BRAVO, Eliezer! Huuzah! It's about time!

I don't know if you have succeeded in becoming a full rationalist, but I know I haven't! I keep being surprised / appalled / amused at my own behavior. Intelligence is way overrated! Rationalism is my goal, but I'm built on evolved wet ware that is often in control. Sometimes my conscious, chooses-to-be-rationalist mind is found to be in the kiddy seat with the toy steering wheel.

I haven't been publicly talking about my contributions to the Singularity Institute and others fighting to save us from ourselves. Part of that originates in my father's attitude that it is improper to brag.

I now publicly announce that I have donated at least $11,000 to the Singularity Institute and its projects over the last year. I spend ~25 hours per week on saving humanity from Homo Sapiens.

I say that to invite others to JOIN IN. Give humanity a BIG term in your utility function. Extinction is Forever. Extinction is for ... us?

Thank you, Eliezer! Once again, you've shown me a blind spot, a bias, an area where I can now be less wrong than I was.

With respect and high regard,
Rick Schwall, Ph.D.
Saving Humanity from Homo Sapiens™ :-|

Two observations:

• In American culture, when you give money to a charity, you aren't supposed to tell people. Christian doctrine frowns heavily on that, and we are all partly indoctrinated with that doctrine. That's why no one sent their "yes" response to the list.

• You just wrote a post with 22 web links, and 19 of them were to your own writings. I think that says more about why we can't cooperate than anything else in the post.

In hindsight, the problem with your fundraiser was obvious. There were two communications channels: one private channel for people who contributed, and one channel for everyone else. Very few people will post a second message after they've already posted one, so the existence of the private channel prevented contributors from posting on the mailing list. Removing all the contributors from the public channel left only nay-sayers and an environment that favored further nay-saying. The fix would be to merge the two channels: publish the messages received from contributors, unless they request otherwise.

I agree with everything you said in your talk, and I think you're brilliant.

I've noticed that I am often hesitant to publicly agree with comments and posts here on LessWrong because often agreement will be seen as spam. While upvotes do count as something, it is far easier to post a disagreement than to invent an excuse to post something that mostly agrees. This can be habit forming.

Comparing say Less Wrong with a Mensa online discussion group I've noticed that my probaility of disagreement is far lower with the self identified rationalists than with the self and test identified generic smart people. The levels of Dark Side Argument are almost incomparable. I have begun disengaging from Dark debates wherever convenient purely to form better habits at agreement.

I'm going to agree with the people saying that agreement often has little to no useful information content (the irony is acknowledged). Note, for instance, that content-free "Me too!" posts have been socially contraindicated on the internet since time immemorial, and content-free disagreement is also generally verboten. This also explains the conference example, I expect. Significantly, if this is actually the root of the issue, we don't want to fight it. Informational content is a good thing. However, we may need to find ways to counteract the negative effects.

Personally, having been somewhat aware of this phenomenon, when I've agreed with what someone said I sometimes try to contribute something positive; a possible elaboration on one of their points, a clarification of an off-hand example if it's something I know well, an attempt to extend their argument to other territory, &c.

In cases like the fundraising one, where the problem is more individual misperception of group trends, we probably want something like an anonymous poll--i.e., "Eliezer needs your help to fund his new organization to encourage artistic expression from rationalists. Would you donate money to this cause?", with a poll and a link to a donation page. I would expect you'd actually get a slightly higher percentage voting "yes" than actually donating, though I don't know if that would be a problem. You'd still get the same 90% negative responses, but people would also see that maybe 60% said they would donate.

To be honest, I suspect a lot of those folks, and I include myself here, were anti-collectivists first.

In my own mind, the emotive rule "I might follow, but I must never obey" is built over a long childhood war and an eventual hard-fought and somewhat Pyrrhic victory. I know it's reversed stupidity, but it's hard to let go.

What good rationalist techniques are there for changing such things?

Wait a second, now we're using Jews trying to run a synagogue as an example of a group who cooperate and don't always disagree with each other for the sake of disagreeing? Your synagogue must have been very different from mine. You never heard the old "Ten Jews, ten opinions - or twenty if they're Reform" joke? Or the desert island joke?

I also agree with everyone. In particular, I agree with Cameron and Prase that it's tough to just say "I agree". I agree with ciphergoth that I worry that I'm sucking up to you too much. I agree with Anna Salamon that we tend to be intellectual show-offs. I agree with Julian that many of us probably started off with a contrarian streak and then became rationalists. I agree with Jacob Lyles that there's a strong game theory element here - I lose big if rationalists don't cooperate, I win a little if we all cooperate under Eliezer's benevolent leadership, but to a certain way of thinking I win even more if we all cooperate under my benevolent leadership and there's no universally convincing proof that cooperating under someone else is always the highest utility option. And I agree with practically everything in the main post.

One thing I don't agree with: being ashamed of strong feelings isn't a specifically rationalist problem. It's a broader problem with upper/middle class society. Possibly more on this later.

I personally see public disagreements as a way to refine the intent of the person under the spotlight rather than a social display of individualism. When I disagree with someone it is not for the sake of disagreeing but rather to refine what I may think is a good idea that has a few weak points. I do this to those I respect and agree with because I hope that others will do this to me.

I think the broader question here is not whether we should encourage widespread agreement in order to create cohesion - but rather if we can ensure that the tenets we collectively agree on are correct conclusions. That is in my mind the main difference between rationalists and what I would call tribalists - in general the majority agree on tenets which are have serious rational flaws or they do simply not raise contest with said tenets. Otherwise if we do follow the leader, then if there are true flaws in that particular modus - we will never discover them.

I agree that it is hard to start a movement based on this - however I see this as a positive attribute. Just as the (flawed) idea of representative democracy was supposed to slow government to a crawl - the rationalist mindset slows group think and confirmation bias to a near halt. It is however a strong movement, however slow.

On 'What Do We Mean By "Rationality"?' when you said "If that seems like a perfectly good definition, you can stop reading here; otherwise continue." - I took your word for it and stopped reading. But apparently comments aren't enabled there.

You have significantly altered my views on morality (Views which I put a GREAT deal of mental and emotional effort into.) I suspect I am not alone in this.

I think there's a fine line between tolerating the appearance of a fanboy culture, and becoming a fanboy culture. The next rationalist pop star might not be up to the challenge.

And for that matter, how many time would you want to risk be subjected to agreement without succumbing? It's not wireheading, but people do get addicted.

"[A] survey of 186 societies found, belief in a moralising God is indeed correlated with measures of group cohesion and size." - God as Cosmic CCTV, Dan Jones

I'm not sure if this was at work in your fundraiser, but I know I tend to see exhortations from others that I give to charitable causes/nonprofits as attempts at guilt tripping. (I react the same way when I'm instructed to vote, or brush my teeth twice a day, or anything else that sounds less like new information and more like a self-righteous command.) For this reason, I try to keep quiet when I'm tempted to encourage others to give to my pet charity/donate blood/whatever, for fear that I'll inspire the opposite reaction and hurt my goal. I don't always succeed, but that's an explanation other than a culture of disagreement for why some people might not have contributed to the discussion from a pro-giving position.

Good points.

This may be why very smart folks often find themselves unable to commit to an actual view on disputed topics, despite being better informed than most of those who do take sides. When attending to informed debates, we hear a chorus of disagreement, but very little overt agreement. And we are wired to conduct a head count of proponents and opponents before deciding whether an idea is credible. Someone who can see the flaws in the popular arguments, and who sees lots of unpopular expert ideas but few ideas that informed people agree on, may give up looking for the right answer.

The problem is that smart people don't give much credit to informed expressions of agreement when parceling out status. The heroic falsfier, or the proposer of the great new idea, get all the glory.

There is no guarantee of a benevolent world, Eliezer. There is no guarantee that what is true is also beneficial. There is no guarantee that what is beneficial for an individual is also beneficial for a group.

You conflate many things here. You conflate what is true with what is right and what is beneficial. You assume that these sets are identical, or at least largely overlapping. However, unless a galactic overlord designed the universe to please homo sapien rationalists, I don't see any compelling rational reason to believe this to be the case.

Irrational belief systems often thrive because they overcome the prisoner dilemmas that individual rational action creates on a group level. Rational people cannot mimic this. The prisoners dilemma and the tragedy of the commons are not new ideas. Telling people to act in the group interest because God said so is effective. It is easy to see how informing people of the costs of action, because truth is noble and people ought not be lied to, can be counter-effective.

Perhaps we should stop striving for the maximum rational society, and start pursuing the maximum rational society which is stable in the long term. That is, maybe we ought to set our goal to minimizing irrationality, recognizing that we will never eliminate it.

If we cannot purposely introduce a small bit of beneficial irrationality into our group, then fine: memetic evolution will weed us out and there is nothing we can do about it. People will march by the millions to the will of saints and emperors while rational causes whither on the vine. Not much will change.

Robin made an excellent post along similar lines, which captures half of what I want to say:

http://lesswrong.com/lw/j/the_costs_of_rationality/

I'll be writing up the rest of my thoughts soon.

Sorry, I can't find the motivation to jump on the non-critical bandwagon today. I had the idea about a week ago that there is no guarantee that truth= justice = prudence, and that is going to be the hobby-horse I ride until I get a good statement of my position out, or read one by someone else.

As the old joke says: What do you mean 'we', white man?

The real reason ostensibly smart people can't seem to cooperate is that most of them have no experience with reaching actual conclusions. We train people to make whatever position they espouse look good, not to choose positions well.

I completely agree with this post. It's heartwarmingly and mindnumbingly agreeable, I would like to praise it and applaud it forever and ever. On a more serious note, personally it feels like not contributing anything into the conversation if you're just agreeing. Like for an example if I read a 100 posts in here, I don't feel compelled to add a comment which says just "I agree." to each of them because it feels like it doesn't add to the substance of the issue. - So I'm totally doing what the post predicts.

I have really read a hundred or so posts and I think the majority of them are brilliant, and to be honest I don't think there have been any posts by Eliezer in particular that I have read which I would've considered really bad. I think they're great. I'm not even stretching it very far when I'm saying that they've changed my look on life.

Personally I truly hope that whoever comes up with the first functional AIs has concern for the future of humanity and takes the time and trouble to ponder moral issues and is responsible about it in general. In fact I believe the world would be a little better place if a larger number of our leaders and political decision makers would demonstrate similar interests - for an example if they could sit down every now and then and contemplate on the meaning of altruism or caring for one another - or they would stop by and read a post on this website.

So this seems like the perfect post to just agree with and add the following suggestion to the conversation: If it feels like you don't want to just agree to something, even if you do really agree, try and find a way to do that while also making a contribution, additional detail or insight. :)

Awesome posts!

On the other hand, if you are only half-a-rationalist, you can easily do worse with more knowledge. I recall a lovely experiment which showed that politically opinionated students with more knowledge of the issues reacted less to incongruent evidence, because they had more ammunition with which to counter-argue only incongruent evidence.

What exactly is the problem with this? The more knowledge I have, the smaller a weighting I place on any new piece of data.

You're awesome, Eli. I love the mix of rationality and emotion here. Emotion is a powerful tool for motivating people. We of the Light Side are rightfully uncomfortable with its power to manipulate, but that doesn't mean we have to abandon it completely.

I recently suggested a rationality "cult" where the group affirmation and belonging exercise is to circle up and have each person in turn say something they disagree with about the tenets of the group. Then everyone cheers and applauds, giving positive feedback. But now I see that this is going too far towards disagreement - better would be for each person to state one area of agreement and one of disagreement with the cult's principles, or today's sermon or exercises, and then be applauded.

I think there's an interesting moral of the anecdote, but I'm not sure it's the one you expressed.

My conclusion is: rationalists who desire to discard the burdensome yoke of their cultural traditions, linked inextricably as they are to religion, will have to relearn an entirely new set of cultural traditions from scratch. For example, they will need to learn a new mechanism design that allows them to cooperate in donating money to cause that is accepted as being worthwhile (I think the "ask for money and then wait for people to call out contributions" scheme is damned brilliant).

Perhaps a way to have comments of agreement that can also work as signalling your own smarts would be to say that you agree, and that the best part/most persuasive part/most useful part is X while providing reasons why. 

Isnt the secret power of Rationality that it can stand up to review? Religious cults are able to demand extreme loyalty because the people are not presented alternatives and are not able to question the view they are handed. One of our strengths seems to be in discernment and argumentation which naturally leads to fractious in-fighting. What would we call "withholding criticism for the Greater Good"?

"But if you tolerate only disagreement - if you tolerate disagreement but not agreement - then you also are not rational. You're only willing to hear some honest thoughts, but not others. You are a dangerous half-a-rationalist."

• Excellent point. I agree completely, and have had similar thoughts about the problem with the "skeptic" community myself. upvote

To point in the rough direction of an empirical cluster in personspace. If you understood the phrase "empirical cluster in personspace" then you know who I'm talking about.

If someone understands the phrase "empirical cluster in personspace," they probably are who you're talking about. =)

This is very interesting; I have usually refrained from replying because I could not think of anything to say that wasn't trivial. Will take care to voice agreement in th future where applicable.

But none of those donors posted their agreement to the mailing list. Not one.

Couldn't you just ask contributors for the right to make their donations public?

Then clearly your fund-raising drive would have benefited from a mechanism for publicizing and externalizing support.

Charitable organizations commonly use a variety of such methods. The example you gave is just one. If correctly designed the mechanisms do not cause support to be swamped by criticism, and they can operate without suppressing any free thought or speech.

E.g. publishing (with their agreement) the names of donors, the amounts, and endorsements; using that information to solicit from other donors; getting endorsements from respected people; appointing wealthy donors to use their own donations as an example when leading solicitation drives among other wealthy donors etc.

The situation does not seem as dire as you suggest.

And you'd better bet that synagogue fund-raising drives get all the gripes that you received, and more!

Way to go Eliezer, you have my full support! And another great posting, btw!

To some extent, this was discussed in "The Starfish and the Spider", which is about "leaderless groups". The book praises the power of decentralized, individualistic cultures (that you describe as "Light Side"). However, it admits that they're slower and less-well coordinated than hierarchical organizations (like the military, or some corporations).

You've outlined some of the benefits (recruitment, coordinated action) of encouraging public agreement and identifying with the group. You've also outlined some of the dangers (pluralistic ignorance, etc.).

Possibly the appropriate answer is to create multiple groups, so that each can be a check against the others turning into cults. Possibly even a fractal of groups and subgroups.

I have been thinking about this subject for a while because I saw the same type of culture of disagreement prevent a group I was a member of from doing anything worthwhile. The problem is very interesting to me because I come from the opposite side of the spectrum being heavily collectivist. I take pleasure in conforming to a group opinion and being a follower but I also have nurtured a growing rationalist position for the last few years. So despite my love of being a follower I often find myself aspiring to a leadership position in order to weld my favored groups into a cohesive whole rather than an un-unified mob. The only solution I have been able to come up with so far is forming a core of beliefs and values which the group can accept without criticism, even if some of the members disagree with some of the parts. This is of course very hard to do.

"Those who had nothing to give, stayed silent; those who had objections, chose some later or earlier time to voice them. That's probably about the way things should be in a sane human community"

Personally I think that you were speaking to the wrong crowd when trying to fund raise. Or perhaps I should say too wide a crowd. Like trying to fundraise for tokamak fusion in a mailing list where people are interested in fusion in the generality. People who don't believe that tokamaks will ever be stable/usable are duty bound to try and convince the other people of that so they won't waste their money (also it means less money in the pot for their projects).

Geek cooperative projects can work, but generally only if there is a mathematical or empirical way to get everyone on the right page, or you have to filter the group you are trying to work with by philosophical position.

With regards to signaling agreement, I think part of the problem is that agreements tend to give little information. If everyone on a certain mailing list said I agree and here is how much money I am donating, I would consider it spam, too much bandwidth for not enough new information... Polls would probably be better, or the organiser of the fund raiser could give running updates (which I believe you did, IIRC).

I have to agree completely.

There's an easy and obvious coordination mechanism for rationalists, which is just to say they're building X from science fiction book Y, and then people will back them to the hilt, as long as their reputation and track record for building things without hurting people is solid. Celebrated Book Y is trusted to explain the upsides and downsides of thing X, and people are trusted to have read the book and have the Right Opinions about all the tradeoffs and choices that come with thing X. 

So really, it all comes down to the thing that actually powers the synagogue's annual appeal - the Torah. The Torah has been doing its job for as long as there have been Jews to read it (or recite it from memory before it was written down), so everyone in the community can agree that the Torah and the Talmud and whatever are reasonable, so coordination becomes trivial. The rabbi standing up in front of the congregation has read all the Torah and Talmud there is, everyone knows and agrees on this fact, so the rabbi is trusted to have the best interests of the community at heart. Since the rabbi has the best interests of the community at heart, the expenses that the community has incurred are obviously real and obviously pressing. Since the rabbi hasn't been going around doing horrible things (he hasn't, right?), everyone knows that the money will actually be spent on the thing the rabbi says it will be spent on, not, I don't know, building nuclear weapons to bomb the competing synagogue across town.

I think rationalists have a deeply good sense of responsibility for their actions and opinions. That's the best thing about them. But I think they also don't have enough respect for the actions and opinions of other people (particularly Other People With Different Opinions Who Are Not As Smart As Me). That's the worst thing about them. As worst things go, it's a pretty minor character flaw; nobody's eating babies alive, they're just kind of smug and condescending in a way that is counterproductive.

I think the thing going on with your pledge drive was that people are afraid to be publicly wrong about something that ends up mattering a great deal, and they don't trust anyone else's opinion about whether they're right or wrong. To break through that force field, we need to start trusting our own wisdom literature, which is science fiction and fantasy, to help us solve the unsolvable challenges that Whatever-Is-Out-There keeps putting in front of us. Sure, mistakes will be made, bad books will be written, people will disagree about what the books mean. All that happens with the Torah and the Talmud too. It just doesn't seem realistic that any one person could act in accordance with most or all the ethical rules laid out in all the science fiction and fantasy books that exist and still end up building something truly evil by coordinating effectively with other people who are also steeped in the science fiction and fantasy traditions. What would that even look like?

I have a modest amount of pair programming/swarming experience, and there are some lessons I have learned from studying those techniques that seem relevant here:

• General cooperation models typically opt for vagueness instead of specificity to broaden the audiences that can make use of them
• Complicated/technical problems such as engineering, programming, and rationality tend to require a higher level of quality and efficiency in cooperation than more common problems
• Complicated/technical problems also exaggerate the overhead costs of trying to harmonize thought and communication patterns amongst the team(s) due to reduced tolerance of failures

With these in mind, I would posit that a factor worth considering is that the traditional models of collaboration simply don't meet the quality and cost requirements in their unmodified form. It is quite easy to picture a rationalist determining that the cost of forging new collaboration models isn't worth the opportunity costs, especially if they aren't actively on the front lines of some issue they consider Worth It.

I agree. I don't often say I agree for efficiency. You've made the point more eloquently than I could and my few sentences in support of you would probably strengthen your point socially, but it wouldn't improve the argument in some logical sense.

I love signaling agreement when I can do it and be just as eloquent as the writing I'm agreeing with. Famous authors put a lot of work into the blurbs they write recommending their friend's books. And that work shows. "X is a great summertime romp, full of adventure!" sure is a glowing recommendation, but it's not that eloquent and I can tell the author didn't put much time into writing it. Guess they didn't think X was worth the time to write a real nice blurb. But when a good author writes an interesting blurb for a book it gives me very high expectations.

I think this applies to ideas as well.

Our culture puts all the emphasis on heroic disagreement and heroic defiance, and none on heroic agreement or heroic group consensus.

There's a lot more of this in anime, I feel. A lot of characters end up trusting someone from the bottom of their hearts, agreeing to follow their vision to the end, and you see whole group of good guys that are wholeheartedly committed and united to the same idea. Even main characters often show this trait toward others.

"Yes, a group which can't tolerate disagreement is not rational.  But if you tolerate only disagreement—if you tolerate disagreement but not agreement—then you also are not rational". Well, agreement may just be perceived default. If I sit at a talk and find nothing to say about (and, mind you, that happens R. A. R. E. L. Y) it means either that I totally agree or that it is so wrong I don't know where to begin.

Also, your attitude on "we are not to win arguments, we are to win", your explicit rejection of rhetorics (up to the seemingly-ignorant question "Why do people think that mentioning the death of some poor fella buying snake oil is argument for regulation?" - because bringing it up like that is a rhetorical argument to that side even if it is not a rational one) may be another weakness more or less common among rationalists. There are ways to sway people to your side, not necessarily including direct lies - and still rationalists tend to refuse using them.

Wow. I don't identify as a cynic or spock, but of the many articles I have read on Less Wrong since I discovered it yesterday, this one is perhaps the most perspective changing.

It makes me happy that those traits you list as what rationalists are usually thought of ----disagreeable, unemotional, cynacal, loners---are unfamiliar. The rationalists I have grown up in the past few years reading this site are both optimistic and caring, along with many other qualities.

Eliezer, I applaud your post. Bravo. I agree.

I'm new to this site and I was compelled to sign up immediately.

There's not much to add here, but that I hope people appreciate the significance of not shutting off all emotions, much like you argue in this post.

Those who suspect me of advocating my unconventional moral position to signal my edgy innovativeness or my nonconformity should consider that I have held the position since 1992, but only since 2007 have I posted about it or discussed it with anyone but a handful of friends.

People are also unwilling to express agreement because they know, and fear, group consensus and the pressure to fit in. Those usually lead to groupspeak and groupthink.

Given that one of the primary messages of the local Powers That Be is that other people's evaluations should be a factor in your own - that other people's conclusions should be considered as evidence when you try to conclude - and that's incompatible with effective rationality, as well as the techniques needed to prevent self-reinforcing mob consensus.

Not only the culture of disagreement takes place. When I see "+1", I think what a mind processes do that: commenter needs some attention but have nothing to say? And so when I want to post "+1", I do not do that, for someone didn't think the same about me. Usually I'm trying to make some complement to original post, or little correction to it with clear approval of the rest. Something not important and, at the same time, not just "+1".

There is a way to solve this problem, but it dangerous. Rationalist can watch discussion closely and not only clever thoughts, but the common effect that discussion have on other watchers, and make some activity every time when discussion have wrong effect. But doing this rationalist makes political discussion from rational.

The only way is to remember the purpose for communication takes place. Not every communications is discussion. And this is the most rational way: rationalist every his move should do knowing the purpose for this move. When we speak about cooperating rationalists, we should also remember that there are common goals and individual goals, and rationalist should weigh both and every time pick the most important at this moment.

And in context of donations: what the reason for rationalist to publish his reasons to not donate? Guilt and attempt to justify himself? Or maybe attempt to draw attention: "now look guys how clever my thoughts"? All the reasons I can imagine is individual goals, that this "rationalist" is considering more important than common goals of community. So either this "rationalist" is enemy of community, or he is just stupid (the same thing, generally).

I wonder if one person can have a big effect on this sort of thing.

For example, I've known charity organizers to publish the number of donors and the total money donated every few days. Even without identifying donors, that does a lot to make people feel less alone.

An alternate explanation: I've noticed a trend where rationalists seem more likely to criticize ideas in general. Perhaps a key experience that needs to happen before some people choose to undergo the rigors of becoming a rationalist is a "waking up" after some trauma that makes them err on the side of being paranoid. I have observed that most people without a "wake up" trauma prefer to simply retain optimism bias and tend to conserve thinking resources for other uses. Someone who thinks as much as you do probably does not feel a need to conserve thinking resources, and probably finds this concept ridiculous, but for most people, stamina for how much thinking they can do in a day is a factor - sad, but true. So, a trauma might be needed to make skepticism appeal to people. It may be that rational thought is often implemented as a defense mechanism and this leads them to create strong habits of doing rational thought in ways that tear ideas down without doing a comparable amount of practice in confirming ideas.

In my opinion, I think the solution to this would be to assist them in reaching a point of satiation when it comes to being great at tearing ideas down. If it's a self-defense mechanism, no amount of brilliant rational appeals will make them give it up. Even if one starts by explaining the risks of tearing ideas down too much, that's only confusing to the self-defense system, people won't know what to do with the cognitive dissonance that causes, so they're likely to reject it. If they feel secure because of a high level of ability with tearing ideas down, they'll probably be more open to seeing the limitations to that and doing more practice with methods of confirming ideas.

organizing atheists has been compared to herding cats, because they tend to think independently and will not conform to authority - The God Delusion

Maybe - but they seem to work together well enough - if you pay them.

Rather than ourselves making the drastic cultural changes that Eli talks about, perhaps it would be more efficient to piggyback on to another movement which is further down that path of culture change, so long as that movement isn't irrational. See this URL:

http://www.thankgodforevolution.com/node/1711

Check out the rest of the web site if you have time, or better yet, buy and read the book the web site is promoting. As you can see from the URL above, cooperation is an important value in the group.

I have been observing the spiritual practices promoted by this web site for just a few weeks, and already it's been giving me tremendous personal benefit. My relationship with my wife and kids is better, I have more enthusiasm for life when I get up in the morning, I no longer find doing chores so onerous, it's much easier for me to refrain from my vices, and I just generally feel more satisfied with the way things are. That's quite a bit for just a few weeks, and I sense the benefits are going to continue to grow with time so long as I adhere to the spiritual practices.

Even though I support Eli's non-profit (that can't be named), I have a very strong urge to give 10-fold as much money to the group that makes such an immediate and real difference in my life.

The really cool thing, though, is that the group is completely compatible with what Eli is trying to do, and should be able to help the cause rather than hinder it, unless we dismiss the group out of hand because their culture is more like a religion than a group of rationalists.

If you think the material on the web site URL I posted above is in any way irrational, please let me know about it. I'd like to hear what you're thinking.

Hrm, overall makes sense. But now, HOW do you suggest, for something here, an online forum, actually doing that sort of thing in the general case without it translating to a whole bunch of people going, effectively, "me too"?

I do remember when for a certain unnamed organization you started the "donate today and tomorrow" drive (or whatever you called it, something to that effect), I did post to a certain mailing list my thoughts that both led me to donate and what I was thinking in response to that sort of appeal, etc etc.

In the pursuit of truth it is rational to argue and, at first glance, irrational to agree. The culling of truth proceeds by "leaving be" the material that is correct and modifying (arguing with) the part that is not. (While slightly tangential, it is good to recall that the scientific method can only argue with a hypothesis; never confirm it.)

At a conference where there is a dialogue it is a waste of time to agree, as a lack of argument is already implicit agreement. After the conference, however, the culling of truth further progresses by assimilating and disseminating the correct material. So while it may not be rational to go to the mike and say, "I agree, you are brilliant", it is a form of true agreement to tell other people that they were brilliant.

Nevertheless, we're human beings, and by that I mean we're not entirely rational in the sense of a deterministic computational machine. We care about our interaction with our community, and in this sense it is rational to give encouragement.

I'm a beginner that thinks meta-discussions are fun..

Eliezer is asking about whether we should tolerate tolerance. Let's suppose -- for the sake of argument -- that we do not tolerate tolerance. If X is intolerable, then the tolerance of X is intolerable.

So if Y tolerates X, then Y is intolerable. And so on.

Thus, if we accept that we cannot tolerate toleration, then also we cannot tolerate toleration of tolerance, and also we cannot tolerate toleration of toleration of tolerance.

I would think of tolerance as a relationship between X and Y in which Y acquires the intolerability of X.

I think that there are parts of life where we should learn to applaud strong emotional language, eloquence, and poetry. When there's something that needs doing, poetic appeals help get it done, and, therefore, are themselves to be applauded.

That may be, but I generally find YOUR poetic appeals to make me throw up in my mouth. I read my mother your bit about how amazing it was that love was born out of the cruelty of natural selection, and even she thought it was sappy.

I don't see how individualism can beat out collectivism as long as groups = more power. for individualism to work each person would have to wield equal power to any group.