Mind Control and Me

Life insurance is usually their bet you won't die early, versus your bet you will. it times out and pays nothing if you're still alive at some agreed age. They don't always have to pay up eventually - or there would be no business model. They give you a policy worth a shade less than their expected payout, in the probabilistic sense, with the difference being the margin.

This is not like investing money in a hedge fund where there are hundreds of competing possibilities. Cryonics Institute or Alcor, pick one.

That only holds if you assume there exists a viable cryonics option. When making a decision like this, an important question to ask is: "Am I being had?" I tend to think both organizations are probably legitimate, but their legitimacy or lack thereof is fundamental to the rationale behind signing up for cryonics in the first place, and not easily dismissed as a "flaw".

But lack of the information you're demanding just isn't something that should stop you from signing up with Alcor or CI.

The above reservations aside, my point wasn't really that it should stop you from signing up (hence "The issue isn't so much the accuracy of those claims, but the nature of them"), my point is that it seems very rational to obtain that information in the course of evaluating your options, and if you can predict that you wouldn't bother to do so, you've possibly identified an irrational operator acting on your decisions.

I should note that if I'm teaching deep things, then I view it as important to make people feel like they're learning deep things, because otherwise, they will still have a hole in their mind for "deep truths" that needs filling, and they will go off and fill their heads with complete nonsense that has been written in a more satisfying style.

Eliezer's writing is clearly not absolutely persuasive, because it didn't persuade me, even when it was correct!

There are some good reasons for being terrified. We are tribal animals. We don't really care about the truth as such, but we care a lot about tribal politics. We can pursue truth when we have very high degree of disinterest in what the truth will be, but that's a really exceptional situation. When we care about the shape of truth, we lose a lot of rationality points, and tribal politics forces make us care a lot about things different than truth. It might be our strongest instinct, even stronger than individual survival or sex drive.

I agree with you that it has its downsides, but I really don't see how you can accept the politics and stay rational. I cannot think of many examples of that.

I'm also really disappointed by so many status indicators all over lesswrong - top contributors on every page. Your social status points (karma) on every page. User names and points on absolutely everything. Vote up / vote down. You might think we're doing fine, but so was reddit when it was tiny, let's see how it scales up. I think we should get rid of as much of that as we can. reddit's quality of discussion is a lot lower than 4chan's, even though it's much smaller.

And this is a great example of what you once posted about - different people are annoyed by different biases. You seem to think social status and politics are mostly harmless and may even be useful, I think it's the worst poison for clear rational thinking, and I haven't seen many convincing examples of it being useful.

Well I don't know that I've got any "rationalist" cred, but as someone who at least attempts to approach life rationally, I am personally terrified by the prospect of being part of a cult because of the way cults seem to warp people's capacity for thinking straightforwardly about reality. (And I could easily lump "religion" in with "cult" here in that regard).

Basically, I don't like the way things I'd call "cultish" seem to disconnect people from concrete reality in favor of abstractions. I've seen some truly awful things happen as a result of that sort of mindset, and have also myself experienced an attempt at "indoctrination" into a sort of cult, and it was one of the worst experiences of my life. A person I knew and thought I could trust, and who seemed smart and reasonable enough, one day managed to trap me in an office under false pretenses and basically sat there berating me and telling me all kinds of horrible things about my character for two hours. And by the end of it, I was halfway ready to believe it, and my confidence and ability to do my schoolwork (this was in college) suffered for months afterward.

So I'm terrified of cults because I know how normal and reasonable their agents can seem at first, and how perfectly horrendous it is to find out what's actually going on, and how difficult it can be afterward to pick up the pieces of your brain and go forward with your life. I don't give a crap about the social-status stuff (well, beyond not wanting to be harassed, if that counts), I just don't want anyone messing with my mind.

I don't get where the sheer power of this fear comes from.

Status/self-image fears are among the most powerful human fears... and the status-behavior link is learned. (In my work, I routinely help people shed these sorts of fears, as they're a prominent source of irrationality, stress, procrastination... you name it.)

Basically, you experience one or more situations (most often just one) where a particular behavior pattern is linked to shaming, ridicule, rejection, or some other basic social negative reinforcer. It doesn't even have to happen to the person directly; it can just be an observation of the response to someone else's behavior. Under stress, the person then makes a snap judgment as to what the causes of the situation were, and learns to do TWO things:

1. To internalize the same response to themselves if they express that behavior, and

2. To have the same response to others having that behavior.

It also works in reverse -- if somebody does something bad to you, you learn to direct anger or attempts at ridicule towards that behavior, and also against yourself, as a result of "judging" the behavior itself to be bad, and the marker of a specific social group or class of people.

This can then manifest in odd ways, like not wanting to exhibit behaviors that would mark you as a member of the group you dislike.

One of the prime issues for me as a rationalist trying to learn about marketing (especially direct/internet marketing) was having to get over the fear of being a "dupe" pulled into a "scam" and "cult" situation. Essentially, if you have learned that some group you scorn (e.g. "suckers" or "fools" or whatever you call them) exhibit joining behavior, then you will compulsively avoid that behavior yourself.

I got over it, of course, but you have to actually be self-aware enough to realize that you chose this attitude/behavior for yourself... although it usually happens at a young enough age and under stressful enough conditions that you weren't thinking very clearly at the time.

But once you've examined the actual evidence used, it's possible to let go of the judgments involved, and then the feelings go away.

I wish I understood why some rationalists find so terrifying the prospect that they might be part of a cult.

For one thing, it would mean that they've been wearing a clown suit for years – and a sort of clown suit that a large part of their identity is defined in opposition to. How humiliating is that?

Ditto fear of being scammed by cryonics, which people seem to regularly treat as the worst thing that could possibly happen. Bad not to conform in belief, worse to be (exposed as) a nonconforming exploitable moron.

Note that hindsight bias can be expected to make being scammed/joining a cult look more moronic than it actually was, and the fundamental attribution error can be expected to make this reflect more badly on the actor than it should.

This still leaves your point that "the possibility of humanity being wiped out seems to have less psychological force than the opportunity to lose five pounds", but near/far probably accounts sufficiently for that.

For Americans (and the cryonics organizations are American) some special factors apply. David Brin has some nice discussion of the ubiquitous pro-individualism propaganda permeating American print and electronic media. Religion is unusually common and powerful in the U.S. so rationalists have more negative affect towards it and anything that resembles it even slightly.

Presumably the minority of people who for whatever reason strongly feel this way (whether rightly or wrongly), are the most likely to self-identify as rationalists.

It doesn't matter that Paul Graham and Stallman don't allow comments. People know them, they have very high reputation, and plenty of fanboys, that all makes them high social status individuals. Mindlessly following the leader is not the same as mindlessly following the group, both are real and distinct behaviours.

People feel differently reading something by Paul Graham and something by blogger they've never heard about. You might have gotten so used to social status indicators you don't consciously see them. Go to 4chan (not /b/), see what discussion is like without them. It is actually surprisingly good.