Some Thoughts Are Too Dangerous For Brains to Think

On reflection, I think I have an obligation to stick my neck out and address some issue of potential dangerous knowledge that really matters, rather than the triviality (to us anyway) of heliocentrism.

Suppose (worst case) that race IQ differences are real, and not explained by the Flynn effect or anything like that. I think it's beyond dispute that that would be a big boost for the racists (at least short-term), but would it be an insuperable obstacle for those of us who think ontological differences don't translate smoothly into differences in ethical worth?

The question of sex makes me fairly optimistic. Men and women are definitely distinct psychologically. And yet, as this fact has become more and more clear, I do not think sexual equality has declined. Probably the opposite - a softening of attitudes on all sides. So maybe people would actually come to grips with race IQ differences, assuming they exist.

More importantly, withholding that knowledge could be much more disastrous.

(1) If the knowledge does come out, the racists get to yell "I told you so," "Conspiracy of silence" etc. Then the IQ difference gets magnified 1000x in the public imagination.

(2) If the knowledge does not come out, then underrepresentation of certain races in e.g., higher learning stands as an ugly fact sans explanation. Society beats its head against a problem of supposed endemic racism for eternity, when the real culprit is statistical differences in mean IQ. Even though - public perceptions be damned - statistical IQ differences should have all the moral weight of "pygmies are underrepresented in basketball."

Knowing about (potential) racial IQ differences is dangerous; so is a perpetual false presumption of racism resulting from ignoring those differences if they exist. Which one generates the most angst, long-term? I don't know. But the truth is probably more sustainable than a well-intentioned fib.

that's a topic worthy of rational discussion (just not now with me).

If this is a plea to be let alone on the topic, then, feel free to ignore my comment below -- I'm posting in case third parties want to respond.

The claim that there has never been a truly benevolent dictator though, that's simply a religious assertion,

Perhaps it's phrased poorly. There have certainly been plenty of dictators who often meant well and who often, on balance, did more good than harm for their country -- but such dictators are rare exceptions, and even these well-meaning, useful dictators may not have been "truly" benevolent in the sense that they presided over hideous atrocities. Obviously a certain amount of illiberal behavior is implicit in what it means to be a dictator -- to argue that FDR was non-benevolent because he served four terms or managed the economy with a heavy hand would indeed involve a "no true Scotsman" fallacy. But a well-intentioned, useful, illiberal ruler may nevertheless be surprisingly bloody, and this is a warning that should be widely and frequently promulgated, because it is true and important and people tend to forget it.

BTW, just from the 20th century there are people from Ataturk to FDR to Lee Kuan Yew to Deng Chou Ping. More generally, more or less The Entire History of the World especially East Asia are counter-examples.

*Ataturk is often accused of playing a leading role in the Armenian genocide, and at the very least seems to have been involved in dismissing courts that were trying war criminals without providing replacement courts, and in conquering territories where Armenians were massacred shortly after the conquest.

*Deng Chou Ping was probably the most powerful person in China at the time of the Tiananmen Square massacres, and it is not clear that he exerted any influence to attempt to disperse the protesters peacefully or even with a minimum of violence: tanks were used in urban areas and secret police hunted down thousands of dissidents even after the protests had ended. One might have hoped that a benevolent illiberal ruler, when confronted with peaceful demands for democracy, would simply say "No." and ignore the protesters except in so far as they were creating a public nuisance.

*FDR presided over the internment of hundreds of thousands of American citizens in concentration camps solely on the basis of race, as well as the firebombing of Dresden, Hamburg, and Tokyo. The first conflagration of a residential area could have been an accident, but there is no evidence of which I am aware that the Allies ever took steps to prevent tens of thousands of civilians from being burnt alive, such as, e.g., taking care to only bomb non-urban industrial targets on hot, dry, summer days. Although Hitler is surely far more responsible than FDR for the Holocaust, a truly benevolent ruler would probably have spared an air raid or two to cut the railroad tracks that led from Jewish ghettos to German death camps. Whatever you might think about FDR's leadership (I would not presume to judge him or to say that I could have done better in his place), it was surprisingly bloody for a benevolent person.

Lee Kuan Yew seems to have been a fairly good dictator, but in his autobiography, he claims to have directly benefited from the US's war efforts in Vietnam, and he says that he would not have remained in power but for the US efforts. For its part, the US State Department explicitly claimed that the Vietnam war was intended to prevent countries like Lee Kuan Yew's Singapore from falling like dominoes after a possible Communization of Vietnam. Although it would probably be unfair to lay moral culpability for, e.g., Mai Lai or Agent Orange on Lee Kuan Yew (and thus I do not say he is in any way to blame), it is still worth noting that Yew's dictatorship was indirectly maintained by years of surprisingly bloody violence. Thus, Yew may be an exception that proves the rule -- even when you yourself, as an aspiring dictator, do not get your hands bloody as power corrupts you, it is possible that you are saved from bloody hands only by a friend who gets his hands bloody for you.

MichaelVassar:

I'm seriously inclined to down-vote the whole comment community on this one except for Peter, though I won't, for their failure to challenge such an overt assertion of such an absurd claim.

I was tempted to challenge it, but I decided that it's not worth to open such an emotionally charged can of worms.

The claim that there has never been a truly benevolent dictator though, that's simply a religious assertion, a key point of faith in the American democratic religion and no more worthy of discussion than whether the Earth is old, at least for usual meanings of the word 'benevolent' and for meanings of 'dictator' which avoid the no true Scotsman fallacy. There have been benevolent democratically elected leaders in the usual sense too. How confident do you think you should be that the latter are more common than the former though? Why?

These are some good remarks and questions, but I'd say you're committing a fallacy when you contrast dictators with democratically elected leaders as if it were some sort of dichotomy, or even a typically occurring contrast. There have been many non-democratic political arrangements in human history other than dictatorships. Moreover, it's not at all clear that dictatorships and democracies should be viewed as disjoint phenomena. Unless we insist on a No-True-Scotsman definition of democracy, many dictatorships, including quite nasty ones, have been fundamentally democratic in the sense of basing their power on majority popular support.

I agree that statements like all As are Bs are likely to be only approximately true and if you look you will find counter examples. But... 'power corrupts' is a fairly reliable rule of thumb as rules of thumb go. I include a couple of refs that took all of 3 minutes to find although I couldn't find the really good one that I noticed a year or so ago.

http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1298606 abstract: We investigate the effect of power differences and associated expectations in social decision-making. Using a modified ultimatum game, we show that allocators lower their offers to recipients when the power difference shifts in favor of the allocator. Remarkably, however, when recipients are completely powerless, offers increase. This effect is mediated by a change in framing of the situation: when the opponent is without power, feelings of social responsibility are evoked. On the recipient side, we show that recipients do not anticipate these higher outcomes resulting from powerlessness. They prefer more power over less, expecting higher outcomes when they are more powerful, especially when less power entails powerlessness. Results are discussed in relation to empathy gaps and social responsibility.

http://scienceblogs.com/cortex/2010/01/power.php from J Lehrer's comments: The scientists argue that power is corrupting because it leads to moral hypocrisy. Although we almost always know what the right thing to do is - cheating at dice is a sin - power makes it easier to justify the wrongdoing, as we rationalize away our moral mistake.

Perhaps it would be more accurate to state "The structural dynamics of dictatorial regimes demands coercion be used, while decentralized power systems allow dissent"; even the Philosopher King must murder upstarts who would take the throne. Mass Driver's comments (below) support this, with Lee Kuan Yew's power requiring violent coercion being performed on his behalf, and the examples of Democratic Despotism largely boil down to a lack of accountability and transparency in the elected leaders - essentially they became (have become) too powerful.

"Power corrupts" is just the colloquial form.

(It is possible that I am in a Death Spiral with this idea, but this analysis occurred to me spontaneously - I didn't go seeking out an explanation that fit my theory)

While I'd disagree with your description of FDR as a dictator, you're quite right about Ataturk, and your other examples expose my woefully insufficient knowledge of non-Western history. My belief has been updated, and the post will be as well, in a moment.

Thanks.

I'm seriously inclined to down-vote the whole comment community on this one except for Peter, though I won't, for their failure to challenge such an overt assertion of such an absurd claim.

I didn't challenge it because I didn't find it absurd. I've asked myself in the past whether I could think of heads of state whose orders & actions were untarnished enough that I could go ahead and call them "benevolent" without caveats, and I drew a blank.

I'd guess my definition of a benevolent leader is less inclusive than yours; judging by your child comment it seems as if you're interpreting "benevolent dictator" as meaning simply "dictators who wanted good results and got them". To me "benevolent" connotes not only good motives & good policies/behaviour but also a lack of very bad policies/behaviour. Other posters in this discussion might've interpreted it like I did.

I know somebody who used to work for Lee Kuan Yew, who has testified that in quite a few ways he at least has been corrupted (things such as creating a slush fund, giving a man who saved his life a public house he didn't qualify for etc).

I'm fine with discriminating in some ways based on intelligence of the individual, and if it does turn out that Group X is statistically less intelligent, then maybe Group X should be underrepresented in important positions. This has consequences for policy decisions.

Agreed. But I should not make decisions about individual members of Group X based on the statistical trend associated with Group X, and I doubt my (or anyone's) ability to actually not do so in cases where I have integrated the belief that the statistical trend is true.

How are you going to help a disadvantaged group if you're blinding yourself to the details of how they're disadvantaged?

The short answer is that I'm not going to. I'm not doing research on human intelligence, and I doubt I ever will. The best I can hope to do is not further disadvantage individual members of Group X by discriminating against them on the basis of statistical trends that they may not embody.

People who are doing research that relates to human intelligence in some way should probably not follow this exact line of reasoning.

I think it may be helpful to clearly distinguish between epistemic and instrumental rationality. The idea proposed in this post is actively detrimental to the pursuit of epistemic rationality; I should have acknowledged that more clearly up front.

But if one is more concerned with instrumental rationality ("winning"), then perhaps there is more value here. If you've designated a particular goal state as a winning one and then, after playing for a while, unconsciously decided to change which goal state counts as a win, then from the perspective of the you that began the game, you've lost.

I do agree that my last example was massively under-justified, especially considering the breadth of the claim.

My hypothesis is that this is a "realist"/"idealist" divide. Or, to put it another way, one camp is more concerned with being right and the other is more concerned with doing the right thing. ("Right" means two totally different things, here.)

Quality of my post aside (and it really wasn't very good), I think that's where the dividing line has been in the comments.

Similarly, I think most people who value PUA here value it because it works, and most people who oppose it do so on ethical or idealistic grounds. Ditto discussions of status.

The reason the arguments between these camps are so unfruitful, then, is that we're sorting of arguing past each other. We're using different heuristics to evaluate desirability, and then we're surprised when we get different results; I'm as guilty of this as anyone.

If there's a discussion about whether or not we should seek truth -- at a site about rationality -- that's a discussion worth having. It's not a side issue.

Like whpearson, I think we're not all on one side or another. I'm pro-truth. I'm anti-PUA. I don't know if I'm pro or anti status -- there's something about this community's focus on it that unsettles me, but I certainly don't disapprove of people choosing to do something high-status like become a millionaire.

You're basically talking about the anti-PC cluster. It's an interesting phenomenon. We've got instinctively and vehemently anti-PC people; we've got people trying to edge in the direction of "Hey, maybe we shouldn't just do whatever we want"; and we've got people like me who are sort of on the dividing line, anti-PC in theory but willing to walk away and withdraw association from people who actually spew a lot of hate.

I think it's an interesting issue because it deals with how we ought best to react to controversy. In the spirit of the comments I made to WrongBot, I don't think we should fear to go there; I know my rationality isn't that fragile and I doubt yours is either. (I've gotten my knee-jerk emotional responses burned out of me by people much ruder than anyone here.)

I think there are divisions within the community, but I am not sure about the correlations. Or at least they don't fit me.

I'm pro discussion of status, I liked red paper clip theory for an example. I'm anti acquiring high status for myself and anti people telling me I should be pro that. I'm anti-pua advice, pro the occasional well backed up psychological research with PUA style flavour (finding out what women really find attractive, why the common advice is wrong etc).

I'm pretty much pro-truth, I don't think words can influence me that much (if they could I would be far more mainstream). I'm less sure about situations, if I was more status/money maximising for a while to earn money to donate to FHI etc, then I would worry that I would get sucked into the high status decadent consumer lifestyle and forget about my long term concerns.

Edit: Actually, I've just thought of a possible reason for the division you note.

If you are dominant or want to become dominant you do not want to be swayed by the words of others. So ideas are less likely to be dangerous to you or your values. If you are less-dominant you may be more susceptible to the ideas that are floating around in society as, evolutionarily, you would want to be part of whatever movement is forming so you are part of the ingroup.

I think my social coprocessor is probably broken in some weird way, so I may be an outlier.

I agree that these politically charged issues are probably not a very good thing for the community, and that we should be extra cautious when engaging them.

Any hypotheses about the common factor?

But most of the time, I believe, if you keep thinking and learning, you'll come round right. (I.e.: when a teenager reads Ayn Rand and thinks that gives him license to be an asshole, his problem is not that he reads too much philosophy.)

"A little learning is a dang'rous thing;
Drink deep, or taste not the Pierian spring:
There shallow draughts intoxicate the brain,
And drinking largely sobers us again."

-- Pope

You will not become a Nazi from reading Mein Kampf, or a Communist from reading Das Kapital.

I became a Trotskyite (once upon a time) partly based on reading Trotsky's history of the Russian Revolution. Yes, I was primed for it, but... words aren't mere.

You will not become a Mormon from reading the Book of Mormon. You will not become a Nazi from reading Mein Kampf, or a Communist from reading Das Kapital. You will not become a racist from reading Steve Sailer.

I don't know if this is a fair characterization of Steve Sailer. I'm quite sure some of his commenters are racist but then again many of the commenters on any major news site are as well. I would call him racialist, or perhaps just a HBDer.

Perhaps I'm somewhat biased in my view of him, but generally for example this interesting video seems typical Steve Sailer style. This is as representative of racism as Das Kapital is of Communism or Mein Kampf of Nazism?

Does racism just have a bad PR guy?

Whatever one calls this it clearly dosen't deserve the few thousand negative karma points racism has in my mind. Perhaps he is putting forward his best face here, but listening to a few parts of this discussion I half expected he would start reciting the litany of Tarski, going into a Hansonian analysis of status or telling everyone that beliefs should pay rent. He certainly touches on these topics in a slightly different vocabulary! Reword a sentence or two and it sounds something a commenter could write on Lesswrong and get upvoted for.

You seem to be particularly worried about accidentally becoming a bigot. (I don't think most of us are in any danger of accidentally becoming supreme dictators.) I think you are safe. Think of it this way: you don't want to be a bigot. You don't want your future self to be a bigot either. So don't behave like one. No matter what you read. Commit your future self to not being an asshole.

He's probably more motivated by not wanting others to become bigots - right, WrongBot?

You will not become a Mormon from reading the Book of Mormon. You will not become a Nazi from reading Mein Kampf, or a Communist from reading Das Kapital. You will not become a racist from reading Steve Sailer.

I will not, but...

This is exactly the crux of the argument. When people say that everyone should be taught that people are the same regardless of gender or race, what they really mean isn't that there aren't differences on average between women and men, etc, but that being taught about those small differences will cause enough people to significantly overshoot via confirmation bias that it will overall lead to more misjudgments of individuals than if people weren't taught about those small differences at all, hence people shouldn't be taught about those small differences. I am hesitantly sympathetic to this view; it is borne out in many of the everyday interactions I observe, including those involving highly intelligent aspiring rationalists.

This doesn't mean we should stop researching gender or race differences, but that we should simultaneously research the effects of people learning about this research: how big are the differences in the perception vs the reality of those differences? Are they big enough that anyone being taught about gender and race differences should also be taught about of the risk of them systematically misjudging many individuals because of their knowledge, and warned to remain vigilant against confirmation bias? When individuals are told to remain vigilant, do they still overshoot to an extent that they become less accurate in judging people than they were before they obtained this knowledge? I would have a much better idea how to proceed both as a society and as an individual seeking to maximize my accuracy in judging people after finding out the answer to these questions.

Those are real and important effects (that should probably have been included in the original post).

A problem with avoiding knowledge that could lead you to discriminate is that it makes it hard to judge some situations - did James Watson, Larry Summers and Stephanie Grace deserve a public shaming?

I suddenly began to believe that matching colored gems was the process by which one constructed sound arguments.

Someone has to write this game.

I spent approximately five minutes trying to tear the wrong section off, mangling the box and cutting my finger twice in the process. This was an astonishing failure to solve a basic physical task.

I have a tendency to do this if I want to solve a basic task and someone is watching me, especially a teacher. (I'm in nursing school, so a lot of my evaluations consist of my teacher watching me assemble equipment, not something I'm talented with to begin with.) Alone, I'll just start experimenting with different ways until I find one that works, but if I'm being watched and implicitly evaluated, paradoxically enough I'll keep trying the same failed way over again until they correct me. I don't know if this is a weird illogical attempt to avoid embarrassment, or if I'm subconsciously trying to hasten the moment that they'll just go ahead and tell me, or if it's just because enough of my brain is taken up worrying about someone watching me that the leftovers aren't capable of thinking about the task, and just default to random physical actions.

I lost count of the number of consecutive cups of flour I was pouring into the bowl; I failed to count to four and a half.

I do this all the time, too. Maybe because my default state, when I'm alone and not under pressure to do something, is a kind of relaxed spacey-ness where I let my thoughts go on whatever association trains they please. People make fun of me for this, and it is irritating, but it's something I'm slowly learning to "switch off" when I really, really have to be focusing my whole attention on something.

I suddenly began to believe that matching colored gems was the process by which one constructed sound arguments. In general. This sensation lasted approximately five seconds before reality reasserted itself.

This kind of thinking happens to me all the time in the state between sleeping and waking, or during dreams themselves. It's occasionally happened to me while awake. I don't find it particularly concerning, since it's easy to notice and wears off fast.

Noting that this thread is nearly two years old: AS is highly correlated with deficiency in executive function. This would explain the bread incident, although not the other two.

I was making bread dough, a process which necessitates measuring out 4.5 cups of flour into a bowl. My mind was not wandering to any unusual degree, nor was I distracted or interrupted. I lost count of the number of consecutive cups of flour I was pouring into the bowl; I failed to count to four and a half.

About half of my caloric intake is bread I bake, and I am terrible at counting. I keep a stack of pennies handy for exactly this reason.

Another version is that bigots can't shut up about it.

Yeah. I have that belief too, but I don't point it out unless that's particularly relevant to the conversation, nor do I try to steer conversations towards that region of topicspace unless I have some compellingly strong reason to do that.

Different groups have different emotional tones . . . (nicer, more honest, more fun, more dignified, etc.).

Downvotes have caused me to put a lot of effort into changing the tone of my communications on Less Wrong so that they are no longer significantly less agreeable (nice) than the group average.

In the early 1990s the newsgroups about computers and other technical subjects were similar to Less Wrong: mostly male, mean IQ above 130, vastly denser in libertarians than the population of any country, the best place online for people already high in rationality to improve their rationality.

Aside from differences in the "shape" of the conversation caused by differences in the "mediating" software used to implement the conversation, the biggest difference between the technical newsgroups of the early 1990s and Less Wrong is that the tone of Less Wrong is much more agreeable.

For example, there was much less evidence IIRC of a desire to spare someone's feelings on the technical newsgroups of the early 1990s, and flames (impassioned harangues of a length almost never seen in comments here and of a level of vitriol very rare here) were very common -- but then again the mediating software probably pulled for deep nesting of replies more than Less Wrong's software does, and most of those flames occured in very deeply nested flamewars with only 2 or 3 participants.

I’ve also observed that people who come to believe that there are significant differences between the sexes/races/whatevers on average begin to discriminate against all individuals of the disadvantaged sex/race/whatever, even when they were only persuaded by scientific results they believed to be accurate and were reluctant to accept that conclusion. I have watched this happen to smart people more than once. Furthermore, I have never met (or read the writings of) any person who believed in fundamental differences between the whatevers and who was not also to some degree a bigot.

This is something I haven't observed, but it's seemed plausible to me anyway. Have there been any studies (even small, lightweight studies with hypothetical trait differences) showing that sort of overshoot? If there are, why don't they get the sort of publicity that studies which show differences get?

I would also be interested in hearing if there are any studies on this subject. For me, much of WrongBot's argument hangs on how accurate these observations are. I'm still not sure I'd agree with the overall point, but more evidence on this point would make me much more inclined to consider it.

Also, WrongBot, it seems possible that the observations you've made could have alternate explanations; e.g., the people that you have witnessed change their behavior based on scientific results may not have been as originally unbiased or reluctant to change their minds on these subjects as you had believed them to be.

In other words, there may be a chicken/egg problem here. Did these people that you observed really become more bigoted/discriminatory after accepting the truth of certain studies, or did (perhaps subconscious) bigotry actually lead them to accept (and even seek out) studies showing results that confirmed this bigotry and gave them "cover" to discriminate?

The shocker was when I realized that the emotional tone is almost certainly the result of what a few high-status members of a group prefer or preferred

Yes, if you have gained temporary influence over others one of the ways you can put that to further use is by trading that influence into an environment that accords with your preferences.

but the emotional tone is generally defended as though it's morally superior

Regardless of how it comes to be established as a social norm, it could be that a particular tone is more suited to a particular purpose, for instance truth-seeking or community-building or fund-raising.

(For instance, academics have a strong norm of writing in an impersonal tone, usually relying on the passive voice to achieve that. This could either be the result of contingent pressure exerted by the people who founded the field, or it could be an antidote to inflamed rhetoric which would detract from the arguments of fact and inference.)

Your point about tone being set top-down (by the high-status, or by inertia in the established community) seems to me to explain why we there are so many genuinely vicious people among netizens who talk rationally and honestly about differences in populations (essentially anti-PC) - even beyond what you'd expect in that they're rebelling against an explicit "be nice" policy that most people assent to.

I'm surprised about the last one. I think it would be quite helpful if you could be prepared for that.

The other two are experiences you wouldn't like to have. If you had the indexical knowledge of what the catchiest jingle was, you could better avoid hearing it.

I have to admit there's information I shield myself from as well.

1. I don't like watching real people die on video. I worry about getting desensitized/dehumanized.

2. I don't want to see 2g1c either. (by extension, most of the grungier parts of the intertubes.)

3. I don't want to know (from experience) what heroin feels like.

I do know people who believe in total desensitization -- they think that the reflex to shudder or gag is something you have to burn out of yourself. I don't think I want that for myself, though.

If you tell me the wild boar
Has twenty teeth, I’ll say, “Why sure.”
Or say that he has thirty three,
That number is quite all right with me
Or scream that he has ninety-nine
I’ll never say that you are lyin’,
For the number of teeth
In a wild boar’s mouth
Is a subject I’m glad
I know nothing about.

-- Shel Silverstein

It's not obvious that knowing more always makes us better off — because the landscape of rationality is not smooth.

The quote in Eliezer's site stating that "That which can be destroyed by the truth should be." sounded to me too strong a claim from the very first time I read it. Many people cultivate falsehoods or use blinkers that are absolutely necessary to the preservation of their sanity (sic), and removing them could terribly jeopardize their adaptability to the environment. It could literally kill them.

I suppose this translates to things you already know, but don't want to consciously attend to. For instance, I feel compelled by the Essendon Football Club's slogan: ]

'Stand as One - One Team. One Dream. Click here or call 1300 GO BOMBERS to become a member and be part of the Bombers team today.Exclusive to essendonfc.com.au'.

While I am tempted to mull of it for a while to dissect it's secrets, I am unlikely, from experience, to get anything meaningful from the experience that I could apply to increase any consequential skill set. Therefore, I'll attend to some other thought associated with my immediate environmental stimuli.

Dark Side epistemology is about protecting false beliefs, if I understand the article correctly. I'm talking about protecting your values.

I am grouping together "everything that goes into your brain," which includes lots and lots of stuff, most of it unconscious. See research on priming), for example.

This argument is explicitly about encouraging people to justify ignoring relevant data about reality. It is, I recognize, an extremely dangerous proposition, of exactly the sort I am warning against!

At risk of making a fully general counterargument, I think it's telling that a number of commenters, yourself included, have all but said that this post is too dangerous.

• You called it "deeply worrisome."
• RichardKennaway called it "defeatist scaremongering."
• Emile thinks it's Dark Side Epistemology. (And see my response.)

These are not just people dismissing this as a bad idea (which would have encouraged me to do the same), these are people are worrying about a dangerous idea. I'm more convinced I'm right than I was when I wrote the post.

Ya. Plus, technically, smaller standard deviation makes for extreme differences in the frequencies at the high IQ range (not that I believe in it or anything).

One argument you could give a Less Wrong audience is that the information about intelligence you could learn by learning someone's gender is almost completely screened off by the information content gained by examining the person directly (e.g. through conversation, or through reading research papers).

experts who are generally labelled as "nurture" supporters like say the respected Richard Lynn

I think you may be confusing Richard Lynn (author of such books as Race Differences in Intelligence: An Evolutionary Analysis) with James Flynn (of Flynn effect fame).

I'm thinking the very hardest bit is going to be getting across to people that it can happen to you. [...] The predatory memes have evolved to eat people who think it can't happen to them.

Certainly there are people who can't be infected with strong cultish memes, and when those people believe that it can't happen to them, they are correct. There are also people who believe so incorrectly, but this is not a strong argument for impossibility of holding that belief correctly. You seem to be overstating the case, implying undue confidence.

I don't understand your second sentence.

In favor of the "power just allows corrupt behavior" theory, Bueno de Mesquita offers two very nice examples of people who ruled two different states. One is Leopold of Belgium, who simultaneously ruled Belgium and the Congo. The other is Chiang Kai-shek, who sequentially ruled China and Taiwan, allegedly rather differently. (I heard him speak about these examples in this podcast. BdM, Morrow, Silverson, and Smith wrote about Leopold here, gated)

I've just identified something else that was nagging at me about this post: the irony of the author of this post making an argument that closely parallels an argument some thoughtful conservatives make against condoning alternative lifestyles like polyamory.

The essence of that argument is that humans are not sufficiently intelligent, rational or self-controlled to deal with the freedom to pursue their own happiness without the structure and limits imposed by evolved cultural and social norms that keep their baser instincts in check. That cultural norms exist for a reason (a kind of cultural selection for societies with norms that give them a competitive advantage) and that it is dangerous to mess with traditional norms when we don't fully understand why they exist.

I don't really subscribe to the conservative argument (though I have more sympathy for it than the argument made in this post) but it takes a similar form to this argument when it suggests that some things are too dangerous for mere humans to meddle with.

I think it would've been better received if some attention was given to defense mechanisms - ie, rather than phrasing it as some true things being unconditionally bad to know, phrase it as some true things being bad to know unless you have the appropriate prerequisites in place. For example, knowing about differences between races is bad unless you are very good at avoiding confirmation bias, and knowing how to detect errors in reasoning is bad unless you are very good at avoiding motivated cognition.

I have a question, then, for people who downvoted this post: what specifically did you dislike about it? This is a data-gathering exercise that will hopefully allow me to identify flaws in my writing and/or thinking and then correct them. Was the argument being made just obviously wrong? Was it insufficiently justified? Did my examples suck? Were there rhetorical tactics that you particularly disliked? Was it structured badly? Are you incredibly annoyed by the formatting errors I can't figure out how to fix?

I upvoted your post, because I think that you raise a possibility that we should consider. It should not be dismissed out of hand.

However, your examples do kind of suck :). As Sarah pointed out, none of us is likely to become a dictator, and dictators are probably not typical people. So the history of dictators is not great information about how we ought to tend to our epistemological garden. Your claims about how data on group differences in intelligence affect people would be strong evidence if it were backed up by more than anecdote and speculation. As it is, though, it is at least as likely that you are suffering from confirmation bias.

Was the argument being made just obviously wrong?

This, primarily. At least obviously wrong by my value system where believing true things is a core value. To the extent that this is also the value system of less wrong as a whole it seems contrary to the core values of the site without acknowledging the conflict explicitly enough.

I didn't think the examples were very good either. I think the argument is wrong even for value systems that place a lower value on truth than mine and the examples aren't enough to persuade me otherwise.

I also found the (presumably) joke about hunting down and killing anyone who disagrees with you jarring and in rather poor taste. I'm generally in favour of tasteless and offensive jokes but this one just didn't work for me.

Situations creating modes of thought that make your corrupted hardware turn you into a bad person.

He referred to something along the lines of "the sensation of being surprised", if I recall it correctly. Would you choose to know everything, if you could, but then never having this sensation again?

I wouldn't claim that any human is actually able to describe their own utility function; they're much too complex and riddled with strange exceptions and pieces of craziness like hyperbolic discounting.

I also think that there's some confusion surrounding the whole idea of utility functions in reality, which I should have been more explicit about. Your utility function is just a description of what you want/value; it is not explicitly about maximizing happiness. For example, I don't want to murder people, even under circumstances where it would make me very happy to do so. For this reason, I would do everything within my power to avoid taking a pill that would change my preferences such that I would then generally want to murder people; this is the murder pill I mentioned.

As for swapping the utilities of spinach and cheesecake, I think the only way that makes sense to do so would be to change how you perceive their respective tastes, which isn't a change to your utility function at all. You still want to eat food that tastes good; changing that would have much broader and less predictable consequences.

This does raise an interesting issue: if I'm a strictly selfish utilitarian, do I not want my utility function to be that which will attain the highest expected utility? Selfishness is not necessary; it just makes the question much simpler.

Only if your current utility function is "maximize expected utility." (It isn't.)

Anorexia could be viewed as an excessive ability to rewrite utility functions about food.

If you don't have the ability to include context, the biological blind god may serve you better than the memetic blind god.

This does raise an interesting issue: if I'm a strictly selfish utilitarian, do I not want my utility function to be that which will attain the highest expected utility?

This is a particular form of wireheading; fortunately, for evolutionary reasons we're not able to do very much of it without advanced technology.

I don't think that makes sense. Changing back is no more desirable than any other change.

Once you've changed, you've changed. Changing your utility function is undesirable. But it isn't bad. You strive to avoid it; but once it's happened, you're glad it did.

It's important to distinguish changes in values, from updating of knowledge about values in response to moral arguments. The latter emphatically shouldn't be opposed, otherwise you turn morally stupid.

You can also try to engage in trade with your future selves, which most good formulations of CEV or its successors should probably enable.

I don't believe I could revert back easily under normal circumstances, so I can't see this advice actually being fruitful unless that fact about me is unusual.

It is not my calculator only, which outputs "12" to the input "5+7". It is every unbroken calculator in the world, which outputs the same.

This would also be true of unbroken brains, if there were any.