Outside Analysis and Blind Spots

post by orthonormal · 2009-07-21T01:00:18.458Z · LW · GW · Legacy · 34 comments

(I originally tried to make this a comment, but it kept on growing.)

I was looking through the Google results for "Less Wrong" when I found the blog of a rather intelligent Leon Kass acolyte, who's written a critique of our community.  While it's a bit of a caricature, it's not entirely off the mark.  For example:

Trying to think more like a mathematician, whose empiricism resides in the realm of pure thought, does not predispose these 'rationalists' to collect evidence from the real world. Neither does the downplaying of personal experiences. Many are computer science majors, used to being in the comfortable position of being capable of testing their hypotheses without needing to leave their office. It is, then, an easy temptation for them to come up with a nice-sounding theory which appears to explain the facts, and then consider the question solved. Reason must reign supreme, must it not?

How seriously do you take this critique?  Do you wonder why I'm bothering with this straw-man criticism of Less Wrong?

Actually, I've deceived you; there's no such Leon Kass devotee.  The quote above is a very minor adaptation of this Kaj Sotala post, which I've changed from the first person plural to the third person plural.  Read it if you like, and then reevaluate the critique.  (Yes, I know it's less coherent out of its actual context.)  Does it seem to be less of a caricature now that you've read a version in which you identify with the writer, rather than one in which the writer is analyzing and criticizing you from outside?

Now I hope that this little trick (which people are starting to expect around here) caught your attention.  We really do seem to react differently to an outside analysis of a person or group in very different ways, depending on whether we've been primed to identify ourselves more with the author or with the object of analysis.  One might say that we strongly object to being modeled as a more or less predictable agent in someone else's scheme: that we instinctively reject any out-group analysis of our own personality and cognition.  (Compare people's reluctance to trust the Outside View even when they know it's more reliable.)

In fact, in some ways this is reminiscent of anosognosia: take for instance the recent study on body language, which discovered that

  1. We can predict a stranger's extraversion scores on the Implicit Association Test quite well by watching them act out a one-minute commercial.
  2. We can improve our predictions further if we're first told about a few nonverbal tics to watch for.
  3. We're bad at predicting our own extraversion scores on the IAT, even if we're given the video of ourselves acting out the commercial and told about the nonverbal tics.

This screams out "blind spot" to me, and one with a nice evolutionary reason to boot: a blind spot toward one's own patterns of action would make it possible to sincerely promise something we'll probably fail to do, which is a pretty nice trick to have up one's sleeve in a social environment.  (Anyhow, the cute ev-psych story can be discarded without incident to the rest of the evidence.)

If it's true that we instinctively react to an out-group analysis of our actions, what might we expect would happen when we're faced with one?  The most likely reaction would be a knee-jerk dismissal of the model, with justification to be constructed after the fact; or perhaps we'd take offense.  If that were so, then we might expect the following kinds of results:

And if a fourth obvious case doesn't occur to you, you must have been somewhere else for the past week.

Fortunately, it seems to me that a fix is available: if the analysis is set up so that the (intended) readers identify more with the analyst than with the objects of analysis, they seem to avoid that blind spot.  (They don't have to share all the characteristics of the analyst to do that; I would bet that female readers didn't have a moment of difficulty identifying with Eliezer rather than the woman on the panel in this anecdote, where the implied divide was "rationalists versus irrationalists" rather than "men versus women".)  Keeping your readers with you is usually not that hard to do in practice; it's what writers call "knowing your audience", and if you imagine delivering your statement to the proper audience, you should (subconsciously, even) do better at avoiding that split between yourself and them.

The key thing, though, is that readers don't seem to do this on their own, not even rationalists.  This is not a failing of one part of this community or another; this seems to be part of the current human condition, and it behooves a good communicator to avoid implicit "Us/Them" splits that leave a good part of the intended readership in "Them".  In particular, writing with more care on gender is worth the cost in extra words and thought: gender-specific pronouns really do seem to cause distraction and dis-identification with the author, and I'd predict that the difference between

most people here don't value social status enough and (especially the men) don't value having sex with extremely attractive women that money and status would get them

and, say,

money and status would make a man more attractive to many women; men who really value a better romantic or sexual life should thus put more priority on money and status

is pretty significant to a female reader (please correct me if I'm wrong).  In the first, it's generally just the male readers who can easily take it as an analysis from their perspective, while female readers identify themselves with the thing being (very crudely) modeled from outside.  In the second, female and male readers can identify themselves with someone making an actual choice or observation, or equally well envision themselves looking at the whole situation from outside.

Therefore, I suggest that when your post or comment touches on a subject that divides the Less Wrong community into identifiable groups (transhumanism or PCT or libertarianism, not just gender), it's good practice to imagine reading your contribution out loud to members of the various subgroups, and edit if you feel it might go over badly.  This goes double if you're analyzing a general tendency in a group you don't belong to.  (ETA: Sometimes it might be necessary to go ahead and damn the torpedoes, but I think that on some subjects we're being far too lax in this respect.)  On the other hand, if someone analyzes you or your group from outside (and, needless to say, gets it wrong), I'd suggest you show a little extra patience with them; neither of you need be exceptionally irrational/sensitive/insensitive for this kind of impasse to arise.

P.S. I've hung back from the Less Wrong Gender Wars for a while, in part because I wanted to observe it a bit before committing myself to a position, and in part because everything I had to say seemed wrong somehow.  I finally started writing out a comment listing several hypotheses for how we could have a situation where one good rationalist feels that a way of speaking is clearly unethical (while not necessarily incorrect in substance), and another good rationalist appears to be, not just disagreeing, but actively mystified about what could be wrong with it.  Then I realized that one of my hypotheses was much better supported than the others.

EDIT: At the request of several, I've stopped diluting the term "Outside View" and called this particular thing "out-group analysis."


Comments sorted by top scores.

comment by Liron · 2009-07-21T03:02:07.024Z · LW(p) · GW(p)

I approve of the algorithm that generated your post.

comment by Z_M_Davis · 2009-07-21T07:50:13.674Z · LW(p) · GW(p)

This is not a failing of one part of this community or another; this seems to be part of the current human condition

This is a failing of all parts of this community, and seems to be a part of the current human condition. (The eighth virtue is humility; the ninth virtue is perfectionism.)

Replies from: Will_Newsome
comment by Will_Newsome · 2011-05-15T19:06:06.646Z · LW(p) · GW(p)

(The eighth virtue is humility; the ninth virtue is perfectionism.)

Ohhhh, good catch. Nice touch, Eliezer.

comment by Cyan · 2009-07-21T02:44:04.809Z · LW(p) · GW(p)

Possibly pedantic nitpick...

I thought "Inside View" and "Outside View" referred to predictions derived from a thorough, assumption-laden* mechanism-based model for something, and predictions derived from embedding the thing in a class of roughly similar things, respectively. You seem to be using the phrases to distinguish between in-group and out-group criticism.

* This isn't pejorative; the assumptions may be well-supported.

Replies from: orthonormal, taw, SoullessAutomaton, Eliezer_Yudkowsky
comment by orthonormal · 2009-07-21T15:26:47.498Z · LW(p) · GW(p)

I think that this distinction shares some interesting features with the Inside/Outside View distinction (or just Near/Far thinking), but it looks like you're right, I shouldn't confuse notation. I think I like taw's suggestion; give me a moment to edit.

comment by taw · 2009-07-21T15:09:45.108Z · LW(p) · GW(p)

I second that changing terminology to "In-group view" and "Out-group view" or something like that would be much appreciated.

"Inside view" vs "Outside view" distinction is one of the most important concept of rationality and we should try to stay clear about it.

comment by SoullessAutomaton · 2009-07-21T11:08:18.780Z · LW(p) · GW(p)

This made me stumble as well. Unnecessary overloading of terminology confuses readers and reduced my enjoyment of an otherwise excellent post.

comment by Eliezer Yudkowsky (Eliezer_Yudkowsky) · 2009-07-21T05:16:52.318Z · LW(p) · GW(p)

That leapt out at me as well. "Outside View" has a technical meaning and this doesn't seem to be it.

comment by teageegeepea · 2009-07-21T02:01:38.624Z · LW(p) · GW(p)

That was a very nice trick you pulled. I was even grimacing at the phrase "rather intelligent Leon Kass acolyte". Too bad it will be hard to do it again as we'll be on alert!

I second Allan, you definitely changed the meaning from OUGHT_TO(x) to GIVEN(x).

I posted this before, but it seems appropriate again, so here is the Mind Hacks post on bias blind-spot.

I do have a favorite blogger that gets under my skin through that sort of outside-group analysis. Unfortunately, the benefits to my epistemic health are not captured in any way to provide incentives for production.

Replies from: Jonathan_Graehl
comment by Jonathan_Graehl · 2009-07-21T04:09:48.199Z · LW(p) · GW(p)

The critique didn't push any of my us vs. them buttons, mostly because it resembled my existing view. Maybe if I remembered who Leon Kass was, I'd have been offended.

comment by AllanCrossman · 2009-07-21T01:24:11.108Z · LW(p) · GW(p)

Voted up, however, your second version of Roko's statement couldn't really fit into his post as a whole.

Basically, Roko's position was "I wish male rationalists who care about saving the world were also more interested in things like money and status and sex with women, because in the pursuit of those goals they would incidentally become better at saving the world."

Or at least, that's what I think he was saying. As I say, your second, "acceptable" version of what he said isn't really what he said at all.

Replies from: Psychohistorian
comment by Psychohistorian · 2009-07-21T03:11:00.950Z · LW(p) · GW(p)

This, I think, is the core of this whole ongoing disagreement.

He could have foregone mentioning "sex with attractive women" altogether, as it is one of many, many things that higher wealth and status may be conducive to the attainment of. He went out of his way to make a normative statement that men should value sex (specifically) with very attractive (only relevant adjective) women (plural), and thus that a failure to desire having sex with numerous attractive women was some form of objective shortcoming that deserves (and was given) suitable derision. This seems seriously unrelated to his point, since (1) if this is your goal, there are probably easier ways to obtain it than by seeking wealth and status, and (2) if you need to be motivated to obtain wealth and status, there are many, many other desires that could legitimately motivate you, not least of which are the goals you already have.

Had he said:

If people here were more motivated to obtain wealth and social status, it would be highly conducive to achieving goals they already have. Indeed, if people were to put a higher value on nice houses, fancy cars, other material goods, or more desirable romantic, social, or sexual partners, this change in values might help them better achieve the goals that they already have.

This, I believe, succeeds both at being less offensive and at better communicating the point in question.

I realize there may be something unpleasant in the implication that having greater wealth and status are conducive to the goal of having more desirable partners, sexual or otherwise, but that does seem to be, in most cases, reality - it often helps and almost never hurts, and the statement is made with an appropriate "might."

comment by djcb · 2009-07-21T20:56:01.011Z · LW(p) · GW(p)

While it's important ('rational') to be polite and to be inclusive of different groups, it should not be necessary to use velvet gloves either.

It's only natural that some of the background of the contributors will show through in their writing -- ie., their native language, gender, education, life experience and so forth will influence what and how they write.

If someone's background makes him or her biased in the truth-seeking process, this should of course be pointed out; but if the background only colors the language a bit, I think we shouldn't be overly sensitive of that, even when we don't really like the color.

comment by [deleted] · 2012-07-15T10:58:51.241Z · LW(p) · GW(p)

money and status would make a man more attractive to many women; men who really value a better romantic or sexual life should thus put more priority on money and status

It's funny how discussing attraction in cold hard psychological terms gives rise to debate. I am a man and therefore hard-wired to look at breasts and hips and feet for cues about whether the woman in question is fertile and non-pregnant.

That is not graceful, I don't even really meta-like it (of course I object-like it, I'm hard-wired to do so).

But that does not change the fact that I think of everyone as 'humans' or 'persons' before 'men' and 'women'.

Now, in the same vein, going with sexual cue-theory, women are just as hard-wired to evaluate which men are wealthy/high status and thus better able to support a family.

That does not change the fact that I am neither wealthy nor high-status (but maybe just a bit suave) but still have a female partner in a stable relationship of mutual trust, dependence and love.

Can we stop discussing genetic mental architectures as if they offend us? Because genetic mental architectures sure as hell don't care.

comment by Kaj_Sotala · 2009-07-21T09:31:20.308Z · LW(p) · GW(p)

Heh. I actually thought your reworded quote was better formulated and more convincing than my original post. Then I went back to look at it to realize that oh yeah, some of that was a straight copy from what I said back then. It may have been because your quote was more concise and to the point, while my post was more rambling. (Though obviously there's also the possibility of subconsciously recognizing those as my thoughts, and a "that must be a smart guy for sharing my suspicions" reaction.)

I've actually found that I often give the criticism of outsiders extra weight compared to that of insiders, exactly because they're less likely to be blinded by the communal biases and groupthink. Of course, this assumes that their critiques are basically coherent and don't start out with misunderstanding the very basic concepts.

A probably related phenomenon - I've noticed that if I'm writing something that's supposed to be persuasive, I sometimes like the first draft and consider it pretty solid. Then I imagine showing it to a certain rather critical friend of mine, and try to imagine what she'd say about it, and suddenly I see a lot of weak points that need more justification.

comment by jimrandomh · 2009-07-21T02:12:22.284Z · LW(p) · GW(p)

IAWYC. That said, I don't like conversations changing topic to grammar because someone misused a word, and I feel that conversations switching to gender issues because someone misused a pronoun are the same thing. The topic is fine when raised directly, as in this post, but obnoxious when it hijacks a conversation about something else, as it often does.

Replies from: orthonormal
comment by orthonormal · 2009-07-21T04:13:55.457Z · LW(p) · GW(p)

I don't think that these conflicts boil down to just grammar and pronoun usage; the problem of keeping your readers identified with you is IMO more subtle, general and significant than those sub-topics.

comment by CronoDAS · 2009-07-21T04:03:24.877Z · LW(p) · GW(p)

I don't know who Leon Kass is, but

Trying to think more like a mathematician, whose empiricism resides in the realm of pure thought, does not predispose these 'rationalists' to collect evidence from the real world. Neither does the downplaying of personal experiences. Many are computer science majors, used to being in the comfortable position of being capable of testing their hypotheses without needing to leave their office. It is, then, an easy temptation for them to come up with a nice-sounding theory which appears to explain the facts, and then consider the question solved. Reason must reign supreme, must it not?

is a definite temptation we need to avoid. We do seem to like coming up with such theories, and I know that I personally rarely seek out a way to test them, but I don't think we consider the question solved when we do, and that seems to be the problematic part.

I'm just here to have fun, though, so I don't really mind the "not testing" part.

comment by anonym · 2009-07-21T01:42:26.838Z · LW(p) · GW(p)

I completely agree with your underlying points (and voted up your article), but I wonder why you had to go and spoil an otherwise excellent post by "catastrophizing" the conflict and framing it as "the Less Wrong Gender Wars". That sort of silly hyperbole doesn't help; in fact, it is likely to make things worse.

Replies from: orthonormal, Bo102010
comment by orthonormal · 2009-07-21T04:08:13.446Z · LW(p) · GW(p)

I hope that unlike the "Culture Wars", the interested parties here would take the hyperbolic designation as a signal to get less rather than more confrontational in their exchange. Then again, priming still works on aspiring rationalists. What might be a better way to designate this cluster of recent arguments?

Replies from: anonym
comment by anonym · 2009-07-21T04:58:39.342Z · LW(p) · GW(p)

If you meant it ironically, I completely missed that, and I've seen enough other references to 'war' and the like here today to think that many people here take such metaphors seriously, too seriously.

comment by Bo102010 · 2009-07-21T02:57:24.218Z · LW(p) · GW(p)

I upvoted this article and this comment. Great post, but the "gender wars" are beneath this community.

comment by jajvirta · 2009-07-21T06:13:12.466Z · LW(p) · GW(p)

Adding to this, it seems to me [1] human brains are adapted to eagerly form group identification just from tiny cues. So we don't need secret handshakes or any other formal group forming rituals. All it takes is just a bunch of well-written articles with subtle cues that trigger our group identification algorithms. Even though we might not objectively be part of a group yet, our brains have already decided that we are and this in turn will form a huge bias in our mind of the subject matter itself.

This occurred to me when I read the start of the article and felt the effect of the trick that original article was intended to bring. I am not yet, objectively, a member of this community, but my brain apparently has already decided that I am. I have no idea what triggered this group identification. All I think I did was to read some interesting articles and discussion and make couple of quite irrelevant comments myself.

This is of course generalizing from one, but I hope someone points to some relevant studies in this subject. (Human group identification, that is.)

[1] I'm assuming already that this has been studied extensively somewhere. I have no references myself, though.

comment by wuwei · 2009-07-21T02:45:34.862Z · LW(p) · GW(p)

Great post.

I agree that you identify a very good reason to take care in the use of gender-specific pronouns or anything else that is likely to create in-group, out-group effects.

I also think there probably was a fair amount of attitude polarization on the question of how acceptable it was to make the statement in question.

comment by wedrifid · 2009-07-27T23:13:21.304Z · LW(p) · GW(p)

Does it seem to be less of a caricature now that you've read a version in which you identify with the writer, rather than one in which the writer is analyzing and criticizing you from outside?

Honestly, no. The only thought that passed my mind when the context was changed was 'Watch out Kaj! Make sure you've secured the aliances and status necessary to make that sort of honest observation. That's a recipe for getting yourself bullied and ostracised!"

Sometimes it might be necessary to go ahead and damn the torpedoes

Damn right.

comment by ajayjetti · 2009-07-21T23:57:29.636Z · LW(p) · GW(p)

Orthonormal, can you please post a link to blog that you quoted?

Out of the 23 comments so far, none has actually properly handled the question[1] raised by orthonormal.

Would love to see eliezer reply to this one

Please do that if possible.

  1. How seriously do you take this critique? Do you wonder why I'm bothering with this straw-man criticism of Less Wrong?
Replies from: orthonormal
comment by orthonormal · 2009-07-22T00:06:43.029Z · LW(p) · GW(p)

Ajay, there is no such blog; re-read the part directly after the cut. I took an excerpt from a Less Wrong post, changed it to the third person, and invented a plausible outside source for it, in order to make a point about our reactions to outside criticism versus critiques from within the group.

The linked post was, of course, quite interesting in its own right, if you want to look through that thread.

Replies from: ajayjetti, Douglas_Knight
comment by ajayjetti · 2009-07-22T01:50:53.777Z · LW(p) · GW(p)

yes, i am stumped!! the thing is clear from one of the comments. actually i was a bit sleepy (still am) and skimmed thru it, and missed the part after the cut. Fantastic post though.

I have a question: I am a real beginner here in this forum. Although i read a lot, the language used by you and many others in this forum is very high quality. The sentences are huge, which have to be re-read sometimes to understand what is being said. Although i know i will feel more comfortable with time, is it really simple english that is used in this forum? simple is relative, but lets say, is the english used here simple when compared with english used in most of the "standard" philosophy books or sites? i hope i am able to put across my point!!!

Replies from: orthonormal, thomblake
comment by orthonormal · 2009-07-22T02:02:53.848Z · LW(p) · GW(p)

I'd say that most of the writing here is much simpler than most academic philosophy, although some writers (including me) are fond of convoluted sentences now and then. Keep in mind also that English is a second language for many of the other contributors as well...

Also, welcome to Less Wrong! If you haven't yet visited the welcome thread, you should click over and say a bit about yourself.

Replies from: rhollerith_dot_com
comment by rhollerith_dot_com · 2009-07-22T04:16:14.281Z · LW(p) · GW(p)

I'd say that most of the writing here is much simpler than most academic philosophy

I would expect the writing here to be more difficult for a reader for whom English is a second language than the writing in the average philosophy book because the writing here is closer to informal spoken language.

comment by thomblake · 2009-07-22T01:57:22.196Z · LW(p) · GW(p)

Good question! We do have a lot of participants from not-primarily-English-speaking-countries, so perhaps one of them would be able to answer this better. However, this forum is filled with a higher-than-average number of quirky intellectuals, and we do love our jargon. Also, to seem more expert and smart, we even throw in technical jargon from our own respective fields! Okay, so there are actually good reasons for that other than showing off. But yes, I'd imagine the English here is very difficult as compared to even other smarty-pants forums on the web.

comment by Douglas_Knight · 2009-07-22T04:00:01.857Z · LW(p) · GW(p)

I got confused, too. I think I interpreted it to mean that the Kass acolyte was criticizing KS, but you hadn't included the quote; so I stopped reading and started scanning for a link to the source.

I would have found it easier if you had inserted "Actually," before "The quote is a..."

Do lots of articles play games with the cut? It probably would have been clearer to me if I'd read it with the cut. Maybe you could imitate the effect with an HR tag?

Replies from: orthonormal
comment by orthonormal · 2009-07-22T17:30:08.399Z · LW(p) · GW(p)

OK, good point— I've edited to make it much less ambiguous. I recall another article or two playing with deceit above the cut, and then revealing the trick below, but it's difficult to find them.

comment by bayesways · 2009-07-21T03:54:04.173Z · LW(p) · GW(p)

Best LW post I've read all week. . .