Overcoming the Curse of Knowledge

No links for 'inferential distance'? The phrase itself is an inferential distance...

Great post. See also this related post by Eliezer.

ಠ_ಠ

When I was young people on LW didn't use pictures. There was text. And if you complained we threw another dozen links to walls of text at you.

Interesting study. It's definitely the intuition of LWers that people tend to underestimate the distance between your mind and someone else's mind. I think here this is commonly discussed as underestimating inferential distances. These ideas seem related to false consensus (actually, I'm interested in seeing some meta research about this). It seems like there should be some generalization of these ideas.

They weren't playing the game right. The way to correctly play the game, especially among siblings, is for a person to always pick the same song. For example, if I'm tapping, I'm tapping 'Jingle Bells'. And we have a near 100% success rate. (It is not quite 100% due to the initial learning curve, but the success rate then steadily improves over time.)

That said... I'm tapping a tune as I type on this keyboard. Can you guess it??

.

1) I suppose this is to be expected given priming, anchoring, and self-anchoring, but it's a worrying bias nonetheless. It certainly does help explain why inferential distances feel so hard to bridge.

2) I like the picture--it's a good visual metaphor for the point you're making.

3) I don't think the table of scientific words is good example--it's not that common people don't know anything about these words, they're simply used to using them in a nontechnical context whereas the scientists are used to using them in a technical context. The scientists' uses of the words are not more valid or more informed than the laymans', they're just contextually different. Many of the examples in the table (especially "values," "bias," and "error") aren't the result of a knowledge gap, but of a simple definitional dispute.

Using technical definitions and ignoring folk meanings is something I've been noticing more in myself and others lately. Until I started making the distinction when listening to others I never realized how painfully bad it must sound when I do it.

"The problem is that tappers have been given knowledge (the song title) that makes it impossible for them to imagine what it's like to lack that knowledge. When they're tapping, they can't imagine what it's like for the listeners to hear isolated taps rather than a song. This is the Curse of Knowledge. Once we know something, we find it hard to imagine what it was like not to know it. Our knowledge has "cursed" us. And it becomes difficult or us to share our knowledge with others, because we can't readily re-create our listeners' state of mind."

Imagine that the tapper is familiar with a song and expects the listener to be familiar with it, but he doesn’t know the song’s title, or any word that he would expect the listener to recognise as a name for it. If he taps out a rhythm from the song, he will have the song ‘playing’ in his head as he does so. This might lead him to overestimate how accurate the rhythm is, or how easy the song is to distinguish from that rhythm alone. When he hears the rhythm that he is tapping out, internally he also hears the song, making it very difficult to judge what it would be like for someone just to hear the rhythm.

Although it would be difficult to create an experiment in which knowledge of a popular song’s title and “musical knowledge” of the song are rigorously separated, we see that the tappers have two kinds of knowledge that the listener lacks: the song title, and musical knowledge of the song – i.e. the ability to replay the song with decent fidelity in one’s mind.

Chip and Dan Heath suggest that knowledge of the song’s title is the critical knowledge that causes the tappers to be overconfident about how likely the listeners are to recognise the song. This is a complex statement about the way in which the human brain works, but we don’t yet understand the brain in great detail; as far as I am aware there is no particular neuroscientific evidence that should cause us to believe that knowledge of the song title, rather than musical knowledge of the song, is the crux of the problem. Therefore I am not inclined to view their explanation as authoritative, and (assuming that there isn't just some problem with the experiment) on the strength of introspection I lean towards the idea that musical knowledge has the greater effect in causing this overconfidence.

The failure to try to correct for this bias could be regarded as an example of irrationally expecting short inferential distances, but I view that as being more related to the understanding of words and concepts - complex areas of the map, that are charted on one person's map in a particular way but not on another's. I think a better fit would be the mind projection fallacy: the error of projecting the properties of one's own mind into the external world. In this case the property being projected is the person's internal soundtrack accompanying the rhythm that he is physically tapping.

Actually, expecting short inferential differences is a special case of the mind projection fallacy – the property being projected is the detailed structure of words and reductions in someone’s map of the territory; the person in question fails to realise that the reductions, the causal relationships and the subtle definitional changes that accompany words in his mental model do not accompany those words when they are processed by the brains of other people.

So, “the curse of knowledge” is essentially the problem of the mind projection fallacy as it applies to a given person’s level of knowledge (of various kinds) with respect to the knowledge possessed by others, and this knowledge need not be encoded verbally.

And it is "dumbing things down", since the not ignorant general understanding, that is of the well-educated, but not necessarily scientifically-educated, population has similar usages. Where do you think the scientific usages came from?

You mean overconfident by a factor of 20.

i see...a blob. where I know the "baby" is. I'm sure it'd resolve into a baby if i let my perceptions keep chewing on it, but I happen to be in an abnormal state of tiredness right now.

I get this to crippling extremes. Not only with science stuff, but with every single culture or subculture I've interacted with, and a bunch of specialized sites like TVtropes or LessWrong. Not to mention things I've just observed on my own and never learned or come up with a name for. I'm basically incapable of communicating anything with anyone at all by this point.

What is being discussed is the differences between the 4 stages of learning.

1st A person doesn't know what they don't know. 2nd. A person knows that they don't know. 3rd. A person needs conscious thinking to know what they know. 4th. A person knows what they know automatically.

The expert who is unable to express themselves to the layman is in the 4th stage. A teachers best position is in the 3rd stage, for they are conscious about what they know and hence can assist others in knowing. A person in the 2nd stage is ready to learn, they accept they don't know it all, but they wouldn't be an effective teacher. A person in the 1st stage isn't ready to learn and couldn't teach either.

My preferred term for this phenomenon is "expert blindspot". See also "illusion of transparency": http://lesswrong.com/lw/ke/illusion_of_transparency_why_no_one_understands/

I notice that general vocabulary has a lot more value judgement in it. Most of this is likely due to descriptive, value-neutral words being generated every time there is a new thing to describe, whereas new good or bad properties are rare (or, at least, rarer: see hacker jargon for an example of specialized vocabulary with lots of value judgements, yet even more value-neutral words). I suppose another effect is that, when not thinking deeply about a problem (and thus not creating jargon), it is convenient to rely on general good/bad feelings about things.