The Fear of Common Knowledge

post by Eliezer Yudkowsky (Eliezer_Yudkowsky) · 2008-07-09T09:48:35.000Z · LW · GW · Legacy · 37 comments

Contents

37 comments

Followup toBelief in Belief

One of those insights that made me sit upright and say "Aha!"  From The Uncredible Hallq:

Minor acts of dishonesty are integral to human life, ranging from how we deal with casual acquaintances to writing formal agreements between nation states.  Steven Pinker has an excellent chapter on this in The Stuff of Thought, a version of which can be found at TIME magazine’s website. What didn’t make it into the TIME version is Pinker’s proposal that, while there are several reasons we do this, the most important reason is to avoid mutual knowledge:  "She probably knows I just blew a pass at her, but does she know I know she knows? Does she know I know she knows I know she knows?"  Etc.  Mutual knowledge is that nightmare where, for all intents and purposes, the known-knows can be extended out to infinity.  The ultimate example of this has to be the joke "No, it wasn’t awkward until you said, 'well, this is awkward.'"  A situation might be a little awkward, but what’s really awkward is mutual knowledge, created when someone blurts out what’s going on for all to hear...

The story of the Emperor’s New Clothes is another example of the power of mutual knowledge...

The power of real deception - outright lies - is easy for even us nerds to understand.

The notion of a lie that the other person knows is a lie, seems very odd at first.  Up until I read the Hallq's explanation of Pinker, I had thought in terms of people suppressing uncomfortable thoughts:  "If it isn't said out loud, I don't have to deal with it."

Like the friends of a terminal patient, whose disease has progressed to a stage that - if you look it up online - turns out to be nearly universally fatal.  So the friends gather around, and wish the patient best hopes for their medical treatment.  No one says, "Well, we all know you're going to die; and now it's too late for you to get life insurance and sign up for cryonics.  I hope it isn't too painful; let me know if you want me to smuggle you a heroin overdose."

So even that is possible for a nerd to understand - in terms of, as Vassar puts it, thinking of non-nerds as defective nerds...

But the notion of a lie that the other person knows is a lie, but they aren't sure that you know they know it's a lie, and so the social situation occupies a different state from common knowledge...

I think that's the closest I've ever seen life get to imitating a Raymond Smullyan logic puzzle.

Added:  Richard quotes Nagel on a further purpose of mutual hypocrisy: preventing an issue from rising to the level where it must be publicly acknowledged and dealt with, because common ground on that issue is not easily available.

37 comments

Comments sorted by oldest first, as this post is from before comment nesting was available (around 2009-02-27).

comment by Hopefully_Anonymous · 2008-07-09T10:10:01.000Z · LW(p) · GW(p)

good post, good find. I'm skeptical of the protective veil of innocence wrapped arouund the category of "nerd" in this post subtract that and I think it's a great, insightful post. Now I'd like to see this analytical insight applied to the range of popular human interactions.

comment by RobinHanson · 2008-07-09T11:18:41.000Z · LW(p) · GW(p)

Michael Chwe has a book on related social effects of common knowledge.

comment by Schizo · 2008-07-09T12:23:21.000Z · LW(p) · GW(p)

Reminds me of this puzzle (http://en.wikipedia.org/wiki/Common_knowledge_%28logic%29#Example). In fact, just read the whole page, it's good stuff, and mentions Pinker.

comment by Ian_C. · 2008-07-09T14:23:03.000Z · LW(p) · GW(p)

There is a version of truth which says things are not true if they match reality, but if enough people believe it. Maybe that factors in to the dying friend thing also.

comment by sophiesdad · 2008-07-09T15:14:56.000Z · LW(p) · GW(p)

Interestingly, there are categories of habitual liars (as distinguished from pathologic liars) who have no fear of common knowledge whatever. They lie in preference to telling the truth, and if caught, suffer no embarrassment or remorse. I once encountered such a person who was telling a story about a Wildlife Officer using an AK-47 assault rifle to kill a grizzly bear in West Virginia. When informed that no wild grizzly has ever been reported east of the continental divide, and that a state agency certainly would not issue a Russian weapon to its officers, the person simply continued, "Yeah, he was firing on full automatic from the hip, and the grizzly would have gotten him if he had only a pistol." See: http://www.answers.com/topic/pseudologia-fantastica-1?cat=health

comment by Zubon · 2008-07-09T16:05:43.000Z · LW(p) · GW(p)

Consider the "One China" policy. Everyone knows that the Republic of China ("Taiwan") is a separate country from the People's Republic of China ("China"). If either country admits this, or any major power tries to force this truth on them, the official Chinese policy is to prevent secession, the official American/Japanese policy is to protect Taiwan, and the expected outcome resembles World War III. We can all treat the two Chinas as separate countries, just so long as we do not admit that the Emperor has no clothes, at which point the bombing starts.

I really hope this is one of the many web pages currently blocked in China. If not, sorry about that. Also my apologies to the blue-eyed people in Schizo's link; I thought you all knew.

comment by Richard4 · 2008-07-09T16:40:22.000Z · LW(p) · GW(p)

See also Nagel's 'Concealment and Exposure':

"Admittedly nonacknowledgment can sometimes also serve the purpose of deceiving those, like children or outsiders, who do not know the conventions. But its main purpose is usually not to deceive, but to manage the distinction between foreground and background, between what invites attention and a collective response and what remains individual and may be ignored. The possibility of combining civilized interpersonal relations with a relatively free inner life depends on this division.

No, the real work is done by leaving unacknowledged things that are known, even if only in general terms, on all sides. The more effective are the conventions controlling acknowledgment, the more easily we can handle our knowledge of what others do not express, and their knowledge of what we do not express. One of the remarkable effects of a smoothly fitting public surface is that it protects one from the sense of exposure without having to be in any way dishonest or deceptive, just as clothing does not conceal the fact that one is naked underneath. The mere sense that the gaze of others, and their explicit reactions, are conventionally discouraged from penetrating this surface, in spite of their unstated awareness of much that lies beneath it, allows a sense of freedom to lead one's inner life as if it were invisible, even though it is not. It is enough that it is firmly excluded from direct public view, and that only what one puts out into the public domain is a legitimate object of explicit response from others."

For example:

"At the same party C and D meet. D is a candidate for a job in C's department, and C is transfixed by D's beautiful breasts. They exchange judicious opinions about a recent publication by someone else. Consider the alternative:

C: Groan....

D: Take your eyes off me, you dandruff-covered creep; how such a drooling incompetent can have got tenure, let alone become a department chair, is beyond me.

The trouble with the alternatives is that they lead to a dead end, because they demand engagement on terrain where common ground is unavailable without great effort, and only conflict will result. If C expresses his admiration of D's breasts, C and D have to deal with it as a common problem or feature of the situation, and their social relation must proceed in its light. If on the other hand it is just something that C feels and that D knows, from long experience and subtle signs, that he feels, then it can simply be left out of the basis of their joint activity of conversation, even while it operates separately in the background for each of them as a factor in their private thoughts."

comment by Richard4 · 2008-07-09T16:44:58.000Z · LW(p) · GW(p)

Incidentally, the lesson I take from Nagel's paper is that it's not really "common knowledge" that's the problem, so much as the act of raising such common knowledge to public salience. (We may still refrain from publicly acknowledging even facts we're privately quite certain that everyone else is also privately certain of, and so on.)

comment by Silas · 2008-07-09T16:56:53.000Z · LW(p) · GW(p)

How does this apply to religion? Is there e.g. a situation in which everyone knows that the tenets of the religion are false, and knows that everyone else knows they're false, but if this becomes common knowledge, the bond between the people breaks apart?

comment by Patrick_(orthonormal) · 2008-07-09T17:26:27.000Z · LW(p) · GW(p)

Silas: Some of the more progressive Christian denominations, perhaps? Most of the elite members have become entirely embarrassed of claiming things like the unique divinity of Jesus, but manage to keep this relatively silent (with the partial exception of defectors like ex-Bishop Spong) so as not to offend the more traditional believers in their communion (who of course know about the elites' unbelief).

The Episcopal Communion, in particular, is going more into schism the more people start to reveal their real theologies.

comment by HalFinney · 2008-07-09T18:48:59.000Z · LW(p) · GW(p)

This is a fascinating topic! I wonder though if Richard is right, whether the issue is not (just) common knowledge. It's confusing, because the usual way of ensuring that something is common knowledge is for someone to state it publicly. However it can be argued that this does more than merely making it common knowledge, in that it is a provocative action which demands a response.

In particular, in situations where there is a known but unacknowledged fact, is that fact actually common knowledge, or not? Everyone knows the fact, and everyone knows that everyone else knows the fact, but is there some point in the recursion where knowledge peters out? Would it be false that everyone knows that everyone knows that everyone knows that everyone knows the fact? Is there a "doubt amplification" effect where even the smallest doubt that everyone else really does know the fact, gets amplified as we proceed down the recursive chain, until when we go deep enough, we really don't know whether everyone else knows they all know it, to the nth degree?

Or OTOH would it be more correct to say that everyone knows that everyone knows... as far as you want to go, that the fact is truly common knowledge, yet it is still unacknowledged. And then as Richard quotes from Nagel, the problem is taking the next step and discussing it openly.

comment by HalFinney · 2008-07-09T18:56:19.000Z · LW(p) · GW(p)

Sorry to be double posting, but as I said... fascinating...

It occurs to me that this may be part of the reason some of us have so much trouble with eye contact. When we look at someone else's face, we see their emotions, and when they look at our face, they see our responses. But our knowledge of each other's intimate feelings is not common knowledge at this point. However, when we look each other in the eyes, said to be windows to the soul, we each know that each of us is seeing each other's feelings. Our mutual knowledge becomes common knowledge. It is a big step from merely looking at the other's body or even face, to looking each other in the eye - and that big step is the transition from mutual knowledge to common knowledge. Common knowledge has an element of infinite recursion and infinite depth to it, and so when we make eye contact we are in a sense moving from a finite degree of knowledge, to an infinite degree. In that sense, it is understandable that it feels like such a huge and difficult transition, and that maintaining eye contact is for example a significant marker of intimacy among lovers.

comment by Pookleblinky2 · 2008-07-09T18:56:26.000Z · LW(p) · GW(p)

Reminds me of that Discordian quote, "All statements are true in some sense, false in some sense, meaningless in some sense, true and false in some sense, true and meaningless in some sense, false and meaningless in some sense, and true and false and meaningless in some sense."

Hogwash for reality, but key to understanding human behavior.

comment by steven · 2008-07-09T19:11:52.000Z · LW(p) · GW(p)

Just to clarify, Hallq uses "mutual knowledge" as if it's synonymous to "common knowledge", but game theorists use the two terms as a contrast: mutual knowledge of A is when everyone knows A, common knowledge of A is when everyone knows that everyone knows that everyone knows (...) A. So this is about raising to common knowledge things that were merely mutual knowledge.

comment by Leo_G. · 2008-07-09T22:48:15.000Z · LW(p) · GW(p)

Not to start a traditional puzzle flame war, but can someone clarify for me the situation where at least three people on the island have blue eyes? It seems that in this case everyone knows that everyone knows [...] that there is "at least one person with blue eyes".

Blue-eyed person A: "Look at that poor sap over there with the blue eyes [B]. I bet he thinks there's only one person with blue eyes [C] on this island. Little does he know!"

Help?

comment by Allan_Crossman · 2008-07-10T00:55:26.000Z · LW(p) · GW(p)

Leo: It seems that in this case everyone knows that everyone knows [...] that there is "at least one person with blue eyes".

I've not actually read this thread, nor the Wikipedia page, because I like this sort of puzzle, and thought it would be fun to think about this. Here's what I came up with:

If there are three people on an island, all of whom have blue eyes, but none of whom know their own eye colour, then:

Alice might believe that Alice has brown eyes...

Therefore, she might believe that Bob believes that both Alice and Bob have brown eyes...

Therefore, she might believe that Bob believes that Carol believes that everyone's eyes are brown.

comment by ES · 2008-07-10T01:11:45.000Z · LW(p) · GW(p)

Leo G, if you assume the islanders were all dropped off on the island on the same day and given the instruction to leave if they have blue eyes, you would be correct. Most versions of the puzzle leave this ambiguous, but for the puzzle to make sense we have to assume that the islanders have no memory of exactly when they came to be on the island.

Even though everyone already knows everyone knows (etc) that at least one person has blue eyes, on the day after the announcement everyone knows everyone knows (etc) that no one left the island that day, therefore everyone knows everyone knows (etc) there must be at least two people with blue eyes, and this continues every day until everyone with blue eyes deduces that there must be one more blue-eyed person on the island than the number he counted and at that point all the blue-eyed people leave.

comment by Allan_Crossman · 2008-07-10T01:12:44.000Z · LW(p) · GW(p)

OK, I've read Wikipedia now (sorry, was being a bit obnoxious there). Replace "brown" with "green", and I think that's the gist of the idea.

comment by michael_webster2 · 2008-07-10T02:26:54.000Z · LW(p) · GW(p)

Eliezer Yudkowsky writes: "I think that's the closest I've ever seen life get to imitating a Raymond Smullyan logic puzzle."

You might find this interesting, then:

http://www.bizop.ca/blog2/due-diligence/what-is-too-goo.html

I think much of our signaling life needs to be informed by Smullyan puzzles.

comment by Phil_Goetz3 · 2008-07-10T14:21:50.000Z · LW(p) · GW(p)

There is a version of truth which says things are not true if they match reality, but if enough people believe it. Maybe that factors in to the dying friend thing also.
If that were so, the dying friend would get well.

comment by michael_vassar3 · 2008-07-10T16:55:50.000Z · LW(p) · GW(p)

Phil: It's more pleasant to believe that the dying friend will get well, so people are conditioned to believe it and to try to believe it with parts of their minds that are monitored by the other parts that do global situational evaluations and assign happiness or sadness. Their efforts are incompletely successful partly because they lead to disappointment and greater long-term unhappiness, but short term considerations are what normal operant conditioning runs on.

comment by HalFinney · 2008-07-10T17:13:21.000Z · LW(p) · GW(p)

The Emperor's New Clothes story has always bothered me. After the little boy says, but he doesn't have any clothes on, why doesn't everyone else just laugh and say, oh, look, the little boy is not sophisticated and clever enough to see the fine new clothes of the emperor, as I can? Wasn't that the con, that only the best quality people could see the clothes? So why should one person's announcement make it common knowledge that there were no clothes? The story doesn't quite work for me.

However, there certainly are situations where an unacknowledged fact is known by everyone, and then someone foolishly or clumsily announces it, making it common knowledge. This leads to a feeling of mutual embarrassment, but sometimes the result is merely an awkward silence, a pause, and then the remark is ignored and things go on as before. This kind of situation suggests that making certain facts common knowledge is only a partial step towards social catastrophe, and depending on the situation it may be possible to rescue things by pretending it didn't happen.

It's possible that such "rescueable" situations are precisely the ones where the uncomfortable fact was already common knowledge, hence announcing it doesn't fundamentally change things, and the only question is whether to respond to the announcement and thereby make the fact even more prominent. Then it might be that in situations where the fact was known but was not common knowledge, where some people suspect that others don't know, or at least that certain people don't know that other people know, the announcement would produce a sudden transition to the common-knowledge state. Then this would transform the social dynamics so dramatically that it would not be possible to pretend it hadn't happened.

Replies from: lucidfox, MugaSofer
comment by lucidfox · 2010-12-03T04:50:37.390Z · LW(p) · GW(p)

The point of that story is that everyone wanted to point out the flaw in the Emperor's "wardrobe", but everyone was too afraid to do it first.

An effect we see in real life fairly often: a chilling effect, an inconvenient truth that's not talked about by implicit consensus, until someone decides to break the ice, at which point the acknowledgement of the problem cascades - which opens a way to its resolution.

I've seen it happen in open source communities, for example. I pointed out organizational flaws in project X, at which point people commented that they are glad someone finally decided to speak out.

Replies from: Yosarian2
comment by Yosarian2 · 2013-01-24T23:56:43.968Z · LW(p) · GW(p)

I think the point in the story was that everyone assumed "I must be too sinful to see the emperor's cloths. Everyone else clearly can see them. There must be something wrong with me, so I should pretend I can see them too." Everyone, including the emperor himself, was burdened by the same self-doubt.

It was only when someone who was clearly without sin or guile said that he couldn't see the cloths that everyone else realized the truth.

(Unfortunately, in reality, I suspect the ending of the story really would have been "The child is a sinner and a heretic, or he would see the clothing just like all the rest of us CLEARLY do. Quick, burn the child!")

comment by MugaSofer · 2013-01-25T11:31:45.012Z · LW(p) · GW(p)

The Emperor's New Clothes story has always bothered me. After the little boy says, but he doesn't have any clothes on, why doesn't everyone else just laugh and say, oh, look, the little boy is not sophisticated and clever enough to see the fine new clothes of the emperor, as I can? Wasn't that the con, that only the best quality people could see the clothes? So why should one person's announcement make it common knowledge that there were no clothes? The story doesn't quite work for me.

Since everyone was pretending they could see the clothes, no-one considered the possibility that they might not exist. The child, who didn't know about the invisible cloth, simply assumed that the emperor was naked; when everyone considered this possibility, they realized they had all simply pretended they could see the clothes.

Of course, since children are often assumed to lack wisdom, the child's inability to see the clothes isn't reason to think they don't exist. (Why was the Emperor deliberately exposing himself to idiots? This always bothered me as a kid.)

comment by The_Uncredible_Hallq · 2008-07-10T17:40:02.000Z · LW(p) · GW(p)

Eliezer: Thanks for the link, man.

Silas & Patrick: Don't think this applies only to liberal denominations. If you accept Dennett's idea that even fundamentalism is less about ordinary belief than belief in belief, there's the potential for common knowledge to disrupt religion's function as belief in belief. But it's more obvious in liberal religion.

Steven: Thanks for correcting my terminology. I was doing my best to follow Pinker's usage, but was unsure of this point.

comment by Steve_Sailer · 2008-07-10T21:38:03.000Z · LW(p) · GW(p)

Re: the Emperor's New Clothes story. In modern America, people don't pay sincere attention to the little boy telling the truth, they just jeer at his obvious stupidity in not understanding that the Emperor is wearing a higher form of clothing that only sophisticates like themselves can see. What really causes a furor, however, is when somebody clearly more sophisticated than everybody else says the same thing as the little boy: see the abrupt end to the jobs of Larry Summers and James Watson for examples.

comment by unfortunate_conflict_of_data · 2008-07-11T04:25:00.000Z · LW(p) · GW(p)

the abrupt end to the jobs of Larry Summers and James Watson

As far as I'm aware, Summers merely claimed more variance in intelligence among males ("more prodigies, more idiots" as Pinker says) which is something a number of sane people have argued for. On the other hand, Watson seemed to claim an innate inferiority among black people. He certainly seemed to be on dubious ground.

(But it's possible to be charitable to Watson and interpret him as only claiming environmental causes for an intelligence difference. He also later hinted he had been misquoted.)

comment by GNZ · 2008-07-11T05:55:00.000Z · LW(p) · GW(p)

my impression of what Watson said was that he was describing not an innate difference (and to a geneticist surely that would be nonsense) but a difference that it is unlikely will disappear (which surely is true). And that his critics interpreted his position as being the more extreme version in order (maybe subconsciously) to protect society from the dangerous idea.

comment by Nick_Tarleton · 2008-07-11T06:43:36.000Z · LW(p) · GW(p)
On the other hand, Watson seemed to claim an innate inferiority among black people.

As far as I know, Watson only claimed lower average intelligence, not general "inferiority" or diminished moral worth. It's funny how many people seem to think he must have meant the latter.

comment by Allan_Crossman · 2008-07-11T18:41:55.000Z · LW(p) · GW(p)

ES: for the puzzle to make sense we have to assume that the islanders have no memory of exactly when they came to be on the island.

I don't see what difference that makes. All that matters is that everyone is present before it's announced that someone has blue eyes, and everyone has made an accurate count of how many other people have blue eyes, and nobody knows their own eye colour.

comment by Laura B (Lara_Foster) · 2008-07-12T02:27:11.000Z · LW(p) · GW(p)

Hal has something there with the eye contact. I think many people don't even realize the extent to which they are themselves 'lying' in social situations. False or mindless compliments you don't mean, laughing at banal jokes, saying you just love that band, putting on the persona of a 14 (or 55) year-old, strongly declaring 'opinions' that have not been thought out, strongly declaring 'emotions' that have not been reflected upon, invoking false bonds of kinship over superficialities, trying to be generally 'likable' to people at the expense of showing what you really think about the world or them (lie by omission)-- all are forms of dishonesty. Some people just can't stomach the masquerade. Others learn to exploit it. Others (myself included) are skilled actors, with a closet full of many other people's skins, though they long for the day when the lines and the costumes are unnecessary... at least not the ones that are no fun to say or wear...

That we presume people directly tell us anything at all about themselves in polite conversation is a fallacy. Anyone of any psychological skill knows the devil is in the details of the subtext. A good actor/expoiter makes eye contact and tells the other person what they are (what they should be) thinking with their reactions.

Eye contact- well that leaves you open to manipulation. If you know you're no good at this game, best to opt out. Or tell all.

comment by Laura B (Lara_Foster) · 2008-07-12T02:43:50.000Z · LW(p) · GW(p)

Also- thought: awareness of common knowledge collapses the wave function. If both Mary Jane and Captain Jack recognize that the situation of being alone together on a deserted beach while MJ's chubby-hubby is all viked-out back home is awkward, then there is an implicit acknowledgement that they are going to need to decide whether or not they are going to have sex, and that such a decision is a deliberate, rational, act on both of their parts, which makes them blame-worthy for the outcome of their actions. No room for plausible deniability if the Cap sticks his hand down MJ's pants, and she either slaps him, claiming she wasn't flirting (really honey-save it for court), or if she kisses him back, and later claims to have been 'suddenly overcome by irrational passion' (right... and you were overcome for the entire hour you spent holding hands on the beach beforehand???) Excuses Excuses Excuses....

comment by ES · 2008-07-12T17:08:08.000Z · LW(p) · GW(p)

@Allan Crossman

You're right. I was doing some sloppy thinking.

comment by Carlllord · 2008-12-16T15:24:30.000Z · LW(p) · GW(p)

if you like puzzles of Raymond Smullyan, sure that this site (http://4chests.blogspot.com) will also enjoy it.

comment by Paul_Crowley2 · 2008-12-24T02:17:37.000Z · LW(p) · GW(p)

Posting much too late, but about the Emperor's New Clothes: I had always interpreted the story to mean not that people stopped believing in the clothes when the little boy spoke out, but that it hadn't quite crossed people's minds until that moment that whether or not the King was clothed, he appeared naked to that boy, and that wasn't how things should be. Everyone laughs because they think, I too can see the nakedness of the King; only then do they realise that their neighbour can also see it, and only after that do they see that there are no clothes to see.

comment by [deleted] · 2016-03-11T14:17:25.085Z · LW(p) · GW(p)

My new friend Harry hates that I go out of.my way to illustrate that im just as knowedgable as he is, or to one up his mental flexing in.some way...reminds me of the psychiatrist writer who pathologises clients who try to act super well read or something. This is interesting..I feel then my knowledge isn't a strategic advantage of the harry situation. He didn't particularly likely from self revelation and his current friends don't suggest that either My high school friend Shane who is kinda similar never said he hates my behaviour of that, but the case still stands since he has dull friends he doesn’t just tolerate, but appreciate. And among my dull friends my knowledge is an obligation more than an asset