Overconfident talking down, humble or hostile talking up

When you’re communicating with people who know more than you, you have two options. You can accept their greater state of knowledge, causing you to speak more honestly about the pertinent topics. Or, you could reject their credibility, claiming that they really don’t know more than you. Many people who know less than you both may believe you over them.

A third option is to claim epistemic learned helplessness. You can believe someone knows more than you, but reject their claims because there are incentives to deceive. It's even possible to openly coordinate based on this. This seems like something I've seen people do, maybe even frequently. I can't think of anything specific, but one method would be to portray the more knowledgeable person as "using their power [in the form of knowledge] for evil".

Claim 2: It’s difficult to judge where on the curve people are who are higher than you, absent of sophisticated systems to support this.

Eliezer mentioned something like this in The Level Above Mine [LW · GW].

This nicely explains why I feel so embarrassed when I learn that someone I'm talking with is more knowledgeable than I thought. I wonder how to avoid subconscious overconfidence- / humility-projecting.

It might work to add a TAP for thinking "if this person were much more/less knowledgeable than me, would I have the same presentation in this conversation?"

As I read through, the core model fit well with my intuition. But then I was surprised when I got to the section on religious schisms! I wondered why we should model the adherents of a religion as trying to join the school with the most 'accurate' claims about the religion.

On reflection, it appears to me that the model probably holds roughly as well in the religion case as the local radio intellectual case. Both of those are examples of "hostile" talking up. I wonder if the ways in which those cases diverge from pure information sharing explains the difference between humble and hostile.

In particular, perhaps some audiences are looking to reduce cognitive dissonance between their self-image as unbiased on the one hand and their particular beliefs and preferences on the other. That leaves an opening for someone to sell reasonableness/unbiasedness self-image to people holding a given set of beliefs and preferences.

Someone making reasonable counterarguments is a threat to what you've offered, and in that case your job is to provide refutation, counterargument and discredit so it is easy for that person's arguments to be dismissed (through a mixture of claimed flaws in their arguments and claimed flaws in the person promoting them). This would be a 'hostile' talking up.

Also, we should probably expect to find it hard to distinguish between some hostile talking ups and overconfident talking downs. If we could always distinguish, hostile talking up is a clear signal of defeat.

In your post, after where it says:

Given these, let’s imagine a few situations:

There is a list that's numbered:

1.

1.

1.

instead of:

1.

2.

3.

I don't think it's inherently difficult to tell the difference between someone who is speaking N levels above you and someone who is speaking N+1 levels above you. The one speaking at a higher level is going to expand on all of the things they describe as errors, giving *more complex* explanations.

The difficulty is that it's impossible to tell if someone who is higher level than you is wrong, or telling a sophisticated lie, or correct, or some other option. The only way to understand how they reached their conclusion is to level up to their level and understand it the hard way.

There's a related problem, where it's nigh impossible to tell if someone who is actually at level N but speaking at level N+X is making shit up completely unless you are above the level they are (and can spot errors in their reasoning).

Take a very simple case: A smart kid explaining kitchen appliances to a less smart kid. First he talks about the blender, and how there's an electric motor inside the base that makes the gear thingy go spinny, and that goes through the pitcher and makes the blades go spinny and chop stuff up. Then he talks about the toaster, and talks about the hot wires making the toast go, and the dial controls the timer that pops the toast out.

Then he goes +x over his actual knowledge level, and says that the microwave beams heat radiation into the food, created by the electronics, and that the refrigerator uses an 'electric cooler' (the opposite of an electric heater) to make cold that it pumps into the inside, and the insulated sides keep it from making the entire house cold.

Half of those are true explanations, and half of those are bluffs, but someone who is barely has the understanding needed to verify the first two won't have the understanding needed to refute the last two. If someone else corrects the wrong descriptions, said unsophisticated observer would have to use things other than the explanation to determine credibility (in the toy cases given, a good explanation could level up the observer enough to see the bluff, but in the case of +5 macroeconomics that is impractical). If the bluffing actor tries to refute the higher-level true explanation, they merely need to bluff more; people high enough level to see the bluff /already weren't fooled/, and people of lower level see the argument see the higher level argument settle into an equilibrium or cycle isomorphic to all parties saying "That's not how this works, that's not how anything works; this is how that works", and can only distinguish between them by things other than the content of what they say (bias, charisma, credentials, tribal affiliation, or verified track records are all within the Overton Window for how to select who to believe).

Here [EA · GW]'s an old thread on the EA forum about how to assess expertise.

I guess this applies mostly to politicized topics? I also like Robin's advice to "pull the rope sideways".

How about adding units to the graph? What the X and what's the Y axis?

I'm generally hesitant to get into this line of thinking (and others like it) because knowledge is such a thoroughly multi-dimensional space and usually the ends people are looking to move towards with these kinds of models aren't terribly realistic.

I think the true answer is that it's both hard to know what anyone knows about a given field and it also very rarely matters. It reminds me of the talk "Superintelligence: The Idea That Eats Smart People" -- there's a habit among intellectuals, academics, and learned professionals (usually in that order) to get so caught up in their work that they think it intrinsically matters, when, really, nothing does (at least not to everyone).

You can be very "knowledgeable" in a field, double down on the wrong side of a schism, and then see years of your work become nearly worthless when your mental framework is empirically proven wrong. That work might also turn out to be useful again decades later for secondary or even unrelated reasons; when and where are you more or less knowledgeable than your peers here?

And to circle back to the Superintelligence talk: we as humans are very adept at finding ways to survive and thrive despite all kinds of uncertainty and threats, and one of the best tools we have for that is ignoring things until they're a major problem for us. In your radio intellectual example, I'd put forward that those kinds of situations arise because the presence or absence of such figureheads (or demagogues) doesn't generally matter to most people most of the time. When such people become burdensome and overbearing in their demands, they are ousted--entire governments have been bloodily overthrown from within and without for such reasons. That feels inefficient to the person who thinks such fields and their heads matter, but it's generally good enough.

My last point would just be that if it's really hard to know how much more knowledgeable than you someone is, how can you have confidence that someone knows more about specific sub-niche X than you, and not just more about overall field Y? Einstein probably knew more about Physics on the whole than just about anyone outside of a group that could fit into a single lecture hall, but if he looked at a suspension bridge's plans and wanted to "make corrections" I'd probably stick with a seasoned civil engineer unless they both agreed on review. The engineer would probably know more about the physics of suspension bridges in their home country than Einstein; if the latter was able to convince me otherwise that's a question of societal status and political skill in general.