Trust the (local) expert

post by rk · 2018-04-09T10:22:13.133Z · LW · GW · 12 comments

Contents

  Misunderstanding is common
  Confirmation bias might have given you an expert far from consensus
  Experts might not be trying to communicate the truth
  Maybe cracked-pots look whole from afar
  Trust your mates
None
12 comments

Related: A Sketch of Good Communication [LW · GW], Argument, intuition & recursion [LW · GW]

Your friend is wrong, but you look like the crackpot. They might be studying the PhD in climate science, but you're deferring to world-renowned climate expert, Prof. Knows-a-Lot! And yet everyone's looking at you weirdly ... You feel like Morgan that time they tried to argue with Lisa. Even though Lisa works for the IMF and Morgan had sort-of read an interview with Nobel-winning economist ...

I think it's likely that the friend with the PhD has the right of the matter, and the friend with the global expert in their corner has it wrong. I mostly think that because often, experts didn't say what we think they said. That is, we misunderstood them. And this is less likely to happen in a direct conversation.

It feels like we should be able to absorb expert opinions and shortcut to correctness on many matters. And fortunately we have access to the views of many top experts: whether books, lectures or editorials, the considered views of the elite are available to the masses. But what to do in a case like the vignette at the beginning of this post: we confront another, lesser expert who assures us we are wrong. Maybe not even wrong, but fundamentally confused! It feels like we can't just flip-flop on our views whenever we hear of a disagreeing expert. Why on earth should we prefer an expert that's close to us in space over one that's far but preeminent? A plausible approach is to stick with the higher status expert.

But then, as I alluded to above, we've also all seen the other side of that. And it doesn't seem like the one quoting the expert is implementing a truth-tracking strategy. A possibility is that we're just responding to status markers. But I think there are good reasons to be sceptical of expert-quoting as an epistemic strategy.

Ideally, we'd like to find out the consensus position. We might not be able to go toe-to-toe with an expert we meet on the road, but at least we can say "I grant that you understand the situation better than I do, but a poll of economists found that 80% disagree with what you're saying now. So I'll assume that's true, even though I can't see a flaw in your argument". Sometimes we might not be able to research consensus in the timeframe in which we need to act. More commonly, consensus is just not available. So we need to figure out what to do with our few data points.

I've given the reasons I can think of for preferring the Professor's op-ed to a local PhD's statements above. Let me recap them. The strongest reason I can think of to ignore a local expert is that we have access to better / more prestigious experts remotely. The second is that our intuitions about who is right and wrong in a conversation often track status more than anything else.

If you can think of others, please let me know.

My strongest reason for preferring to believe the expert in front of you is just the likelihood that you have misunderstood something that the global expert said. That is, I accept the probability that someone with a PhD is wrong about something in their field is greater than in the case of a professor. However, the probability that the flaw lies in your understanding of what the professor claimed is much greater. I'll give some reasons for this below.

I also have three weaker reasons for preferring the local expert.

The first is that you may, perhaps unconsciously, have cherry-picked the expert that agrees with you. Confirmation bias might have got you! This obviously only applies if you're defending a view that you associate with.

The second reason is that prestigious, public experts might be optimising for status and making their audience feel smart & special rather than for communicating truth. I think this probably happens sometimes. But I also think there are countervailing factors, like the market for punditry (more below).

The final reason is that there may be markers of crackpottery that you can only use locally. So you might reliably be able to read if the local expert is very fringe, and correctly discard their viewpoint, but have less luck with the someone presenting their views on the internet. (So under this view, I'm saying you can discard a local expert's view, but only if you think they're a crackpot, not if you think they're a lesser expert)

Misunderstanding is common

I claimed above that you are more likely to have misunderstood the global expert than that the substantive disagreement is between the local and the global expert. To support this, consider the intuitions that the global expert has built up, so they have a much clearer sense about what is reasonable. They will have absorbed years of technical concepts and vocabulary. The inferential distance between you and them will be very large. Between imperfect communication on their part, and naïvety on your part, there's a lot of space for something to get caught in the gap.

For example, heritability is a concept that I and many others in my peer group have misunderstood and misapplied frequently. A trait is heritable if we can explain some of its variance with variance in the genotype. The more of the variance we can explain, the more heritable it is. It doesn't mean that the feature is biologically determined! There's still room for environmental factors, and even large background environmental effects. (Imagine a nuclear wasteland where tumours and radiation poisoning are the norm. Evolved resistances to the radiation might quickly come to dominate the phenotypic variation in tumours, but the environment would still be the largest causal factor in the absolute number of tumours)

If a local expert is telling you that you're wrong, and that childhood malnutrition would have a big effect on chance of making it as a professional athlete, while you rely on an expert who was arguing against a fad of 'baby all-star diets' by pointing out the high heritability of height, you are way off. The experts are in agreement. Because you're cut off from the prestigious expert's model [LW · GW], you are failing to integrate the arguments of the expert in front of you.

These kind of mistakes really are common: a friend of mine recently asked for help online in a subject where he has a master's. He didn't realise for a few days that he had misunderstood the professor who replied as saying two things implied each other when in fact he had been saying they were mutually exclusive. Communication is hard!

That said, doesn't my story have the problem that you could just have misunderstood the local expert? So though it may be more likely that the local expert is right than that your understanding of the global expert is right, that's comparing apples to oranges. We should compare your understanding of the global expert to your understanding of the local expert.

I think that's fair, but that there's a big asymmetry in terms of your understanding of the local expert vs the global expert. If you are working with someone in conversation, you can nail down precisely what the parameters of a disputed claim are. And then the expert just has to communicate whether they agree or not. The problem is, it often takes the conversational back and forth to delimit the claim.

Confirmation bias might have given you an expert far from consensus

Human brains are capricious, and we seek out support for positions we're already committed to. This could be a reason you end up deferring to a global expert, but reliably wrong. There're a lot of different professors in the world, chairs funded by groups with agendas, and just a lot of variation around some consensus position. If you are referencing a global expert, but you held the position you're defending before you were exposed to the expert, you should probably be pretty wary. In fact, unless you are confident that it is a consensus position, rather than this expert's position, you might want to lean towards ignoring the expert as evidence.

That said, even if you have a lot of handy tricks in your cognitive toolbox for defeating confirmation bias, or even if you had never heard of this position before and don't find it particularly appealing, you should still be careful when disagreeing with a local expert (for the other reasons listed here)

Experts might not be trying to communicate the truth

Popularising expertise might not be about communicating to the populace. That is, an expert has incentive to boost their readership and to cause a buzz. They may have less of an incentive to stick to the unadorned, uncontentious truth. What's more, not only do they have an incentive, they won't stay in the paper and you won't find their blog if it's not getting traction, so there's a strong selection effect.

While the incentives doubtless shade an expert's action, I think it's likely most are at least partly motivated by truth, which has an ameliorating effect. Similarly, there are enough truth-motivated readers and jealous colleagues around to lessen the severity of the incentive gradient. You're not going to make it long in the business of reasonably-mainstream popularisation if you can't make some show of honesty.

If the expert you get your info from was a particularly fun read, if you got a profound sense of understanding after listening to them speak, or if they are particularly popular, be extra careful when trying to draw on their views.

But even if you got a bootleg recording of a reclusive, fame-hating, supercilious mathematician trying to explain what they're working on to their spouse over dinner, and even if you receive divine confirmation that you understood them perfectly, and, yes, even if you'd never heard of the idea and you find it repugnant, it could still be that you are unusually poorly placed to detect fringeness of their view.

Maybe cracked-pots look whole from afar

One other weak reason to think you should trust a local expert over a global one, is that you may be less able to evaluate the all-in expertise and conventionality of a faraway expert than a nearby one. Maybe all and only crackpots are wild-eyed and crumple-suited, but their manner of dress isn't apparent online.

More seriously, maybe you have quite good info about the people in your social circles, and would know that they're doing a PhD in eldritch climatology rather than straight-down-the-line climatology. Whereas online, perhaps the clues are much weaker. If they're listed as part of a university's faculty, maybe have an article or two in national news outlets, they seem fairly legit. But it's possible that lets through people who, if you knew them personally, you would know are far outside the mainstream.

As I said, I don't put much weight on this reason, but I think it's possible.

Trust your mates

There are many subjects where it's worth becoming informed even if we can't dedicate the resources to becoming an expert in them. Often the material we grapple with to understand these topics comes from the top of their fields. We are then in a difficult situation when we personally meet a less impressive expert who disagrees with what we understand to be the top expert's opinion.

Considering others, we get the intuition that people often trap themselves with false beliefs by not deferring to experts they meet.

Considering ourselves, it seems there's no reason to prefer an expert that's close to us than one that's far but smarter.

I argue that we should prefer the closer expert for several reasons. The most important of these is that the probability of error between us trying to understand a expert and convert it to accurate beliefs is much higher with a global expert than a local one. We can push back on a local expert, and double-check we mean the same thing. Our flawed understanding of ideas that an expert wouldn't even bother to explain will become apparent to the local expert but not the global.

So: trust your low-level expert friends when they tell you you're wrong, even if you think you know a high-level expert who would disagree.

12 comments

Comments sorted by top scores.