Great minds might not think alike
post by Eric Neyman (UnexpectedValues) · 2020-12-26T19:51:05.978Z · LW · GW · 45 commentsThis is a link post for https://ericneyman.wordpress.com/2020/12/26/alike-minds-think-great/
Contents
I. II. III. IV. V. VI. None 45 comments
[Previously known as "Alike minds think great"]
I.
It is famously the case that almost everyone thinks they’re above average. Derek Sivers writes:
Ninety-four percent of professors say they are better-than-average teachers.
Ninety percent of students think they are more intelligent than the average student.
Ninety-three percent of drivers say they are safer-than-average drivers.
Interesting. Intuitively this seems to suggest that people are prone to vastly overestimate their competence. But is that true? As Bill Kuszmaul points out, these people aren’t necessarily wrong!
There’s no fundamental reason why you can’t have 90% of people be better than average. For example, more than 99.9% of people have an above-average number of legs. And more than 90% of people commit fewer felonies than average. These examples are obvious, but they’re not so different than some of the examples [in Sivers’ post].
This has something to it! On the other hand, I don’t think this explains everything. Is the quality of a professor’s teaching really so skewed that 94% are above average? But more importantly, do you really think that way fewer people would answer “yes” if you just replaced the word “average” with “median” when asking the question?
That said, I don’t think these numbers necessarily point to a bias! That’s because the interpretation of “above average” is left entirely up to the person being asked. Maybe you think a good driver is one who drives safely (and so you drive safely and slowly) whereas I think a good driver is one who gets from point A to point B efficiently (and so I drive quickly but not safely). We are both, from our own perspectives, above average drivers!
Put otherwise, for any skill where “goodness at that skill” doesn’t have an objective, agreed-upon measure, we should expect more than 50% of people to think they’re better than the median, because people optimize for things they care about.
To give a personal example, I suppose I would call myself an above average blogger. This isn’t true in some objective sense; it’s just that I judge bloggers by how interesting their thoughts are to me, and obviously I write about things that are interesting to me! There’s no bias I’m falling for here; it’s just that “Are you an above average blogger?” leaves “above average” open to my interpretation.
II.
There is, however, a closely related bias that I and lots of other people have. This bias occurs when we take a situation like those above, but now create a more objective test of that skill. To illustrate with an example, suppose you asked all the students at a university whether they have an above-median GPA. If 90% of students said yes, that would demonstrate a widespread bias — because unlike “Are you a better than the median student”, here there’s no room for interpretation.
The way this bias manifests in me (and many others I imagine) is: I tend to underestimate the competence of people who think very differently from me. I started thinking about this the other day when I listened to Julia Galef’s podcast episode with David Shor (which I highly recommend1). Shor is a young Democratic political strategist, originally hired by Barack Obama’s 2012 reelection campaign to run their data operation and figure out how the campaign should spend its money. Shor says:
When I first started in 2012, I was 20 and I was like, “Oh, I’m going to do all of this math and we’re going to win elections.” And I was with all these other nerds, we were in this cave. We really hated these old school consultants who had been in politics for like 20 years. […] We had all these disagreements because the old school consultants were like, “You need to go up on TV, you need to focus on this..” And we really disagreed.
Put yourself in Shor’s shoes: you join an operation that’s being run by consultants who have no background in data, haven’t looked at randomized controlled trials on different interventions, etc. You know tons of math, start reading about the RCTs, looking at poll numbers, and they point you toward really different things from what the campaign was doing. You’d probably be really concerned that the campaign was doing things totally wrong! But as Shor goes on to say:
I think going back, probably 80% of the disagreements I had with these old school consultants in 2012, looking back, I think they were right.
I don’t want to accuse David Shor of any particular bias without evidence, but allow me for illustrative purposes to create a fictional David Shor who may or may not reflect the actual thoughts of the real David Shor in 2012. If you’d like, you can think of this character “strawman-Shor”, or as my own self-insertion.
Suppose Obama were to ask David Shor: “Be honest, do you think you’re better than the old school consultants at real-world reasoning?” If he suppressed any false modesty, he might have said something like:
Well, I have a good grasp of statistics and good logical reasoning skills. Show me a trend in home ownership and I’ll tell you if it’s robust or just some nice-looking noise. Tell me the sensitivity and specificity of a cancer screening test and the prevalence of that cancer, and I’ll tell you how likely someone who tested positive is to have cancer. Give me a spreadsheet of historical polling data and I’ll tell you if you’re likely to win the election. I bet the old school consultants can’t do that as well as I can. So yeah, I’m better at real-world reasoning.
Fair enough. Now suppose Obama were to ask an old school consultant (let’s call her Constance): “Do you think you’re better than David Shor at real-world reasoning?” She might have said:
Wait, who’s David Shor again?
and then
Oh, that quant we just hired who’s fresh out of college? Listen, I understand how people think really well. Give me an opinion piece and I’ll read between the lines to figure out the author’s motivations. Drop me off at a party full of people I haven’t met before and by the end I’ll have a pretty good idea of what makes them tick. Ask me to spend ten minutes with a swing voter and I’ll tell you what things really motivate them. I bet Shor couldn’t do any of these things. So yeah, I’m better at real-world reasoning.
Which — again — fair enough! Neither of them are wrong; they just place a high value on different skills. The sort of real-world reasoning that is salient to Shor is figuring out how systems work by analyzing them logically and statistically. By contrast, Constance thinks of real-world reasoning as understanding how people behave. If Shor is better at the former and Constance is better at the latter, but they think of “real-world reasoning” in different ways, both of their replies are “correct”.
But now, Obama asks Shor a follow-up question. “So, do you think you’re better than the consultants at figuring out how to spend our money so as to most increase our chances of winning?” And I say:
Yeah — this is a great example of real-world reasoning, and I’m better at that.
And when Obama asks Constance, she also says:
Yeah — this is a great example of real-world reasoning, and I’m better at that.
In this case — if David Shor’s 80% estimate from the podcast is right — Constance would have been right most of the time. It could have easily been the reverse, if Obama chose to ask about a different concrete measure! But the point is: Shor and Constance both — predictably — said they’d be better at the thing Obama asked them about.
I would guess that this sort of reasoning happens a lot. In concrete terms:
- A person (call her Alice) forms a heuristic — “I am good at X” — where X isn’t perfectly defined. (“I am good at real-world reasoning”; “I am good at driving”; “I am a good math teacher”.) She forms it because she’s good at X on a particular axis she cares about (“I am good at statistical problem solving”; “I drive safely”; “My algebraic geometry classes consistently get great reviews”).
- Alice encounters a specific problem where being good at X is important, though maybe in a different form. (Figuring out how to spend campaign money; Getting to the airport really quickly; Teaching young kids arithmetic.)
- Alice pattern matches the problem as an instance of X, thinks I’m good at X, and (this is the fallacy) concludes I’m good at this problem.
Shor and Constance are both good at real-world reasoning, but in different ways. If they both end up committing this fallacy, the end result is that each of them thinks they are better than the other at the concrete problem at hand (spending campaign money). More generally, misapplying heuristics in this way would lead you to overestimate the competence of like-minded people (who are good at X along the dimensions you value) relative to those who think differently.
The last thing I want to say here is: from personal experience, I’m pretty sure that overestimating the competence of like-minded people (if you want a concept handle for this bias, may I suggest “alike minds think great”?) actually happens. But why do I think that the mechanism I just proposed is the right explanation? Well, I don’t think it’s all of the explanation (I can think of other plausible mechanisms2) but I would speculate that it’s part of what’s going on, if only because I’m pretty sure that my brain often makes the sort of mistake I outline in steps 1 through 3.
III.
It might be inconvenient that Shor and Constance underestimate each other’s competence, but the problem runs deeper. If this bias were the entire problem, the two of them could have laid out their arguments, judged the relative merits, and resolved the disagreement over campaign spending. The deeper issue is that Shor and Constance have different ways of thinking about the world, so resolving this disagreements is really tricky. Shor is inclined to distrust Constance’s reliance on intuition and past experience, and Constance is inclined to distrust Shor’s reliance on statistical methods. Like many smart people with different ways of thinking, they’re likely to talk past each other. But, like, they’re on a team together. They have to come to a decision. How are they supposed to do that?
First, both Shor and Constance could stand to be a bit less confident. If only because the previously discussed bias is a real possibility, they should both entertain the notion that the other is right and they are wrong. From there it isn’t too hard to strike some sort of balance between the two approaches.
But, striking some arbitrary (maybe 50/50) balance isn’t great either! If Shor could have figured out back then that Constance was right 80% of the time, they would have been able to allocate Obama’s resources much better. So it’s truly important for them to really figure out who’s right.
And for that, they need a really important sort of person. The more I think about it, the more I believe that this is an utterly crucial role for some members of society to fill. This role is the translator.
(Edit: Kelsey Piper independently coined the same word for the same concept in 2015!)
IV.
In June, when Scott Alexander put his blog on pause over a possible New York Times article that would reveal his real name, outrage erupted on Twitter. The vast majority of comments were supportive of Scott and called for the New York Times to withhold his name from the piece. But I also saw some criticisms of Scott, of which one stuck out to me. It went something like this:
Scott Alexander’s blog is really overrated. People seem to think he’s really insightful, but as far as I can tell he just takes really obvious concepts and explains them in an unnatural, convoluted way.
I dismissed this critique as… obviously wrong?… at the time. Scott’s essays were consistently enlightening, fitting different pieces of the world together like a jigsaw puzzle, helping me understand the world better. Some examples:
- I Can Tolerate Anything Except The Outgroup, which made me understand why people often get much more upset at those with a slightly different opinion than at those who are completely opposed to everything they believe in.
- The Toxoplasma Of Rage, which made me understand why activists so often seem to use divisive tactics.
- Right Is The New Left, which gave me a model of how beliefs interact with social status, and also gave me a model of how fashion trends work, something I previously had no conception of at all.
In all three of these (admittedly somewhat cherry-picked) examples, Scott explained a social phenomenon with an analytical model. This was fantastic for me! It’s more difficult for me than for most others to understand social dynamics at an intuitive level; but analytical models — that’s totally my thing.
Whereas from the critic’s perspective, these explanations didn’t appear particularly insightful. Of course people get upset at the outgroup more than the far-group. Of course it makes sense for activists to use divisive tactics. Of course changes in fashion styles are a the result of a status game. Perhaps Scott Alexander’s critic grasped all of these things on an intuitive level. If you will, the critic is somewhat like Constance, and I’m somewhat like Shor.
Scott Alexander is what I would call a translator. He takes concepts that are really natural to the Constances of the world and explains them in a way that makes sense to the Shors of the world. And this is really, really useful. It’s useful to Shors because it makes them understand the world better. But it’s also useful to Constances because it’s good for them too if Shors understand these things. It makes it easier for them to reconcile disagreements, and it makes Shors come to better conclusions in e.g. matters of public policy that affect Constances too.
(Edit: I don’t mean to suggest that Scott Alexander’s explanations can’t be useful for Constance — but at minimum it’s harder for her to understand the insights since they aren’t written in Constance’s “native language”. And I certainly don’t mean to suggest that translation is the only value of the posts.)
There are no doubt effective translators in the other direction, too. An example would be a mathematician or statistician who’s really good at using real-world examples to explain abstract concepts to people who aren’t mathematically inclined. Jordan Ellenberg and Eugenia Cheng might be good examples (though I don’t know, since I’m not their audience!).
V.
Being a translator requires a pretty unique skill set. Scott Alexander needs to understand social dynamics really well. He also needs to understand analytical methods well enough to use them in his explanations. He also needs to be able to speak the language of Shors, because his explanation will ultimately be in their language. Think in terms of language-to-language translators: to translate really well from language A to language B, you need be masterful at A (to understand all the subtleties of the meaning) and also at B (to convey that meaning while preserving those nuances). That’s why good translators (in both senses) are so rare.
But because there are so many Shors and so many Constances in the world, translators are really, really important. Having translators would have given David Shor and the consultant an opportunity to figure out which of them was right (or at least come closer to an agreement). Let me give an example of what a Constance-to-Shor translator could have said:
Constance’s intuition that Obama should use a slogan that highlights his position on economic issues comes from her work in the 1990s, when there were lots of persuadable voters and you could measure effects of candidates’ speeches in real time. From that time period we have lots of examples of economic issues playing better for Democrats than cultural issues. Now, times have changed, and these effects have gotten smaller as the number of persuadable voters has diminished, but the “what persuades undecided voters hasn’t changed” prior is a reasonable one to start with. Now, your data should make us update toward considering the alternative more effective, but given the messiness of our data and the small effect sizes, I would argue that we should mostly stick to our prior.
How do I know that this would have been the right thing to say? Because in the podcast, this is how David Shor currently explains why the old-school consultants ended up being right! It would have been really useful for Shor to have this sort of explanation at the time. But — I theorize — there wasn’t an effective translator. This is no surprise — as I mentioned, translators are few and far between — but this example goes to show how useful a translator can be.
VI.
I’ll conclude by noting that while I talked specifically about translators between analytically-minded and socially-minded people, translators can exist and are extremely useful between any two groups of people who think really differently! Some examples:
- Translators between different ideologies. Liberals and conservatives not only have different beliefs, they reason about political issues totally differently. They have different notions of fairness and justice, different models of society, etc. It’s really useful to have someone who can present conservative arguments in liberal-friendly language and vice versa. A good example here, perhaps, is Bill Clinton, who was called “explainer in chief” for being able to explain liberal ideas in ways that appealed to conservatives.
- Translators between different cultures. People from different cultures are often brought up to see the world in different ways, and translators are important for bridging cultural divides.
- Translators between different socio-economic classes. We often talk about a disconnect between coastal elites and non-elites. In fact, this is (probably correctly) pointed to as a major source of political polarization. Effective translator could bridge this divide, perhaps decreasing polarization (or at least slowing its increase).
Fostering mutual understanding is really important for social cohesion and for truth-seeking. Doing this between two people who have similar thought patterns is relatively easy. Helping Republicans understand Democrats, economic elites understand middle-Americans, Shors understand Constances — that is hard. For that, you might just need a translator.
1. The day before the episode was released, I wrote a tweet that listed Julia Galef and David Shor as two of the four people I follow on Twitter who reliably have great tweets. (They both “liked” the tweet, no doubt knowing that they were about to make me quite happy.) I sure had high expectations, and the episode did not disappoint.
2. One other possible mechanism: when people discuss technical things with people who think differently from them, they often don’t understand those people’s points (to a greater extent than discussing with like-minded people). Their brains jump from “this don’t make sense to me” to “this person’s thoughts don’t make sense”, causing them to underestimate the person’s competence.
45 comments
Comments sorted by top scores.
comment by Austin Chen (austin-chen) · 2020-12-27T21:43:57.160Z · LW(p) · GW(p)
Great post! One note:
Think in terms of language-to-language translators: to translate really well from language A to language B, you need be masterful at A (to understand all the subtleties of the meaning) and also at B (to convey that meaning while preserving those nuances). That’s why good translators (in both senses) are so rare.
Mastery of both A and B is great, obviously, but if you can choose only one, choose B.
I've spent a decent chunk of my life scanlating manga from Japanese to English, and my observation is that fluency in the target language (English, in this case) is much more important for a good translation than fluency in the source. I can overcome a misunderstanding in Japanese with copious amounts of research (Google Translate, JP dictionaries, etc); but the thing that my readers consume is a product in English, which is much harder to "fake".
Two takeaways, continuing on the translation analogy:
- If you want to get into cultural translation, start by writing for the audience you know really well, and then do research into the source culture. My bet is that Scott is more "fluent" in the analytical audience, not the social one.
- Scanlation teams often have a JP to Eng translator, fluent in JP, and a second English editor who can clean up the script. Cultural translation may also benefit from two people from different cultures collaborating (SSC's adversial collaboration comes to mind)
↑ comment by Eigil Rischel (eigil-rischel) · 2021-01-02T17:24:45.544Z · LW(p) · GW(p)
My mom is a translator (mostly for novels), and as far as I know she exclusively translates into Danish (her native language). I think this is standard in the industry - it's extremely hard to translate text in a way that feels natural in the target language, much harder than it is to tease out subtleties of meaning from the source language.
↑ comment by DirectedEvolution (AllAmericanBreakfast) · 2020-12-27T22:22:42.720Z · LW(p) · GW(p)
This is a nice productive way of extending the conversation and generating a piece of actionable advice from the model. Strong upvote.
comment by mingyuan · 2020-12-28T07:03:30.766Z · LW(p) · GW(p)
This was a great post on a subject that hadn't occurred to me before. My main criticism is that it was not at all what I expected from the title, and therefore I think "alike minds think great" is not a good name for the concept. If I were referencing it, my hypertext would probably say something like, "you optimize for being good at the things you care about people being good at," which admittedly is a whole lot less pithy, but I feel it's clearer.
Replies from: UnexpectedValues, Measure↑ comment by Eric Neyman (UnexpectedValues) · 2020-12-29T03:10:13.727Z · LW(p) · GW(p)
Thanks for the feedback. Just so I can get an approximate idea if this is the consensus: could people upvote this comment if you like the title as is (and upvote mingyuan's comment if you think it should be changed)? Thanks!
Also, if anyone has a good title suggestion, I'd love to hear it!
Replies from: mark-xu, Pattern↑ comment by Mark Xu (mark-xu) · 2020-12-29T16:49:54.845Z · LW(p) · GW(p)
Here are some possibilities:
- great minds might not think alike
- untranslated thinking sounds untrustworthy
- disagreement as a lack of translation
↑ comment by Eric Neyman (UnexpectedValues) · 2021-01-01T22:22:27.651Z · LW(p) · GW(p)
Thanks! I've changed the title to "Great minds might not think alike".
Interestingly, when I asked my Twitter followers, they liked "Alike minds think great". I think LessWrong might be a different population. So I decided to change the title on LessWrong, but not on my blog.
↑ comment by Neel Nanda (neel-nanda-1) · 2020-12-31T16:23:26.427Z · LW(p) · GW(p)
I love " great minds might not think alike"
↑ comment by Pattern · 2020-12-29T18:26:09.146Z · LW(p) · GW(p)
Alike minds think they think great.
Alike minds think alike minds think great.
(Systematically) Overestimating the effectiveness of similarity*
*This one points towards possibilities like
1. people aren't evaluating 'how good/effective is (someone else)' but 'how well would I work with them', or 2. something about the way 'the value of similar contributions' is valued.
These seem testable.
comment by Unnamed · 2020-12-26T21:12:17.469Z · LW(p) · GW(p)
Related to section I: Dunning, Meyerowitz,& Holzberg (1989) Ambiguity and self-evaluation: The role of idiosyncratic trait definitions in self-serving assessments of ability. From the abstract:
Replies from: VaniverWhen people are asked to compare their abilities to those of their peers, they predominantly provide self-serving assessments that appear objectively indefensible. This article proposes that such assessments occur because the meaning of most characteristics is ambiguous, which allows people to use self-serving trait definitions when providing self-evaluations. Studies 1 and 2 revealed that people provide self-serving assessments to the extent that the trait is ambiguous, that is, to the extent that it can describe a wide variety of behaviors.
↑ comment by Vaniver · 2020-12-27T17:47:41.867Z · LW(p) · GW(p)
As it happens, I discovered this point in high school; I thought of myself as "the smartest kid at school," and yet the mental gymnastics required to justify that I was smarter than one of my friends were sufficiently outlandish that they stood out and I noticed the general pattern. "Sure, he knows more math and science than I do, and is a year younger than me, but I know more about fields X, Y, and Z!" [Looking back at it now, there's another student who also had a credible claim, but who was much easier to dismiss, and I wouldn't be surprised if he had dismissed me for symmetric reasons.]
comment by habryka (habryka4) · 2021-01-01T21:04:42.472Z · LW(p) · GW(p)
Promoted to curated: This post covered a topic that has been covered in a number of other posts in a pretty comprehensive way, and feels like one of the best references on this argument. It also said a substantial number of new things that I hadn't put that way before. In particular I really liked the opening, which besides it's main point also helped me put into better words some problems I've had with Dunning-Krueger-like arguments in the past.
I also share people's sense that the title of the post didn't really fit. I skipped over this post for a few days because of that.
Overall, thank you a lot for writing this!
comment by Felix Karg (felix-karg) · 2020-12-27T16:48:21.027Z · LW(p) · GW(p)
Someone should start a collection of good translator-resources between different thinking styles. I might end up working on something similar in the far future ...
comment by ChristianKl · 2020-12-27T14:40:33.260Z · LW(p) · GW(p)
It's worth noting that there's a huge conflict of interest between the Shors and Constances. Their salaries depend on people believing that their respective skills are more important.
Replies from: Oliver Sourbut↑ comment by Oliver Sourbut · 2021-01-02T09:01:48.422Z · LW(p) · GW(p)
Good point. I guess a good manager in the right context might reduce that conflict by observing that having both a Constance and a Shor can, in many cases, be best of all? And working well together, such a team might 'grow the pie' such that salary isn't so zero-sum...?
In that model, being a Constance (or Shor) who is demonstrably good at working with Shors (Constances) might be a better strategy than being a Constance (or Shor) who is good at convincing managers that the other is a waste of money.
Replies from: ChristianKl↑ comment by ChristianKl · 2021-01-02T11:38:26.784Z · LW(p) · GW(p)
In reality, there are more people then just Constance and Shor. In particular the Constances and Shors in the Obama campaign managed to get grassroots organizers defunded (Howard Dean's 50 state stategy did get defunded). The consolidations of the campaign spending on a few firms is part of the background of this conflict.
comment by Measure · 2020-12-27T18:20:41.012Z · LW(p) · GW(p)
Another aspect of the above-average effect is that in situations like street legal driving there isn't much opportunity to display exceptional skill. This results in people seeing lots of reasonably competent drivers and a few really bad drivers - exactly the skew that would result in most drivers being above average.
Replies from: avishek-shaw↑ comment by Avishek Shaw (avishek-shaw) · 2021-01-05T13:51:08.845Z · LW(p) · GW(p)
That's a good point. If a "skewed distribution" exists, one might use this point as an explanation. I would be really interested in knowing whether such a "skewed distribution" exists. However, how would one go about constructing such a distribution? Which characteristics of drivers should one consider? Extrapolating to other avenues like teaching, construction work, sales etc. can one define variables to measure such a "distribution" of competence?
Replies from: Measure↑ comment by Measure · 2021-01-05T21:42:44.747Z · LW(p) · GW(p)
To be clear, my point was that perceived driver skill has a skewed distribution because of the upper cap on what can be demonstrated and observed. (There is also a lower cap due to the requirement to pass a test to get a license, but the upper cap is much more restrictive.)
In situations where the lower cap is more restrictive (high barriers to entry and/or lots of opportunity to display proficiency) there should be a "below-average" effect. Perhaps this is a contributing factor to Impostor Syndrome?
comment by lejuletre · 2020-12-26T20:37:14.842Z · LW(p) · GW(p)
This is a very good post! I've found David MacIver to be a very effective translator for emotional processing, if that's something you're interested in.
comment by IrenicTruth · 2021-01-02T20:38:33.644Z · LW(p) · GW(p)
The concept of a translator between conceptual frameworks reminds me of the narrator of Blindsight - a man who had special cybernetic enhancements to allow him to do this type of translation.
comment by Kaj_Sotala · 2021-01-02T11:41:04.380Z · LW(p) · GW(p)
I would guess that this sort of reasoning happens a lot. In concrete terms:
- A person (call her Alice) forms a heuristic — “I am good at X” — where X isn’t perfectly defined. (“I am good at real-world reasoning”; “I am good at driving”; “I am a good math teacher”.) She forms it because she’s good at X on a particular axis she cares about (“I am good at statistical problem solving”; “I drive safely”; “My algebraic geometry classes consistently get great reviews”).
Relevant Scott Alexander (again):
5. You know what you know, but you don’t know what you don’t know. Suppose each doctor makes errors at the same rate, but about different things. I will often catch other doctors’ errors. But by definition I don’t notice my own errors; if I did, I would stop making them! By “errors” I don’t mean stupid mistakes like writing the wrong date on a prescription, I mean fundamentally misunderstanding how to use a certain treatment or address a certain disease. Every doctor has studied some topics in more or less depth than others. When I’ve studied a topic in depth, it’s obvious to me where the average doctor is doing things slightly sub-optimally out of ignorance. But the topics I haven’t studied in depth, I assume I’m doing everything basically okay. If you go through your life constantly noticing places where other doctors are wrong, it’s easy to think you’re better than them. [...]
7. You do a good job satisfying your own values. [Every doctor] wants to make people healthy and save lives, but there are other values that differ between practitioners. How much do you care about pain control? How much do you worry about addiction and misuse? How hard do you try to avoid polypharmacy? How do you balance patient autonomy with making sure they get the right treatment? How do you balance harms and benefits of a treatment that helps the patient’s annoying symptom today but raises heart attack risk 2% in twenty years? All of these trade off against each other: someone who tries too hard to minimize use of addictive drugs may have a harder time controlling their patients’ pain. Someone who cares a lot about patient autonomy might have a harder time keeping their medication load reasonable. If you make the set of tradeoffs that feel right to you, your patients will do better on the metrics you care about than other doctors’ patients (they’ll do better on the metrics the other doctors care about, but worse on yours). Your patients doing better on the metrics you care about feels a lot like you being a better doctor.
comment by Portia (Making_Philosophy_Better) · 2023-04-21T11:27:05.571Z · LW(p) · GW(p)
This is how I have always perceived my job.
I work on consciousness, a topic where interdisciplinarity is crucial to progress: you cannot properly understand this phenomenon without precisely identifying the resulting experience of interest itself (philosophy of mind, phenomenology), analysing the brain that brings it forth (neurobiology), the behavioural function this serves (psychology, ethology) and the evolutionary context it developed in, using reference models across a wide reference span (psychiatry, neurorehabilitation, animal minds) to identify what is arbitrary and what is necessary, and to spot the workings from bugs occurring, rebuilding what you have observed via coding to ensure you have understood it (computational neuroscience, machine learning, where suddenly you also transition from academia to industry) and then using math and theoretical physics to make the result precise in ways our language is incapable of doing (theory of machine learning, mathematical theories of consciousness). Yet the span of disciplines makes working together really difficult.
There isn't just different terminology; often, they use the same terms but mean something related, but different (the term "consciousness" itself can refer to several related, but crucially distinct phenomena - being awake rather than in a coma, phenomenal consciousness, or access consciousness; so three people who say they are looking for the function of or neural correlates of consciousness can mean completely different aspects of it; and the term "recurrent network" means something crucial and related in both machine learning and neuroscience, but not the same thing, etc.). In other scenarios, they end up talking about/discovering/analysing the same phenomena, but use different terms and do not even realise, to a degree where they replicate a finding decades after it has already been disproven in another. (E.g. philosophers developed and then discarded epiphenomenalism, and then the same thing was developed in biology later, with the biological mindset making them unable to spot the - for philosophers - obvious bug.) You need to understand the content already, and read in quite deeply, to realise they are hitting on the same thing - it will not be apparent from titles or keywords, which is why the researchers involved don't find each other in their literature search.
Just getting them together is already tricky. The researchers in question congregate in different places, and use different mediums (separate journals and conferences) and styles, many of which are not consciously reflected or pose significant barriers (I know tech minded folks who will refuse to read word files rather than LaTeX files, even though many journals outside of natural science and IT refuse to accept LaTeX files; or not engage in anything without math in it, because they cannot imagine another way of saying something precise enough to be meaningful; and philosophers who will find even the most approachable formula intimidating, while simultaneously littering the text with jargon; and then empirically minded folks who feel both kinds of work are so abstract as to make them meaningless) and different standards and focusses. There is also a lot of existing bad blood, often because the different fields felt underappreciated and attacked the contributions by other fields, making everyone feel disrespected and angry. They do very different status signalling, which is mutually distracting and distancing. It is socially delicate.
A big problem is that each of them is an expert in their field, but incompetent in other fields, which makes it extremely hard to recognise each others competence. And this makes it super hard to get them to listen to each other. A philosopher will look at a computational neuroscience paper and go, well, I don't get the programming, the data analysis or the formulas, I'd need to really put in a lot of work to understand this, is it worth it? Wait, here, the dude is working on something I understand well, and in plain text! I will read that, and if it is good, then I will work through the rest. - And of course, this part is garbage. The same scientist who was so precise in his calculations will confuse multiple philosophical terms, equate things that are different, commit a logical fallacy, ignore an argument that is very well known in philosophy... and so the philosopher goes, the part that I can judge is terrible, why should I learn to understand the part that I currently cannot judge and that is really hard to follow? So the philosopher ends up dismissing all the other stuff, despite the fact that that stuff would have been gold. (And the same happens in reverse: the philosophy paper will have either no math, or terrible formatted math that has an obvious error in it, so the actual argument is never considered by the angered mathematician.) And the fact that if they got together, and understood each other, and fixed each others errors, the improvement in the paper would be drastic.
But if you manage to translate to them, show them how these findings are meaningful to them, justified to them, how they matter, the amount of low-hanging fruit is mind-boggling. It is like walking back and forth between several groups of researchers all studying the same damn thing, but barely talking to each other, and going... wait, you know the other group solved this a decade ago, right? Or; goddamn, the other group will be so happy to hear about this finding, they are totally stuck on it! You save so much time. And ideally, together, you understand a thing neither would have understood alone.
It is incredibly rewarding and wonderful work. Challenging, but allowing leaps of connection and productivity. And wonderfully varied. Seeing a phenomenon through such different lenses makes a huge difference in understanding for me. It is also a kind of work where my neurodivergence finally works in my favour. It helps to analytically analyse social structures. It helps that my brain makes more connections between things than regular brains, at the cost of a sole focus on one thing.
comment by Kaj_Sotala · 2021-01-02T11:44:15.344Z · LW(p) · GW(p)
I would guess that this sort of reasoning happens a lot. In concrete terms:
- A person (call her Alice) forms a heuristic — “I am good at X” — where X isn’t perfectly defined. (“I am good at real-world reasoning”; “I am good at driving”; “I am a good math teacher”.) She forms it because she’s good at X on a particular axis she cares about (“I am good at statistical problem solving”; “I drive safely”; “My algebraic geometry classes consistently get great reviews”).
Here's a mistake which I've sometimes committed and gotten defensive as a result, and which I've seen make other people defensive when they've committed the same mistake.
Take some vaguely defined, multidimensional thing that people could do or not do. In my case it was something like "trying to understand other people".
Now there are different ways in which you can try to understand other people. For me, if someone opened up and told me of their experiences, I would put a lot of effort into really trying to understand their perspective, to try to understand how they thought and why they felt that way.
At the same time, I thought that everyone was so unique that there wasn't much point in trying to understand them by any other way than hearing them explain their experience. So I wouldn't really, for example, try to make guesses about people based on what they seemed to have in common with other people I knew.
Now someone comes and happens to mention that I "don't seem to try to understand other people".
I get upset and defensive because I totally do, this person hasn't understood me at all!
And in one sense, I'm right - it's true that there's a dimension of "trying to understand other people" that I've put a lot of effort into, in which I've probably invested more than other people have.
And in another sense, the other person is right - while I was good at one dimension of "trying to understand other people", I was severely underinvested in others. And I had not really even properly acknowledged that "trying to understand other people" had other important dimensions too, because I was justifiably proud of my investment in one of them.
But from the point of view of someone who had invested in those other dimensions, they could see the aspects in which I was deficient compared to them, or maybe even compared to the median person. (To some extent I thought that my underinvestment in those other dimensions was virtuous, because I was "not making assumptions about people", which I'd been told was good.) And this underinvestment showed in how I acted.
So the mistake is that if there's a vaguely defined, multidimensional skill and you are strongly invested in one of its dimensions, you might not realize that you are deficient in the others. And if someone says that you are not good at it, you might understandably get defensive and upset, because you can only think of the evidence which says you're good at it... while not even realizing the aspects that you're missing out on, which are obvious to the person who is better at them.
Now one could say that the person giving this feedback should be more precise and not make vague, broad statements like "you don't seem to try to understand other people". Rather they should make some more specific statement like "you don't seem to try to make guesses about other people based on how they compare to other people you know".
And sure, this could be better. But communication is hard; and often the other person doesn't know the exact mistake that you are making. They can't see exactly what is happening in your mind: they can only see how you behave. And they see you behaving in a way which, to them, looks like you are not trying to understand other people. (And it's even possible that they are deficient in the dimension that you are good at, so it doesn't even occur to them that "trying to understand other people" could mean anything else than what it means to them.)
So they express it in the way that it looks to them, because before you get into a precise discussion about what exactly each of you means by that term, that's the only way in which they can get their impression across.
comment by DirectedEvolution (AllAmericanBreakfast) · 2020-12-27T22:20:29.557Z · LW(p) · GW(p)
I notice that there seem to be two points to this post:
- If we define X differently for each person, then everybody can be "above average at X."
- Translators between people's different ways of expressing themselves are an important social role.
One question I wanted to clear up: is the translator's role primarily to keep people who actually agree from getting confused over linguistic and cultural differences? Or is it to allow people who have different and contradictory predictive models, as well as different ways of expressing those models, to share and reconcile their models? Both?
Also, I couldn't quite follow how point (1) relates to point (2). It seems like the key disagreement between Shor and Constance is not over who they think is generally better at real-world reasoning. It was over what should be done in concrete political decisions in Obama's campaign. The resolution didn't come from Shor and Constance realizing that they were just competent at two different aspects of "real-world reasoning," right? It came from Shor realizing that Constance was usually correct in predicting the future, and that he was ignoring her because she didn't speak his language.
This post also made me think about the ideological Turing test, which strikes me as kind of a crappy bad-faith perversion of translation. Instead of attempting to get past linguistic barriers to share and reconcile predictive models to perform useful work, the participants have no meaningful shared task and instead want to set up a formal display of the emptiness of their respective rhetorics.
Replies from: UnexpectedValues↑ comment by Eric Neyman (UnexpectedValues) · 2020-12-27T23:18:33.495Z · LW(p) · GW(p)
Yeah, I agree that the post isn't quite sequential. Most of Section II isn't necessary for any of the translator stuff -- it's just that I thought it was an interesting possible explanation of "alike minds think great" bias. (This somewhat disconnected logical structure was a hangup I had about the post; I was considering publishing it as two separate posts but decided not to.)
But, what I was trying to say about Shor and Constance and their need for a translator is: regardless of whether Shor underestimated Constance and vice versa because of this bias, they weren't in a position to understand each other's arguments. A translator's job is to make them understand each other (convert their thoughts into a language that's easily understandable by the other). This allows for reconciliation, because instead of Shor seeing his own argument and "black box Constance belief which I should update on even though I don't understand it", he sees his own argument and "Constance argument translated into Shor language", which he now has a much better idea what to do with. (And likewise symmetrically for Constance.)
Replies from: AllAmericanBreakfast↑ comment by DirectedEvolution (AllAmericanBreakfast) · 2020-12-28T04:27:18.911Z · LW(p) · GW(p)
That message seems in step with the theme of the need to better communicate science, anxieties about basing decisions on black-box AI algorithms, and controversy over how much to take expert opinion on faith.
You know, with Scott’s posts, I often get the impression not that he’s translating a thoughtful model in a language I don’t understand, but that he’s modeling social dynamics that the participants themselves don’t understand.
Take the “right is the new left” post. I really doubt that anyone participating in fashion has anything like that model in mind. Instead, a designer has the demands of a specific segment of society in mind, and tailors their product to suit. They’re not analyzing macro trends to decide how to do their shirts. They have a way of putting out clothes that works for their corner of the industry, in terms of how much it changes year to year, what other shops they look to to.
Even if the macro trends in fashion line up perfectly with Scott’s mode, I don’t think anybody in fashion has the equivalent in their head and is thoughtfully using it to base their decision.
By contrast, I think that Shor and Constance probably both do have models in their head, and that Shor’s mature take on Constance’s model does reflect her thought process.
So I’d distinguish between a translator like Shor and someone like Scott, who’s an analyst but not necessarily a translator.
comment by [deleted] · 2021-01-02T23:46:53.370Z · LW(p) · GW(p)
Great work.
I wonder just how far this concept can be stretched. Is focusing a translation from the part of you that thinks in feelings to the part of you that thinks in words? If you're translating some philosophical idea into math [LW · GW], are you just translating from the language of one culture to the language of another?
And if so, it strikes me that some languages are more effective than others. Constance may have had better ideas, but if Shor knew the same stuff as Constance (in his own language) perhaps he would have done better. Shor's language seems to be more expressive, precise and transferable.
So:
- In a given context, which language is "best"?
- In a given context, which languages have the best ideas/data?
- Where might we find large opportunities for arbitrage?
For example, I think we should be translating spiritual ideas into the language of cognitive science and/or economics. Any others?
comment by digital_carver · 2021-01-03T11:38:38.284Z · LW(p) · GW(p)
Related: Career advice from Scott Adams (Dilbert's creator) suggests becoming "very good (top 25%) at two or more things." (He even goes on to suggest: "At least one of the skills in your mixture should involve communication, either written or verbal. ")
Being a translator is often a natural outcome of this; when you have two or more mental spaces to pick ideas and metaphors from, it becomes easier to describe complex things in one field without resorting to jargon, using the language of a different field. Trying to be a translator can also be useful to clarify your own understanding, for this reason - you can't hide behind jargon or "common knowledge" beliefs any more, and so have to go through, clarify, and perhaps reconstruct your mental models for what you're trying to translate.
comment by Jalex Stark (jalex-stark-1) · 2020-12-29T11:39:04.978Z · LW(p) · GW(p)
You talk mostly about community-level information gain. I'd like to add that the act of translation is a good way for an individual to generate new insights. In theorem-proving academia, there's a lot of juice to get out of shuttling information back and forth between people in math, physics, CS, and econ departments.
comment by Bunthut · 2020-12-28T11:13:51.800Z · LW(p) · GW(p)
I'm not sure if "translation" is a good word for what youre talking about. For example it's not clear what a Shor-to-Constance translation would look like. You can transmit the results of statistical analysis to non-technical people, but the sharing of results wasn't the problem here. The Constance-to-Shor translator described Constances reasons in such a way that Shor can process them, and what could an inverse of this be? Constances beliefs are based on practical experience, and Shor simply hasn't had that, whereas Constance did get "data" in a broad sense. Now we could invent anecdotes that would make Constance conclude the same thing as the analysis, but we can do that no matter who is actually right.
"The same idea" can also bring very different capabilities depending on the way of thinking it is used in. As we learn formal thinking, we generally also incorporate some of its results into more analogical styles. This develops mathematical intuition, which lets us navigate the formal system vastly faster than brute force. On the other hand, it is only when our everyday heuristic that getting more money is good is embedded into more systematic theories that we can think its strange that useless metal is so valuable, realize that money needed to have an origen, and that gold-mining isn't productive (caveats for technical uses apply now), etc.
"Translations" in your sense also require not only familiarity with the source and target style, but often also significant work thinking in those styles. Expressing something analytically can be hard even if you are good at analytic thinking. Finding cruxes is something that you need to sit down and do for a while. I've seen a few examples where the way an analytic expression is found is that someone develops it for reasons internal to analysis, and only afterward is it realized that this could be the reality that drove intuition xyz. In the other direction, "How to actually act on the things you claim to believe" has been a significant and constant topic of this site.
Replies from: Nonecomment by Dagon · 2020-12-26T23:25:46.977Z · LW(p) · GW(p)
Useful insights. I think you should expand on the types of measures that support this phenomenon (multiple distinct valid rankings). I think the main criteria you don't mention is multidimensional weighting. "real-world reasoning", "teaching", "learning", "driving", and "translating" are all ambiguous in the same way: they comprise many distinct sub-activities, and how they combine (often non-linearly) is the point of disagreement about ranking on a single aggregate dimension.
Replies from: AllAmericanBreakfast↑ comment by DirectedEvolution (AllAmericanBreakfast) · 2020-12-27T16:37:34.432Z · LW(p) · GW(p)
Your comment made me consider that most experts don’t address the question “who’s better at X fuzzy task, you or your colleague?”
They might dodge the question give face to the other person out of professional diplomacy, or add enough nuance to say “I’m good at X1, they’re good at X2.”
Maybe we’re most likely to see people arguing over “who’s better at X?”, while ignoring or fighting against nuance, in perceived zero-sum signaling contests.
comment by CronoDAS · 2021-01-03T02:51:04.689Z · LW(p) · GW(p)
It seems like David Shor tried to do the same thing to politics that Billy Beane's team successfully did to baseball - replace intuition and qualitative analysis with statistics and careful quantitative analysis - but he ended up doing worse than the existing experts, rather than better. Why was that?
Replies from: UnexpectedValues↑ comment by Eric Neyman (UnexpectedValues) · 2021-01-03T06:27:14.382Z · LW(p) · GW(p)
Two things, I'm guessing. First, there's the fact that in baseball you get thousands of data points a year. In presidential politics, you get a data point every four years. If you broaden your scope to national and state legislative elections (which wasn't Shor's focus at the time), in some sense you get thousands per election cycle, but it's more like hundreds because most races are foregone conclusions. (That said, that's enough data to draw some robust conclusions, such as that moderate candidates are more electable. On the other hand, it's not clear how well those conclusions would translate to high-profile races.)
Second, electoral politics is probably a way harder thing to model. There are many more variables at play, things shift rapidly from year to year, etc. Meanwhile, baseball is a game whose rules of play -- allowable actions, etc. -- are simple enough to write down. Strategy shifts over the years, but not nearly as much as in politics. (I say that without having much knowledge of baseball, so I could be... um, off base... here.)
comment by Garrett Baker (D0TheMath) · 2020-12-27T17:22:34.072Z · LW(p) · GW(p)
I’ve been vaguely grasping at this concept—that I give too little credence to people who think differently from me—and this was a great crystallization.
comment by Neel Nanda (neel-nanda-1) · 2020-12-27T15:39:51.059Z · LW(p) · GW(p)
I really enjoyed this post, and it helped put a vague intuition into words, thanks! Strongly upvoted
comment by Martin Čelko (martin-celko) · 2023-02-13T23:14:29.971Z · LW(p) · GW(p)
When driving good drivers are invisible to us.
Bad drivers are seemingly everywhere.
:D
If people judge what they judge is their own knowledge they are familiar with, against what they see as most critical failing of people around.
People who are wrong are alien to us.
People who are right and agree with us don't make us emotional.
SO they are not that obvious.
comment by colonelDensity · 2021-01-04T21:44:22.560Z · LW(p) · GW(p)
I really liked the post. I just wanted to add a reference that could be fun to read through. Some economists showed that overconfidence is logically totally reasonable even with so called rational actors.
https://www2.um.edu.uy/dubraj/documentos/apparentfinal.pdf
I haven't read this paper in awhile, however, the gist is basically, if you receive positive signals on your quality, Bayesian updating mechanically makes your estimates "over-confident".
comment by dominicq · 2021-01-03T18:51:54.183Z · LW(p) · GW(p)
This is no surprise — as I mentioned, translators are few and far between — but this example goes to show how useful a translator can be.
In addition to that, they can be explicitly unwanted or feel unwanted. I think that this is partially because translation is often done by people who argue for moderation to give off an air of wisdom which isn't there.
But another, maybe more significant part, is the fact that even good translators (like Scott Alexander) have limited power. Not everyone wants to read Scott Alexander-like bloggers, and not everyone wants a competing perspective. That leaves you with the option to stretch your translation, but stretch it too far, and you get to a point where you just have a useless analogy to something your audience already understands. Try to be true to the original worldview and nobody listens you unless they exceed some level of openness.
Great post.
comment by [deleted] · 2022-09-14T19:41:12.839Z · LW(p) · GW(p)
The tails come apart.