Double-Dipping in Dunning--Kruger

I like and endorse the general theme of this post, but have some issues with the details.

The takeaway is this: if you're the kind of person who worries about statistics and Dunning--Kruger in the first place, you're already way above average and clearly have the necessary meta-cognition to not fall victim to such things.

I feel like this is good motivation but bad world-modelling. Two important ways in which it fails:

• Social interactions. You gave the example of people not really knowing how funny they are. I don't think worrying about statistics in general helps with this, because this might just not be the type of thing you've considered as a failure mode, and also because it's very difficult to substitute deliberate analysis for buggy social intuitions.
• People being bad at philosophy. There are very many smart people who confidently make ridiculous arguments - people smart enough to understand Dunning-Kruger, but who either think they're an exception, or else pay lip service to it and then don't actually process any change in beliefs.

The world is being run by people who are too incompetent to know it; people who are only in power because they're the ones who showed up, and because showing up is most of the battle.

I dislike lines of argument which point at people on top of a pile of utility and call them incompetent. I think it is plausibly very difficult to get to the top of the society, but that the skills required are things which are really difficult to measure or even understand properly, like "hustle" or "ambition" or "social skills" or "pays less attention to local incentive gradients" or "has no wasted mental motion in between deciding that x is a good idea and deciding to do x".

From now on, unless you have evidence that you're particularly bad at something, I want you to assume that you're 15 percentile points higher than you would otherwise estimate.

Nit: I prefer using standard deviations instead of percentile points when talking about high-level performance, because it better allows us to separate people with excellent skill from people with amazing skill. Also because "assume that you're 15 percentile points higher" leaves a lot of people above 100%.

Participants scoring in the bottom quartile on our humor test (...) overestimated their percentile ranking

A less well-known finding of Dunning--Kruger is that the best performers will systematically underestimate how good they are, by about 15 percentile points.

Isn't this exactly what you'd expect if people were good bayesians receiving scarce evidence? Everyone starts out with assuming that they're in the middle, and as they find something easy or hard, they gradually update away from their prior. If they don't have good information about how good other people are, they won't update too much.

If you then look at the extremes, the very best and the very worst people, of course you're going to see that they should extremify their beliefs. But if everyone followed that advice, you'd ruin the accuracy of the people more towards the middle, since they haven't received enough evidence to distinguish themselves from the extremes.

(Similarly, I've heard that people often overestimate their ability on easy tasks and underestimate their ability on difficult tasks, which is exactly what you'd expect if they had good epistemics but limited evidence. If task performance is a function of task difficulty and talent for a task, and the only things you can observe is your performance, then believing that you're good at tasks you do well at and bad at tasks you fail at is the correct thing to do. As a consequence, saying that people overestimate their driving ability doesn't tell you that much about the quality of their epistemics, in isolation, because they might be following a strategy that optimises performance across all tasks.)

The finding that people at the bottom overestimate their position with 46 percentile points is somewhat more extreme than this naïve model would suggest. As you say, however, it's easily explained when you take into account that your ability to judge your performance on a task is correlated with your performance on that task. Thus, the people at the bottom are just receiving noise, so on average they stick with their prior and judge that they're about average.

Of course, just because some of the evidence is consistent with people having good epistemics doesn't mean that they actually do have good epistemics. I haven't read the original paper, but it seems like people at the bottom actually thinks that they're a bit above average, which seems like a genuine failure, and I wouldn't be surprised if there are more examples of such failures which we can learn to correct. The impostor syndrome also seems like a case where people predictably fail in fixable ways (since they'd do better by estimating that they're of average ability, in their group, rather than even trying to update on evidence).

But I do think that people often are too quick to draw conclusions from looking at a specific subset of people estimating their performance on a specific task, without taking into account how well their strategy would do if they were better or worse, or were doing a different task. This post fixes some of those problems, by reminding us that everyone lowering the estimate of their performance would hurt the people at the top, but I'm not sure if it correctly takes into account how the people in the middle of the distribution would be affected.

(The counter-argument might be that people who know about Dunning-Kruger is likely to be at the top of any distribution they find themselves in, but this seems false to me. I'd expect a lot of people to know about Dunning-Kruger (though I may be in a bubble) and there are lots of tasks where ability doesn't correlate a lot with knowing about Dunning-Kruger. Perhaps humor is an example of this.)

↑ comment by Douglas_Knight · 2019-05-31T14:11:09.631Z · LW(p) · GW(p)

In other words, regression to the mean. The predictions form a line, with a positive slope. Less than 1, but only perfect predictions would have slope 1. The intercept is high, which is overconfidence. But the intercept is a statement about the whole population, not about the lowest bin.

Here are the graphs. A lot of information has been destroyed by binning them, but it doesn't sound like DK thought that information was relevant or made use of it:

The second is different. That better matches the cartoons one finds for an image search for Dunning-Kruger. But I'm not sure it matches this post. (The third could be described as yet another shape, but I'd classify it as a line with a very low positive slope.)

The fourth graph is of a more complicated intervention. It seems like it has the opposite message of this post, namely it finds that the 4th quartile is better calibrated than the 3rd.

I observe that the idea of incorrectly believing I'm bad at something doesn't disturb me much, while the idea of incorrectly believing I'm good at something is mortifying.

I smell some kind of social signaling here.

Have tried to tackle hard-seeming problems. They turned out even harder than they seemed.

I, too, have some objections, including issues with the underlying model on which this post is written.

First off, impostor syndrome, self-esteem. It's mostly an endocrine "problem", it's a body's way of adapting status negociation to the social environment. It's almost entirely determined by testosterone levels in males (and probably in females as well). Of course, winning social competitions results in a boost in T. But the conscious level evaluation of where you you rank among others, in what percentile you are at something is completely different than the underlying, system-1 self-evaluation. Of course, gathering evince that you are good at something will update your system-1 evaluation automatically (e.g. having produced something you are proud of, receiving praise and admiration that you perceive as genuine etc).

The mechanism by which this happens is fairly complex. Testosterone also has a strong anxiolytic effect, by regulating the GABA neurotransmitter, which is the main inhibitory neurotransmitter in the mammalian central nervous system. This also has a very strong impact on stress-regulation, the ability of the organism to cope with stress. The result of this is that people with higher T levels are never concerned with "how good they are", or where they rank.

> if you're the kind of person who worries about statistics and Dunning--Kruger in the first place, you're already way above average and clearly have the necessary meta-cognition to not fall victim to such things
Non sequitur. Just because you're interested in statistics doesn't automatically make you good at every other field. This reminds me of all those "Why aren't rationalist winning" posts (e.g. https://www.lesswrong.com/posts/LgavAYtzFQZKg95WC/extreme-rationality-it-s-not-that-great). Being more rational doesn't give you more domain knowledge in specific domains.

But I would also not pay too much attention to social science studies such as that, which is probing surface level effects in human populations. These things could vary significantly in different cultures, and a lower level understanding of self-esteem sort of dissolves this question. Besides, you don't know whether you should update down or down.

> the smartest people I know join Google or go work at B2B startups and are simultaneously bored in their day jobs
If you consider the risk-reward gradient, joining Google or similar high-paying jobs is a great choice in terms of maximizing status and money, even though it is a local-maxima. And these are the basic human drives. Even on HN, the HQ of people with entrepreneurial spirit, people generally recommend getting a job at google or similar bigcorp if you have the opportunity, early in your career for several reasons.

Richard Hamming's advice of tackling the biggest problem, is I think applicable only to tenured professors. To anyone else it is a recipe for disaster, i.e. taking on a huge-reward/huge-risk without having either siginificant wealth or social capital, you are just getting yourself out of the gene pool. The thing is, the gut feeling has been shaped by generations of evolution, and ended up being a good heuristic. If your gut feeling is telling you not to attempt something to risky, to try something smaller, it is because that's the natural sequence in which things are done, including by world-leading scientists. People like Einstein haven't started with General Relativity. And by starting with small things and succeding, you become more confident and attempt slightly bigger problems and so on. It's an iterative cycle.

> the least funny people are so tragically unfunny that they wouldn’t know humor if it bit them
This isn't really relevant in the grand scheme of things, but humor itself is highly dependent on the person's social status. So how funny you perceive someone to be is directly influenced by his status. And obviously, how confident he is, which again is directly related to T levels (citation needed).

> Most things just aren’t that hard, and a good percentage of the people doing those jobs are phoning it in anyways.
> The bar on competency is tragically low. And yet the world revolves.
This is somewhere where you are perfectly right. Having seen how disfunctional most organisations are, it's very surprising how most of them even work. Even places like Google, I've seen way too many incompetent people getting hired, despite the place having had a certain reputation (more so in the past).

> The world is being run by people who are too incompetent to know it; people who are only in power because they’re the ones who showed up, and because showing up is most of the battle
This is not at all how I model the world. Sure, if your self-esteem is too low to even try, then of course, nobody (other than your family) will hand the power to you on a platter. But other than that, people who end up in power don't end up there accidentally, for reasons similar to why you won't find a $20 bill in the Grand Central. (although I actually think I have seen that happening, several times).

Overall, this whole post clearly sends that motivational vibe, while also seemingly containing too many applause lights. Which is a good or a bad thing, depending on how you look at it. It could probably help some people.


==============================
My personal experience with this phenomena was related to burnout. After some structural changes to the organization (software $bigcorp), which lead to the creation of a highly politicized environment, I was exposed to a lot more stress. Gradually, after about 1 year, I just couldn't take it anymore.

I was having all sorts of symptoms by then, such as constant fatigue, concentration issues, basically I was close to non-functioning. I knew I had to get out, but decided to take up on one of the job offers I had at the time (higher pay, in a field I was very interested in). After the interview, they told me I was the most qualified engineer they ever interviewed, they would be really exited to have me there. And yet, I barely lasted a couple of days.

I was barely done setting my hardware up, that I was experiencing strong panic attacks, something that hadn't happened to me before. I had to get out for a walk, I simply couldn't sit on a chair, or the world felt like it was crashing on me. Even though I was objectively better qualified than most of my peers, I had extremely low self-esteem. I could point to hundreds of objective things that would justify my competence (projects I've done in the past, winning several significant competitive programming competitions, graduating top of my class etc). Of course, I took some time of to figure out what is wrong with me, and get myself in a functional state again. After eliminating all sorts of hypotheses, like doing sleep studies (my sleep was terrible at the time, I was fairly sure I had something like sleep apneea, although I was lean), still nothing. I was tired all day. I did a hormone panel test, and T levels came out below the chart, worse than you'd expect from a healthy 90yo. I was 23 at the time, which is around when T levels should peak. No other significant issue found. Also, anxiety was a big issue at the time.

Can you make your advice more specific? I work for Google and sometimes write LW posts about decision theory. What's the world waiting for me to do instead?

This is probably good general advice, but it's a different matter when there is evidence that points to being an actual imposter. For example, when I write novels that do not sell, or blog posts that get downvoted to oblivion, it is difficult to get honest feedback as to how I might improve my writing. The feedback I get is almost always positive, but reviews are self-selected because people rarely are motivated to review something unless they especially like it. Plus, politeness prohibits people from being harsh when you ask for feedback. For these reasons, I am more apt to trust the hard metrics and view myself as a poor writer who has managed to fool a few people. Improving myself is a far difficult task, in light of this. I guess my point is that knowing that there may be a blind spot means you can adjust for it, but it is also an opportunity to actually check it.

See, the problem is now that I've also internalized this (seemingly true) lesson, the +15% might double-boost my ass numbers.

But maybe if we accumulate enough lessons we can get increasingly close to the truth by adding these "higher order terms"?

I don't think so - the error bars do not necessarily diminish. For example:

• Ass number for drawing ability percentile: ~70%
• Dunning Kruger correction: ~50%
• Double-dip correction: ~65%

Did I do it right? I have no idea. Every step might have already been taken into account in the first asstimate. Every system-2 patch that we discover might have immediately patched system-1.

One (admittedly lazy) way out is to chuck all context-sensitive formal rules like 'add/subtract X%' and leave it entirely to system-1: play calibration games for skill-percentiles.

Please link more of your posts here. I looked through the history on your blog and there are quite a few that I think would be relevant and useful for people here. In particular, I think people would get a lot out of the posts about how to make friends. Some other other posts have titles that look interesting too but I haven't had time to read them yet.

The bar on competency is tragically low. And yet the world revolves.

You buried the lede. This entire post (and most of our effort, I believe) should be in pursuit of figuring out how to raise the sanity waterline (and/or "total competency sum") of conscious entities, not about estimating one's own competency higher.

Pre-schizophrenic induced brain damage I used to think I was 45th percentile good at something. Post brain damage I realize I was actually 100th percentile