Confidence Levels in Forecasts and Psychological Surveys

post by Fer32dwt34r3dfsz (rodeo_flagellum) · 2022-03-31T02:54:45.364Z · LW · GW · No comments

This is a question post.

Contents

  Could measures of overconfidence or underconfidence in judgmental forecasting be used to control for the effects of psychological survey participants gravitating away from or towards, respectively, the "neutral" option in surveys? 
None
  Answers
    7 Unnamed
None
No comments

Epistemic Status: I am very uncertain about this topic, but think that, if anything, I may simply be mislabeling or misunderstanding some real signal.

I imagine that there may be aspects of a person's particular tendencies in judgmental forecasting that reveal something about their personality, such as their OCEAN (Big Five) measures or their Moral Foundations. 

The following sequence of thoughts - "My life experiences indicate that I may be more conscientious than my OCEAN results suggest", "I remember selecting neutral for many answers that were likely related to conscientiousness", "One reason for why I selected neutral so often might be that I am underconfident about aspects of my personality, provided I have a track record of being underconfident, especially in judgmental forecasting" - led me to the question:

Could measures of overconfidence or underconfidence in judgmental forecasting be used to control for the effects of psychological survey participants gravitating away from or towards, respectively, the "neutral" option in surveys? 

Relatedly, one might ask "Is over- or under- selecting the neutral option in psychological surveys an actual phenomenon?", "What occurs when there is no neutral option on a psychological survey versus when there is?", "How does under- or over- confidence in judgmental forecasting relate to under- or over- confidence in other facets of people's lives? 

More generally, I suppose this intrigue might fall into the wider category of "reflections of cognitive biases or personality present in people's forecasts and forecasting tendencies". I would be content if people shared with me their thoughts on my above questions, or if people shared any resources they think would point me in a good learning direction (e.g., recommend I look into some line of research or work adjacent to these ideas). Thank you.

Answers

answer by Unnamed · 2022-03-31T04:37:51.978Z · LW(p) · GW(p)

Response bias or "response style" is the keyword for phenomena like these. "Extreme responding" or "extreme response style" are terms for gravitating away from the neutral option.

I don't know if there's research on whether this is correlated with overconfidence.

↑ comment by Viliam · 2022-04-01T19:17:02.440Z · LW(p) · GW(p)

Thanks for the link! I was aware that this style exists, but the correlates are interesting. Seems that the "extreme/mild" style is correlated with:

  • culture (Middle East, Latin America: extreme; East Asia, Western Europe: mild);
  • intelligence and education (stupid, uneducated: extreme; smart, educated: mild);
  • but also the wording of the question.

The intelligence/education link seems obvious in hindsight, but that is an illusion because if someone told me the opposite conclusion, I would have a good "explanation" for that, too.

There is probably some relation to the Dunning-Kruger effect, only instead of abilities here we have personality traits. In D-K, most people seem to have the mild response; believe themselves to be somewhat more average than they really are.

*

I am generalizing from one example, but I would assume that most people "feel normal" from inside. Like, if you are X, relative to the population average, it sometimes doesn't feel like you are X, it just feels like too many people are non-X. This would create a mildness bias.

For example... I am highly intelligent, but I don't actually feel so, because for every problem I can solve, there are ten problems I can't solve, and hundred problems where I don't even understand the fucking question.

Apparently, some people are the opposite of this. If they are X, they feel "super X". Actually, that kinda reminds me of some people.

*

I could even imagine an opposite bias, where people who are X feel like they are "anti-X". Like, if you ask a paranoid person whether they are careful enough, it might remind them of many security precautions they didn't take. Mindkilled people would probably consider themselves more objective than average; scrupulous people more sinful, etc.

No comments

Comments sorted by top scores.