Ambiguity in cognitive bias names; a refresher
post by nerfhammer · 2012-02-21T04:37:24.759Z · LW · GW · Legacy · 8 commentsContents
8 comments
This came on the nyc list, I thought I would adapt it here.
Cognitive biases have names. That's what makes them memetic. It's easier to think about something that has a name. Though I think the benefits outweigh the costs, there is also the risk of a little Albert: a concept living on after the original research has been found to be much more ambiguous than first realized.
There are many errors that are possible with respect to named ideas, and despite being studied generally scientifically, cognitive biases are no exception. There is no equivalent to cognitive biases as the Académie Française is to French.
Let's describe some. Here they are:
- different people in different fields will "discover" virtually the same bias but not be aware of each other and assign it different names. For example, see the Curse of Knowledge which I think George Loewenstein came up with vs. the Historian's Fallacy by David Hackett Fischer, presentist bias, creeping determinism, and probably many others, not all of them scientific. Sometimes researchers in seemingly closely related subfields are remarkably insular to each other.
- researchers will use one term predominantly while an offshoot will decide they don't like the name and use a different one. For example the Fundamental Attribution Error has also been called the overattribution effect, the correspondence bias, the attribution bias, and the actor-observer effect. In this case the older term still predominates, and is used in intro textbooks without asterisks. Of the naming errors this is one of the least harmful, since everyone agrees what the FAE is, some just prefer a different name for it.
- an author will decide he doesn't like the names of some biases will invent idiosyncratic names of his own. Jonathan Baron has a good textbook on cognitive bias but he uses names of his own invention half the time.
- the same term will sometimes have different polysemous meanings. For example the "Zeigarnik Effect" has been used to refer to a memory bias in having a superior recall for unfinished tasks, and the term has also been used to refer to an attentional bias in which unfinished tasks tend intrude on consciousness; almost, but not quite exactly, the same thing. The term "confirmation bias" has several different but related meanings, for example, to seek out confirming information, to notice confirming information, to ask confirming questions, etc. which are not all quite exactly the same thing. The different meanings may have completely different contexts, boundary conditions etc., leading to confusion. Furthermore some of the senses may be at least partially disproven but not necessarily others, for example, the tendency to ask confirming questions has turned out to be more complicated than once thought. You might never know from reading about the attentional Zeigarnik that there is also a memory Zeiganik effect that is conceptually somewhat different. I recall seeing even prominent researchers occasionally making mistakes of this category. Of all the naming ambiguities I think is the most dangerous.
- an offshoot of researchers may knowingly use the same term with a conflicting definition. For example "heuristic" in "Heuristics and Biases" versus "Fast and Frugal Heuristics", the latter of which was an intentional reaction to the former. In this case those involved know there is a disagreement in meaning, but those unfamiliar to the topic might be confused.[This is a point of contention which I'm willing to yield on]
- the same term may be redefined by researchers who may not aware of each other. There has been more than one paper trying to introduce a bias to call "the disconfirmation effect". But this only happens for really obscure biases.
- a bias may have different components which do not have names of their own and/or a bias may overlap partially but not completely with another bias. For instance, hindsight bias has different components one of which has some overlap with the curse of knowledge.
- the same bias term will be used as a rough category of experimental effect and also as a singular bias. For example, the term "an actor-observer bias" could refer to any difference in actors and observers, whereas "the actor-observer bias" refers to the Fundamental Attribution Error specifically; the same is true of "an" vs. "the" attribution bias, also referring to the FAE. This could confuse only those who are unfamiliar with the terminology.
- sometimes authors have tried to enforce strict, distinct meanings for the subterms "bias" vs. "effect" vs. "neglect" vs. "error" or "fallacy"; other times, perhaps more often, these terms are used only by convention. For example the conjunction fallacy vs. the conjunction error, correspondence bias vs. the fundamental attribution error, base rate neglect vs. base rate error. Sometimes the originators of a bias try to use the terminology precisely while later authors citing it aren't as careful. Sometimes even the originators of a bias do not try to choose a subterm carefully. You might suspect what permutation of a term catches on is based on whichever has a better ring to it.
8 comments
Comments sorted by top scores.
comment by Sniffnoy · 2012-02-21T11:09:19.324Z · LW(p) · GW(p)
an offshoot of researchers may knowingly use the same term with a conflicting definition. For example "heuristic" in "Heuristics and Biases" versus "Fast and Frugal Heuristics", the latter of which was an intentional reaction to the former. In this case those involved know there is a disagreement in meaning, but those unfamiliar to the topic might be confused.
Huh? How are these different? Seems to me they differ only in connotation.
Replies from: nerfhammer↑ comment by nerfhammer · 2012-02-21T14:12:02.192Z · LW(p) · GW(p)
That's a valid opinion. There is only a subtle difference really so maybe it's not the best example
Replies from: thomblake↑ comment by thomblake · 2012-02-21T14:43:28.869Z · LW(p) · GW(p)
You did not answer the question. How are they different?
Replies from: nerfhammer↑ comment by nerfhammer · 2012-02-22T06:22:13.972Z · LW(p) · GW(p)
Fast and Frugal heuristics can be descriptive (meaning human beings naturally use them at some level) or prescriptive (here are some good heuristics you can learn to use). Heuristics in Heuristics and Biases are only descriptive.
The Heuristics and Biases theorists would never suggest someone should try to "use" one of their heuristics, nor probably could you even if you tried. You could not intentionally reproduce the pattern of cognitive biases that their heuristics allegedly cause, many appear to be irretrievably outside of conscious awareness or control. For that matter, they often appear to be nearly impossible to stop using even if you wanted to.
Fast and Frugal heuristics, however, you can learn and use intentionally. The Fast and Frugal theorists generally don't suggest that it would be difficult to stop using their heuristics should you be aware of them and have the desire to. Descriptive heuristics may even be discoverable via introspection.
Heuristics in Heuristics and biases are defined as having negative side effects. There are no heuristics in H&B that aren't revealed via errors. Heuristics in H&B are presumed to be either needed by some necessary efficiency or could be an evolutionary quirk like the blind spot in your eye. Fast and Frugal heuristics do not require negative side effects and are usually not described with any. Descriptive F&F heuristics aren't evolutionary quirks. Heuristics in F&F are defined as being a helpful efficiency gain.
So they are mutually exclusive in some properties, besides the obvious that Fast and Frugal heuristics are "good" while heuristics as in Heuristics and biases are "bad".
Replies from: jake987722, thomblake↑ comment by jake987722 · 2012-02-23T01:48:48.054Z · LW(p) · GW(p)
Have to disagree with you on, well, several points here.
Heuristics in Heuristics and Biases are only descriptive. [...] Heuristics in Heuristics and biases are defined as having negative side effects.
If your claim is that heuristics are defined by H&B theorists as being explicitly not prescriptive, in the sense of never being "good" or "useful," this is simply not the case. For instance, in the opening paragraph of their seminal 1974 Science article, Kahneman & Tversky clearly state that "...people rely on a limited number of heuristic principles which reduce the complex tasks of assessing probabilities and predicting values to simpler judgmental operations. In general, these heuristics are quite useful, but sometimes they lead to severe and systematic errors." Gigerenzer et al. would not necessarily disagree with this definition (they tend to define heuristics in terms of "ignoring information" rather than "reducing complexity," although the end result is much the same), although they would almost certainly phrase it in a more optimistic way.
...nor probably could you even if you tried. You could not intentionally reproduce the pattern of cognitive biases that their heuristics allegedly cause, many appear to be irretrievably outside of conscious awareness or control.
Representativeness, one of the earliest examples of a heuristic given by the H&B program, is certainly used in a conscious and deliberate way. When asked, subjects routinely report relying on representativeness to make frequency or probability judgments, and they generally see nothing wrong or even really remarkable about this fact. Nick Epley's work also strongly suggests that people very deliberately rely on anchoring-and-adjustment strategies when making some common judgments (e.g., "When was George Washington elected president?" "Hmm, well it was obviously some time shortly after the Declaration of Independence, which was in 1776... so maybe 1786?").
Fast and Frugal heuristics, however, you can learn and use intentionally.
One can certainly learn to use any heuristic strategy, but for some heuristics proposed by the F&F camp, such as the so-called fluency heuristic (Hertwig et al., 2008), it is not at all obvious that in practice they are utilized in any intentional way, or even that subjects are aware of using them. The fluency heuristic in particular is extremely similar to the availability heuristic proposed decades earlier by Kahneman & Tversky.
Descriptive F&F heuristics aren't evolutionary quirks.
I'm not sure what you mean here. If an "evolutionary quirk" is a locally optimal solution that falls short of a global maximum, then the heuristics described by both H&B and F&F theorists are most certainly "evolutionary quirks." The claim being advanced by F&F theorists is not that the heuristics we tend to use are optimal in any sense of having maximal evolutionary adaptedness, but simply that they work just fine thanks. Note, however, that they are outperformed in simple inference tasks even by relatively simple strategies like multiple regression, and outperformed in more challenging prediction tasks by, e.g., Bayes Nets. They are decidedly not globally optimal.
...besides the obvious that Fast and Frugal heuristics are "good" while heuristics as in Heuristics and biases are "bad".
This impression is entirely due to differences in the framing and emphasis employed by the two camps. It does not represent anything like a fundamental distinction between how they each view the nature or role of heuristics in judgment and decision making.
Replies from: nerfhammer↑ comment by nerfhammer · 2012-02-23T03:47:53.068Z · LW(p) · GW(p)
Heuristics in Heuristics and Biases are only descriptive. [...] Heuristics in Heuristics and biases are defined as having negative side effects.
If your claim is that heuristics are defined by H&B theorists as being explicitly not prescriptive, in the sense of never being "good" or "useful," this is simply not the case.
No, no, that's not what I'm saying. The claim that heuristics have negative side effects does not entail a claim that negative side effects are the only characteristics they have. The 'side effect' terminology might be taken to imply that there is a main effect which is not necessarily negative.
They have always claimed that heuristics are right most of the time. But they wouldn't recommend you purposefully try to "use" them. They only propose heuristics that could theoretically explain empirically observed biases. F&F heuristics do not necessarily need to explain biases. A F&F heuristic might only explain when you get something right that you otherwise shouldn't. I'm not even sure that an F&F heuristic need explain anything empirically observed but rather could be a decision strategy that they modelled as being effective that everyone should learn (what I clumsily meant by 'prescriptive'). And they have published ways to teach use of some of their heuristics.
Representativeness, one of the earliest examples of a heuristic given by the H&B program, is certainly used in a conscious and deliberate way. When asked, subjects routinely report relying on representativeness to make frequency or probability judgments, and they generally see nothing wrong or even really remarkable about this fact.
I don't recall introspective interviews with subjects taking place in H&B research, though I may apparently be wrong about that. What I had in mind when I wrote that was that I seem to recall K & T and Gigerenzer sparring over the validity of doing that.
Except.... now that I think of it I seem to recall something like that in the really early K & T papers... maybe as I understood it, which may be obsolete, is that introspection could be useful to help generate empirical theories but could not be used to validate them whereas I seem to recall Gigerenzer arguing that they could provide validity. Maybe the camps have converged on that, or my memory continues to be faulty.
[irrelevant digression: representativeness was the absolute earliest, and by a large margin if you include "the law of small numbers" as the germ of representativeness. But if you count the law of small numbers as a heuristic and separately then it was the first.]
Nick Epley's work also strongly suggests that people very deliberately rely on anchoring-and-adjustment strategies when making some common judgments (e.g., "When was George Washington elected president?" "Hmm, well it was obviously some time shortly after the Declaration of Independence, which was in 1776... so maybe 1786?").
It implies that anchoring-and-adjustment is consciously available as a strategy at least some of the time.
When it theoretically appears in the anchoring bias ("Are there more or less than 60 nations in the UN from Africa?") it's virtually impossible to debias, suggesting it's outside of conscious control in that case.
So it does force the concession that it's not always true, though.
Fast and Frugal heuristics, however, you can learn and use intentionally.
One can certainly learn to use any heuristic strategy, but for some heuristics proposed by the F&F camp, such as the so-called fluency heuristic (Hertwig et al., 2008), it is not at all obvious that in practice they are utilized in any intentional way, or even that subjects are aware of using them. ...
Wasn't aware of that one. I haven't kept up with the literature since 2005 or so. If there are some F&F heuristics that are outside of conscious awareness and some H&B heuristics that are within awareness then conscious awareness is eliminated as a possible distinction.
There are some F&F heuristics that they argue we should use more than we already would. I'm not sure if there are any H&B heuristics for which that would be true.
Descriptive F&F heuristics aren't evolutionary quirks.
I'm not sure what you mean here. If an "evolutionary quirk" is a locally optimal solution that falls short of a global maximum...
I mean like a dead-end local maxima that we could be "stuck" in but doesn't hurt us that much. We would have better vision if we didn't all have a little blind spot. There's no reason for it being there, invertebrates that have highly developed eyes don't have it. But we're stuck with it since it goes back to the way the first vertebrates. I don't think an H&B theorist would object to the idea of evolutionary "mistakes" as an explanation whereas I think an F&F theorist very well might. Maybe that's not a very good a distinction.
I do not think that F&F theorists think that their heuristics are globally optimal, something that was globally optimal would no longer be a heuristic of any stripe.
[edit: I think I see where I was going wrong here. H&B theorists study biases that are not necessarily theoretically caused by heuristics. For instance, prospect theory isn't a heuristic. Or, framing isn't caused by any heuristic that I can think of. But it's orthogonal to their definition of what a heuristic is.]
...besides the obvious that Fast and Frugal heuristics are "good" while heuristics as in Heuristics and biases are "bad".
This impression is entirely due to differences in the framing and emphasis employed by the two camps. It does not represent anything like a fundamental distinction between how they each view the nature or role of heuristics in judgment and decision making.
I meant those as scare quotes, meaning I don't necessarily endorse them. I agree that framing and emphasis is a very large part of the difference between the camps. I'm not 100% convinced it is entirely the difference.
I think there may be still the issue that a heuristic in F&F can be something that they modelled which is not empirically used, or at least not empirically seen as much as it should be optimally, but it would be good if we could be taught to use it whereas I don't think that an H&B heuristic would ever have that set of characteristics. though perhaps you could convince me otherwise.
comment by postcardprinting · 2012-02-21T06:20:15.073Z · LW(p) · GW(p)
Yes, biases are everywhere. However, they are not that readily obvious since people assign different names to refer to them every time.