There is no No Evidence

post by MalcolmOcean (malcolmocean) · 2021-05-19T02:44:49.462Z · LW · GW · 15 comments

Contents

  There is no No Evidence
  Exactly what evidence isn't there?
  Covid: the good and the ugly
None
15 comments

Zvi recently coined [LW · GW], and has now written up [LW · GW] this Law of No Evidence:

Law of No Evidence: Any claim that there is “no evidence” of something is evidence of bullshit.

Considered next to Eliezer's Absence of Evidence Is Evidence of Absence [LW · GW], it might seem like a contradiction, but as far as I can tell, it actually follows directly.

If we treat the "is" in Absence of Evidence is Evidence of Absence as an "implies" (which it seems to me to be) and then apply modus tollens to it, we get "if you don't have evidence of absence, you don't have absence of evidence" and it is precisely this bullshit that Zvi is calling. If you have evidence of absence, say so.

The Very Serious covid spokespeople spouting bullshit like "no evidence of human-to-human transmission" are doing a sort of naive equivocation version of "absence of evidence is evidence of absence", by saying "no evidence for" as if that implies "evidence against" when in fact they're trying to support some agenda or worldview when there's not very much clear evidence in any direction and someone could just as easily say "there's no evidence there isn't human-to-human transmission". At least, that's in the best case. Sometimes the evidence does favor a particular direction, they just don't like it and don't want to count it. Desperately clinging to priors?

Hm.

So Zvi's Law of No Evidence can be seen as taking Absence of Evidence is Evidence of Absence a step further and instead saying "if your absence of evidence is real, then it'll actually be evidence of absence, in which case call it that. otherwise shut up." But there's still a point to be made about a sense in which "absence of evidence" is its own thing—it's just different from no evidence. The important piece is that "no evidence" is bullshit, but "we didn't see this particular thing we would more expect to see if X were true than if X weren't true" is a vital component of successful reasoning about the world. It's the very basis of Bayes.

There is no No Evidence

If you observe {not[evidence of X]} then it makes sense to update towards {not[X]}, but the speech act of claiming "There is no evidence of X" only occurs when there exists some apparent evidence that someone wants to claim doesn't count as evidence. This is a frame battle disguised as argument, since there's some evidence for almost any hypothesis you can possibly imagine.

• There's evidence for aliens on Mars. Not much of it, and on net there's probably a better explanation.
• There's evidence for the Earth being flat. It's not good evidence, but you might be surprised how little of it you can refute yourself.
• There's evidence for UFOs. Probably more of it in the last year or so (with military declassifyings) than in the decade before that. I've had strong priors that this sort of thing is mostly perceptual illusions, fakery, or other psychosocial dynamics, but I would be bullshitting if I let the strength of my priors allow me to claim there was "no evidence" that our planet has had actual unexplainably-flying objects. And maybe I'm confused!
• The mere fact of being able to imagine something is ever-so-slightly more likely in universes where it's true, compared to something so bizarre that you couldn't even imagine it. Your best friend is probably more likely to be secretly a dragon than to be secretly some other strange fantasy monster that hasn't ever even been conceived of by anyone.

So no, there is never no evidence.

I think there's one situation in which you can almost usefully talk about "no evidence", which is essentially "I've seen no evidence that this question is worth considering at all (given my priors) so I haven't bothered looking into it." But that's a subjective statement—you're not asserting "there is no evidence", just perhaps that you have a high bar on what would count as meaningful evidence. And even so, the fact that you're somehow talking about it at all is now at least a tiny bit of evidence that the question is worth considering, so oops.

(If you're confused about the difference between {no evidence} and "no evidence", you may want to learn more about Simulacra levels.)

Exactly what evidence isn't there?

If you want to actually talk about the "absence of evidence" that you're using as evidence of absence, you have to actually refer to the specific evidence that is missing, that you'd expect to see if the proposition were true. So you might say:

This alleged crime would have taken place in a crowded park, so the absence of any photos, videos, or eyewitness accounts of it, despite our searching for those, is evidence that the story was completely made up, and thus we have evidence against Mortimer Q. Snodgrass being guilty of any crime.

It is the absence of some specific type of evidence that is a valid argumentative tool, because it is evidence against some world in which you'd expect that evidence. This is not "no evidence" in general.

From xkcd:

The hover text adds "Not to be confused with 'making money selling this stuff to OTHER people who think it works'". There's not no evidence for these phenomena—people thinking it works is some evidence. But there's some very important missing evidence that is much stronger in the "doesn't work" direction than the "people thinking it works" is in the other direction.

There is evidence for and evidence against but there is no "no evidence".

Covid: the good and the ugly

Most official covid takes has been spectacularly bad on this front, which is of course what prompted Zvi to coin the law now, even though Overcoming Bias had Doctor, There are Two Kinds of "No Evidence" back in 2008. Here's one egregious one:

But progress is possible: a conversation I and others had with a epidemiologist on twitter managed to prompt him to actually revise his statements to avoid any "no evidence" constructions, and it turned out pretty well. Instead, he notes an absence of a specific kind of evidence (cases detected as genetically matching certain strains) and then contextualizes that to conclude that despite that lack of evidence-for-those-variants-in-the-USA, we can't actually confidently conclude said variants aren't in the USA, by offering another explanation for the absence-of-evidence other than absence-of-intended-to-be-measured-phenomenon.

There's much more work yet to be done to spread the word that "no evidence" is bullshit. This post is intended to be one place to point people for that purpose, but honestly it's written using pretty ingroupy jargony language since it started as a comment critiquing this post [LW · GW] that made a case in favor of Zvi's Law of No Evidence but a case that I thought was confused.

I would encourage someone to write up a more generally accessible article on this!

Update: there's now Zvi's canonical Law of No Evidence [LW · GW] post and Scott's The Phrase “No Evidence” Is a Red Flag for Bad Science Communication.

15 comments

Comments sorted by top scores.