Link: "When Science Goes Psychic"

One lesson of the common misuse of statistics is to not "defy the data" until you're sure what it says.

Here's an important reply cited in the other threads:
http://www.ruudwetzels.com//articles/Wagenmakersetal_subm.pdf

Does psi exist? In a recent article, Dr. Bem conducted nine studies with over a thousand participants in an attempt to demonstrate that future events retroactively affect people’s responses. Here we discuss several limitations of Bem’s experiments on psi; in particular, we show that the data analysis was partly exploratory, and that one-sided p-values may overstate the statistical evidence against the null hypothesis. We reanalyze Bem’s data using a default Bayesian t-test and show that the evidence for psi is weak to nonexistent.

Hofstadter's response is quite to the point.

I also liked "No Sacred Mantle" - I believe we should promote more widely some basic techniques for critical reading of science-related claims. I recently pushed out an essay on "Fact and folklore in software engineering" that went ridiculously viral thanks to HN, even though I'll be the first to admit it's not my clearest writing.

I think there's a lot of pent-up demand for things like "how to read a popular article reporting on a science fact", "how to read a scientific paper in a field you don't know", etc. No surprise there - after all, in terms of built-in preferences rationality is primarily about defending yourself from "facts" that you shouldn't accept.

I can't believe Hofstadter (or anyone, really) is arguing that Bem's paper should not have been published. The paper was, presumably, published because peer-reviewers couldn't find substantial flaws in its methodology. This speaks more about the nature of standard statistical practices in psychology, or at least about the peer-review practices at the journal in which Bem's paper was published, which is useful information in either case.

N.B. It's Daryl Bem again.

I have a bizarre nitpick.

Hofstadter writes:

We all have heard it claimed that 13 is an "unlucky number." [...]

There would be no plausible scientific mechanism to make any such things happen, unless there were an extremely bizarre kind of intelligence ruling our universe -- and this would go so profoundly against the laws of physics as we know them that our entire scientific worldview would be toppled if such a pattern involving the number 13 were true. [...]

The point is that such an article would not only be about the luckiness or unluckiness of the number 13, but it would also necessarily (if only implicitly) be about the entire nature of the universe we live in, since if 13 really were provably unlucky in any sense at all, then all bets would be off about the laws of physics, for the laws of physics are simply not compatible with such a finding.

Why add the detail of incompatibility with the laws of physics? The predictions made from laws of physics alone (in 13-ish situations), and so the model conferred by the laws of physics, would err if there was this additional bizarre factor, but among all this confusion one can't make conclusive statements about the laws of physics. The strangeness could well be implemented within physics as we know it.