Are you being p-hacked? Time to hack back.
post by Jacob Falkovich (Jacobian)
This is a link post for http://putanumonit.com/2016/12/18/multiplicitous
Comments sorted by top scores.
comment by cousin_it ·
2016-12-21T22:49:32.194Z · LW(p) · GW(p)
Thank you for writing that! To sum up:
Alice: I got this result with p<0.05
Bob: How many hypotheses did you test?
Bob: Then you got the result with p<0.5
This is going straight into my toolbox.
comment by Lumifer ·
2016-12-20T22:16:51.270Z · LW(p) · GW(p)
maybe WE SHOULD STOP USING FUCKING P-VALUES FOR HYPOTHESIS TESTING.
There's some... tension between this correct statement and the rest of your blog post :-/
Replies from: Jacobian
↑ comment by Jacob Falkovich (Jacobian) ·
2016-12-21T19:00:15.177Z · LW(p) · GW(p)
You're right, I added a paragraph explaining why I do this:
P-values aren’t very useful in getting close to the truth, but they’re everywhere, they’re easy to work with and they’re moderately useful for getting away from bullshit. Since the latter is our goal in this essay we’ll stick with looking at p-values for now.
Since you see p-values reported all the time and they're easy to calculate (because they're uniformly distributed), there's value in extracting whatever useful evidence there is in p-values.