Most empirical questions are unresolveable; The good, the bad, and the appropriately under-powered

post by Davidmanheim · 2017-01-23T20:35:29.054Z · LW · GW · Legacy · 4 comments

This is a link post for https://medium.com/@davidmanheim/the-good-the-bad-and-the-appropriately-under-powered-82c335652930

Contents

4 comments

4 comments

Comments sorted by top scores.

comment by sarahconstantin · 2017-01-24T02:00:36.312Z · LW(p) · GW(p)

Yep! this is important. You can't make everything evidence-based. Some empirical questions have to be decided by heuristic, by theory, by analogy, or sometimes "shrug, flip a coin."

An important corollary is that "this isn't evidence-based" isn't always a damning insult. It's good to know what is and isn't evidence-based, but sometimes you have to make decisions without good empirical evidence.

comment by BiasedBayes · 2017-01-30T21:35:27.516Z · LW(p) · GW(p)

Thanks for the link, the title is a bit misleading though ("most empirical questions..").

Replies from: Davidmanheim
↑ comment by Davidmanheim · 2017-02-08T16:12:02.893Z · LW(p) · GW(p)

Really? I'd expect that very few questions outside of the hard sciences are resolvable given model uncertainty, the limits on running RCTs, and the limited samples available given finite population and insufficient variation regarding variables of interest.

Do you disagree?

Replies from: BiasedBayes
↑ comment by BiasedBayes · 2017-02-08T19:41:24.125Z · LW(p) · GW(p)

Then why not to state exactly that position in the article ? To answer to your question i dont if you mean every possible crazy hypothesis being in the space of possible hypothesis but disagree if you mean in terms of pragmatic usefull hypothesis being well resolved. There is not a single doubleblinded RCT on smoking causing cancer as far as I know, but its pretty resolved that smoking causes cancer, agreed?