Posts

Comments

Comment by LauWren on Beautiful Probability · 2011-02-16T14:48:41.660Z · LW · GW

We have additional information about researcher 2's experiment. If researcher 2 didn't look at the data before that point, then the procedures were the same, so the data should be treated the same.

If researcher 2 did check the data along the way--a reasonable enough assumption, given researcher 2's goal--then there were other tests which all came out below 60%. There was an upswing in successes at the end, and we know it. The other experiment may well have experienced the same thing, but in experiment 2, I don't have to look; I see it. Was there an important variable that we overlooked?

Of course, this is only due to extra information I happen to have about the latter. I haven't bothered to check whether something similar was there for the previous, because there wasn't anything to make it catch my attention. If I have the tools to do so, I would still like to treat them both the same--I want to see researcher 1's results lined up by time as well. If the upswing is repeated...well, that's funny.