Hard to find factors messing up experiments: Examples?

post by Hazard · 2019-11-15T17:46:03.762Z · LW · GW · 11 comments

This is a question post.

Contents

  Answers
    8 johnswentworth
    6 avturchin
None
11 comments

In Richard Feynman's talk on cargo cults he mentions a story about experiments with rat mazes (skip if you recognize it)

All experiments in psychology are not of this type, however.  For example, there have been many experiments running rats through all kinds of mazes, and so on—with little clear result.  But in 1937 a man named Young did a very interesting one.  He had a long corridor with doors all along one side where the rats came in, and doors along the other side where the food was.  He wanted to see if he could train the rats to go in at the third door down from wherever he started them off.  No.  The rats went immediately to the door where the food had been the time before.

The question was, how did the rats know, because the corridor was so beautifully built and so uniform, that this was the same door as before?  Obviously there was something about the door that was different from the other doors.  So he painted the doors very carefully, arranging the textures on the faces of the doors exactly the same.  Still the rats could tell.  Then he thought maybe the rats were smelling the food, so he used chemicals to change the smell after each run.  Still the rats could tell.  Then he realized the rats might be able to tell by seeing the lights and the arrangement in the laboratory like any commonsense person.  So he covered the corridor, and, still the rats could tell.

He finally found that they could tell by the way the floor sounded when they ran over it.  And he could only fix that by putting his corridor in sand.  So he covered one after another of all possible clues and finally was able to fool the rats so that they had to learn to go in the third door.  If he relaxed any of his conditions, the rats could tell.

I also recall some experiment where it was eventually discovered that the control of some microbiology experiment was screwed because the paper towels used in some process were made with wood that was treated with a particular chemical before it was turned into the paper towel (was this linked by gwern at some point?).

I'm interested in collecting any and all examples you have of these events; through painstaking effort someone realized that XYZ was interfering with an intended experiment (extra points if XYZ was completely outside the experimenters initial frame of "factors that might be relevant to this experiment") . Links to source if possible.

Answers

11 comments

Comments sorted by top scores.