Self-experiment Protocol: Effect of Chocolate on Sleep

post by NoSignalNoNoise (AspiringRationalist) · 2019-07-22T03:32:19.106Z · LW · GW · 8 comments

Contents

  Epistemic Status
  Introduction and Goals
  Experimental Protocol
  Statistical Analysis
None
8 comments

Epistemic Status

Preregistering an experiment, not very confident in the design.

Because it is important to decide how the data will be analyzed before collecting it, I'm posting my planned experiment here, may modify the design based on feedback (I'll post an update if I do), and then I will start the experiment, and finally I'll post the results when it's finished.

All feedback is appreciated!

Introduction and Goals

I often suffer from fatigue in the early evening that leads to anxiety later at night, which makes it difficult to sleep. The purpose of this experiment is to determine whether eating a small amount of chocolate to counteract fatigue in the early evening affects my sleep, and if so, how. I could easily imagine the effect on sleep being either beneficial (counteract the pattern of fatigue and anxiety that makes it hard to sleep) or harmful (the theobromine might keep me awake).

Because I'd rather not add the fat and sugar that comes with chocolate to my diet, the effect on sleep needs to be appreciable for it to be worthwhile. I have fairly arbitrarily decided that the chocolate is worth it iff it causes me to fall asleep at least 15 minutes earlier than I otherwise would.

Experimental Protocol

Every work day evening when I go directly home from work and do not have a social event planned, I will get home, make any necessary adjustments to the air conditioning (temperature affects my sleep a lot, and I want to make sure I don't accidentally bias the results by setting the AC differently depending on whether I have chocolate), and then with probability 1/2 (decided by a coin flip or die roll) eat a square of dark chocolate.

I will use a spreadsheet to track which nights I did and did not eat a square of chocolate, and I will use sleep data from a Fitbit Blaze smartwatch to record what time I fall asleep each night. The Blaze uses heart rate and movement data to decide what time I fall asleep (I don't need to actively tell it when I go to bed). In my experience, the time it says I fell asleep generally matches my subjective memory of when I fell asleep (not when I went to bed).

Statistical Analysis

I care about the magnitude of the effect, not just the direction, and my goal is to make the correct decision rather than to get published, so testing whether the observed effect is statistically significantly different from 0 is not very useful. While it would be a mistake to keep going until I get the result I want [LW · GW], if the results are close (in either direction), it will be worth gathering more data.

Because I have arbitrarily picked an absolute effect size of -15 minutes (I fall asleep 15 minutes sooner than I otherwise would), I will be focusing on determining which side of that cut-off the effect is. To do so, I plan on analyzing the data as follows:

1. Run the experiment until I have used up one bag of chocolate (15 individually-wrapped squares).
2. Calculate the standard error of the difference of sample means (I am not assuming equal variance in the two samples).
3. If the observed effect is at least one standard error away from the -15 minute cut-off (my sleep is inconsistent enough that I expect this condition probably won't be met at this point), stop the experiment.
4. Otherwise, keep going until either the stop condition from step 3 is met or the standard error is less than 5 minutes. If I end with a standard error less than 5 minutes and the observed effect is within 5 minutes of -15 minutes, I will consider the experiment inconclusive.

8 comments

Comments sorted by top scores.