Eight Hundred Slightly Poisoned Word Games

post by Scott Alexander (Yvain) · 2021-08-09T20:17:17.814Z · LW · GW · 5 comments

[cross-posted from my blog Astral Codex Ten]

In 2012, a Berkeley team found that indoor carbon dioxide had dramatic negative effects on cognition (paper, popular article). Subjects in poorly ventilated environments did up to 50% worse on a test of reasoning and decision-making. This is potentially pretty important, because lots of office buildings (and private houses) count as poorly-ventilated environments, so a lot of decision-making might be happening while severely impaired.

Since then people have debated this on and off, with some studies confirming the effect and others failing to find it. I personally am skeptical, partly because the effect is so big I would expect someone to have noticed, but also because submarines, spaceships, etc have orders of magnitude more carbon dioxide than any civilian environment, but people still seem to do pretty hard work in them pretty effectively.

As part of my continuing effort to test this theory in my own life, I played a word game eight hundred times under varying ventilation conditions.

…okay, fine, no, I admit it, I played a word game eight hundred times because I’m addicted to it. But since I was playing the word game eight hundred times anyway, I varied the ventilation conditions to see what would happen.

The game was WordTwist, which you can find here (warning: potentially addictive). You get a 5x5 square of letters and you have to find as many words as possible (of four letters or more) within three minutes. You can move up, down, right, left, or diagonal, and get more points for harder words. A typical board looks like this:

I played this game about 5-10x/day over three months. During this time, the carbon dioxide monitor in my room recorded levels between 445 ppm (with all windows open and the fan on) and 3208 ppm (with all windows closed and several people crammed into the room for several hours). I discounted a stray reading of 285 as an outlier, since this is climatologically impossible (I’m not claiming my monitor is perfectly calibrated, just that it clearly shows higher levels when my room is less well ventilated). CO2 445 is basically the same as outdoors; 3208 is considered extremely poor air quality, likely to cause headaches, nausea, and other minor ailments. The Berkeley study looked at levels between 600 and 2500, so my range was comparable to theirs.

I correlated my adjusted score (my score as a percent of the average score for that board) for each game with the CO2 level in my room when I was playing it. R was 0.001, p = 0.97 - there was absolutely no correlation.

Why might these results not be valid? Well, CO2 level in my room wasn’t randomly determined - I just played a game when I felt like it and recorded whatever the ambient CO2 level was at the time. CO2 level was lower if I had the window open or air conditioning on, higher if I’d been in the room for a long time, and highest if I’d just woken up after being asleep in the room all night. It was also higher when other people were in my room. In theory things like this could confound the results. For example, if CO2 really did affect performance, but I performed better when I was hot, then turning the air conditioning on might improve performance (by decreasing CO2) but also hurt performance (by making it colder), and those effects could cancel out. Or if I performed worse after exercise, and I often went out of my room to exercise, then I might perform worse when I had just come back into my room (which was often when CO2 was lowest).

In practice I’m skeptical this mattered. For one thing, the studies found huge positive effects - so for me to find zero effect would require a huge negative effect of the exact right size to cancel out the huge positive one. For another thing, I checked if temperature had any effect, and it didn’t (r = -0.008, p = 0.83). For another, I ran a few controlled experiments to see if they got the same results as the naturalistic ones, and they did. For another, I did get to test an exogenous shock - about halfway through the experiment, I moved to a new house with better ventilation. The difference in average CO2 reading between the old and new houses was significant (p < 0.001), but the difference in score wasn’t (p = 0.15). Although it was in the expected direction (new house > old), I attribute this to me improving on the word game with practice, and I didn’t improve any more during the month when I switched houses than in an average month.

I consider this to be very strong evidence that at least for me, on this specific task, carbon dioxide has zero effect on cognition. To rescue the hypothesis that it matters, you’d either have to find that it affects other people more than it does me (why would it?) or that it affects other aspects of cognition more than it affects the skills associated with this particular word game. This second one is moderately plausible - I don’t think the word game tests “decision-making” per se. But it would be surprising for this not to be a general health effect, and would potentially be important in the study of intelligence and neuroscience to explore which skills do or don’t suffer under carbon dioxide poisoning.

I was excited to read the Less Wrong post Chess and cheap ways to check day to day variance in cognition [LW · GW] by KPier, who does something similar with chess instead of a word game; they haven’t checked carbon dioxide levels yet, but I’d be excited for them to try. I’m also interested in hearing from anyone else who often repeats some objectively-scoreable cognitive task, to see how they do. A CO2 monitor costs about $100 on Amazon, but if money is the only reason you’re not going to do some really good experiment, please let me know and I’ll buy it for you.

If you’re planning on testing this, please post about it below as a form of preregistration.

EDIT: You can download the original data here, some explanations of what the columns mean here.

5 comments

Comments sorted by top scores.