We Still Don't Know If Masks Work 2021-07-05T04:52:54.435Z
There Is No Control System For COVID 2021-04-06T17:16:04.171Z
Effective Epidemiology 2020-10-22T19:45:42.742Z
COVID Strategy 101 2020-10-05T19:30:43.529Z


Comment by Mike Harris (mike-harris-1) on We have some evidence that masks work · 2021-09-01T14:31:21.644Z · LW · GW

The study was just released

For the cloth mask they got a 5% reduction in seroprevalence (equivalent to 15% for 100% increase) and for surgical masks they got an 9.3% reduction (equivalent to 28% for 100% increase).

I unequivocally lost the bet and will send my donation. Let me know if you have a preferred charity.

Comment by Mike Harris (mike-harris-1) on We have some evidence that masks work · 2021-07-31T18:35:52.810Z · LW · GW


My current belief state is that cloth masks will reduce case load by ~15% and surgical masks by ~20%.

Without altering the bet I'm curious as to what your belief state is.

Comment by Mike Harris (mike-harris-1) on We have some evidence that masks work · 2021-07-30T16:40:41.344Z · LW · GW

I can't accept the wording because the masking study is not directly measuring Rt. I would prefer this wording

"Gavin bets 100 USD to GiveWell, to Mike's 100 USD to GiveWell that the results of NCT04630054 will show a median reduction in cumulative cases > 15.0 % for the effect of a whole population wearing masks [in whatever venues the trial chose to study]."

Comment by Mike Harris (mike-harris-1) on We have some evidence that masks work · 2021-07-30T07:33:33.516Z · LW · GW

I am also not convinced that zeroing out mask levels at the start solves this problem. The random walk variable is also a learned per region factor. Even if the starting value can no longer be influenced by mask levels, the starting value + 1 month of random walk value for the region can be influenced.

Comment by Mike Harris (mike-harris-1) on We have some evidence that masks work · 2021-07-30T04:48:21.593Z · LW · GW

I'd like to respond to some of the points raised.

First I'd like to apologize for not reaching out to you before publishing my critique, I tried to integrate your responses from our email conversation but should have given you a chance to respond before publishing.

A minor point, for the data extrapolation you are reading the graph incorrectly. A higher growth difference meant that the growth rate (a rough approximation of R0) fell more sharply. The point of this section was not that the effect wasn't large, but that it pointed weakly in the wrong direction. Regions which increased mask wearing the most had their growth rates fall more slowly. I don't think this is strong evidence, but it does point against the effectiveness of mask wearing. This is the spreadsheet used to compute the graph:

The core point of dispute if I understand it correctly is that knowing the absolute level of mask wearing in a region does not give evidence as to the overall R0 (even taking into account mobility and NPIs), but knowing the change in mask wearing over time gives evidence as to the change in R0.

In the model in your paper the absolute effect and relative effect are not disambiguated and an effect size mean of 25% reduction is observed.

In your proposed specification you try to isolate the relative effect by zeroing out starting mask wearing and observe a higher impact.

In my proposed specification I try to isolate the absolute effect by zeroing out the relative changes and observe a smaller impact.

These two observations don't contradict each other. The data is consistent with two distinct causal stories.

  1. High inherent transmission -> Mask Wearing -> Lower transmission. This is your preferred model and indicates that the absolute effect is spurious.

  2. High inherent transmission -> Mask Wearing + Lower Transmission later. This is my preferred model and indicates that the relative effect is spurious.

One reason to favor the second story is that although the model is described as measuring R_0, it is actually measuring Rt. The model does not include population infections as a modifier to the growth rate. This is a known causal factor which would artificially make masks look more effective.

As further evidence that our actual belief states are not too far apart, 15-25% reduction in case load (different from R0) was my best guess for the results of the mask RCT. I will make the $100 bet with the caveat being that payment is in the form of a donation to GiveWell.

Comment by Mike Harris (mike-harris-1) on There Is No Control System For COVID · 2021-04-06T23:03:57.718Z · LW · GW

I agree that this is a relative weakness of the model. I think part of it is that the division into vulnerable/invulnerable is a simplification. If for instance you injected somebody with COVID then everybody would be "vulnerable". So in some environments conditions are ideal for spread which makes many relatively immune people become infected.

Comment by Mike Harris (mike-harris-1) on Effective Epidemiology · 2020-10-26T15:27:32.516Z · LW · GW

I'm sympathetic to the case that education is signaling, but I think that case is less strong for early education. For instance this paper from Argentina uses teacher strikes to value a year of education at 6% of lifetime earnings.

That estimate is not wildly different and seems pretty immune to signaling.

Comment by Mike Harris (mike-harris-1) on Effective Epidemiology · 2020-10-23T03:37:26.895Z · LW · GW

Yeah, I don't think that HVAC in schools is something which will make a difference to their safety in time. My point was more

  1. We have a large set of buildings we should open regardless of safety
  2. We have an expensive intervention which may drastically reduce transmission.
  3. Here is an opportunity to experiment.

I would imagine that we could install experimental HVAC systems in a few hundred schools for not much money and get decent data.

Comment by Mike Harris (mike-harris-1) on COVID Strategy 101 · 2020-10-06T15:48:58.569Z · LW · GW

I did not compute the odds precisely before writing up that section.

The two cases cited have attack rates of 53/61 and 104/122.

For a 25% cross immunity rate that would correspond to ~2% and .4% probability respectively. For a 33% cross immunity rate that would correspond to a .02% and a .0001% probability respectively.

The actual claim of what percentage of the population is immune is fairly nebulous but anything beyond 25% would be hard to justify.