microCOVID.org: A tool to estimate COVID risk from common activities
post by catherio · 2020-08-29T23:01:02.081Z · LW · GW · 36 commentsThis is a link post for https://microcovid.org/
This is a linkpost for a model and web tool (that I and several friends created) to quantitatively estimate the COVID risk to you from your ordinary daily activities:
This website contains three outputs of our work:
- a web calculator that you can use to calculate your COVID risk (in units of microCOVIDs, a 1-in-a-million chance of getting COVID).
- a white paper that explains our estimation method. EAs might be particularly interested in the footnotes throughout, and the detailed research sources section.
- a spreadsheet to compute your COVID risk in more detail and to track your risk over time. EAs might find this more customizable and powerful than the web calculator.
If you have different beliefs than us and would like to use a version of the model that reflects your beliefs rather than ours, you can make modifications to your copy of the spreadsheet, or fork the repository and make a personal copy of the web calculator. We also hope you will submit suggestions, either by emailing us or by making issues or pull requests directly on Github.
Our group house has been using this model as the basis of a shared agreement/protocol, based on a budget of 3,000 microCOVIDs per year to spend outside the house (about 58 per week). We know of another group house that (last we heard) was operating on *4* microCOVIDs per week!
We hope this helps you personally live a better pandemic life with more safety and more flexibility.
(also linkposted to the EA Forum [EA · GW])
36 comments
Comments sorted by top scores.
comment by Raemon · 2020-08-30T01:32:16.898Z · LW(p) · GW(p)
I'd also like to include this additional risk calculator by Peter Hurford, both for cross-checking the various risk-levels, and because I found Peter's spreadsheet to be helpful for orienting around "what sort of risk do I want to expose myself to?".
https://docs.google.com/spreadsheets/d/1LBZWHEk2Jo-IFvZK_smrwYoOTOykB7H-oHb0qYjg2ys/edit#gid
comment by Raemon · 2020-08-31T22:42:27.366Z · LW(p) · GW(p)
Curated.
We've been steering away from frontpaging Covid content on LW because it's not timeless, but make occasional exceptions for things that seem particularly important. "Importance" is a bit of a judgment call, but this post and app seems to make a significant improvement in how people can reason about their covid safety. I know a lot of people who were defaulting to pretty strict rules because thinking about each individual case was too cognitively taxing, and having this guide is really helpful for moving into the longterm "how to reason about covid for the next year."
I'd be interested in people continuing to sanity-check the numbers and some of the judgment calls in the microcovids.org math/science, but it looks like it's gotten enough impromptu review for me to feel comfortable at least using it as a supplemental decisionmaking aid.
comment by Raemon · 2020-08-29T23:15:26.369Z · LW(p) · GW(p)
Thanks. This is great.
A thing I'd be interested in (but I acknowledge it's a bit tricky to navigate), is somehow better leveraging the wisdom of crowds here. I like that the tool as-is is pretty clean and simple, and I like that you provide the raw spreadsheet for people to go tweak the variables to match their own epistemics.
It'd be nice if I could see how much disagreement there was on the risk analysis of individual components, and ideally see what people's reasoning was.
There's a lot of trickiness in "if you just let anyone submit disagreeing statements, you're opening yourself up to managing arguments about whether so-and-so is a crackpot or whatever" and that sounds like a huge pain, I'm not sure if there's a way to sidestep that.
But, my ideal version of this lets me see different estimates with associated reasoning, and then make some kind of judgment call on my own of whether to go with microcovid.org's default estimate, or wisdom of crowds, or subset-of-wisdom-of-crowds if I trust some people's judgment more than others.
Replies from: adamzerner, adamzerner↑ comment by Adam Zerner (adamzerner) · 2020-09-01T20:26:24.873Z · LW(p) · GW(p)
To me microCOVID's defaults seem close enough to the truth that the ideal version you describe wouldn't provide too much marginal value.
Especially since, at least to me, the value is mostly in knowing what activities I will/won't do rather than nailing down the precise number of microCOVIDs. Eg. knowing that eating at a restaurant inside is 8,500 microCOVIDs instead of 10,000 wouldn't be enough to get me to eat at a restaurant inside, so it doesn't really matter to me whether the real number is 8,500 or 10,000. However, given the wide confidence intervals, maybe this point doesn't have too much weight.
↑ comment by Adam Zerner (adamzerner) · 2020-09-01T20:29:35.663Z · LW(p) · GW(p)
There's a lot of trickiness in "if you just let anyone submit disagreeing statements, you're opening yourself up to managing arguments about whether so-and-so is a crackpot or whatever" and that sounds like a huge pain, I'm not sure if there's a way to sidestep that.
I don't think it'd really be possible to side step it 100%, but if you eg. only accept statements from people with PhDs, maybe that'd be good enough. Eg. maybe the benefit of the extra inputs would outweigh the fact that the sources aren't fully vetted.
comment by Adam Zerner (adamzerner) · 2020-08-31T01:26:22.293Z · LW(p) · GW(p)
I think it'd be cool to go from microCOVIDs to expected QALYs lost, and then from there put a rough dollar figure on it based on the value of a QALY.
Edit:
- 10 microCOVIDS
- = 1 in 100k chance of getting COVID
- = 1 in 10M chance of dying from COVID @ 0.1% fatality rate
- = 0.000005 expected QALYs lost @ 50 QALYs available to lose
- = $0.10 @ $200k/QALY
- = $0.01 / microCOVID with these assumptions
Eg. 10 microCOVIDs = 0.0005 expected QALYs lost (assuming 50 QALYs available to lose) = $100 (@ $200k/QALY).
Knowing that it "costs" about $100 to hang out with two friends outside feels a lot more concrete and actionable than knowing that there's a 1 in 100k chance it gives me COVID, in no small part due to scope insensitivity.
↑ comment by Austin Chen (austin-chen) · 2020-08-31T06:47:22.606Z · LW(p) · GW(p)
A microCOVID (μCoV) is a 1/1,000,000 chance of catching COVID, not of *dying from* COVID. Off the top of my head, COVID has between a 1% and a 0.1% case fatality rate, so the cost of hangout death is something like $1 to $0.10. (Equivalently, a μCoV costs between 1 and 10 cents.) That hangout seems pretty cheap now!
You might separately check cost of going through COVID, which ranges from "no symptoms" to "pretty sick for a week to a month". Being an affluent, reasonably healthy young person, I'd pay $1k to avoid the COVID experience but not $10k -- so this additional cost is between $0.01 and $0.001 per μCoV. For simplicity's sake, I'm combining both these figures into an overall 1 cent per microCOVID.
I definitely agree that a dollar framing helps make things actionable. Eating inside (5000μCoV = $50) is ridiculously expensive compared to outside (300μCoV = $3), and hitting up a bar, at 40k μCoV, has an astronomical cost of $400!
Replies from: Zvi, adamzerner, jalex-stark-1↑ comment by Zvi · 2020-09-01T10:40:53.084Z · LW(p) · GW(p)
If I thought it was only worth $10k to avoid Covid, then I would stop trying to not catch it entirely, I'm giving up way, way more value than that and none of my actions make sense, and also the lockdown is a beyond obvious mistake (e.g. $10,000 times 300 million extra cases is both the beyond-worst-case and also 3 trillion so just let it happen modulo the most vulnerable).
Also, dollar framing this way excludes impact on others, so it'll be a vast underestimate. Including it would likely give people a very bad idea.
Replies from: austin-chen, adamzerner, complexmeme↑ comment by Austin Chen (austin-chen) · 2020-09-03T03:16:33.351Z · LW(p) · GW(p)
I accept that my 10k figure is lower than typical; again, I'm relatively young (25) and risk tolerant. I'm curious where you'd bound your "cost of avoiding covid" at - $100k? $1M?
I did not model impact on others, and agree that this is a major oversight - as OP stated, there aren't great models of this yet and we should try to do better.
But crucially, I don't think "my current parameters leads to a massive underestimate" logically equates to "dollars are a bad framing device for understanding risk". I almost feel like the fact that you can have such strong immediate opinions on seeing these dollar figures, means that converting to dollars provided a lot of clarity around our respective thought processes.
Of course it's a simplification; it paints over, for example, the fact that two people with different incomes but the same risk tolerance would assign different dollar values. But on the flip side, literally every member of our society has grown up assessing the prices of things in dollars. QALY, micromorts, now microcovids, etc are incredibly esoteric by comparison.
↑ comment by Adam Zerner (adamzerner) · 2020-09-01T16:43:23.225Z · LW(p) · GW(p)
He was saying that it is worth $10k to him to avoid the experience of being sick with but not dying from Covid.
Impact on others can be incorporated into the dollar estimate using R0 and the value you place on those other lives as parameters.
Edit: microCOVIDs also excludes impact on others.
↑ comment by complexmeme · 2020-09-04T13:43:08.386Z · LW(p) · GW(p)
Yeah, I've seen some posts trying to make similar "lockdown goes too far" arguments (including this one on the SSC Tumblr) that seem to be comparing life with COVID-19 mitigation to normal 2019 life or to that plus some chance of getting sick. Aside from understating the potential for long-term consequences, I think there's a trend in those dollar-cost estimates towards significantly underestimating the negative effects of unmitigated pandemic spread beyond the effect on one's personal health.
(Not that I expect that you disagree with this, but it stands out to me that "let it happen modulo the most vulnerable" is already begging the question. I'd expect if that were driving public policy that the "modulo the most vulnerable" part largely wouldn't happen. It's hard to protect any particular group from infectious disease when it's widespread in the general population.)
↑ comment by Adam Zerner (adamzerner) · 2020-08-31T16:42:26.989Z · LW(p) · GW(p)
Ah I see. My mistake for missing that!
↑ comment by Jalex Stark (jalex-stark-1) · 2020-08-31T16:05:50.254Z · LW(p) · GW(p)
surviving COVID might cost a lot of QALYs from permanent lung and brain damage. It might also cost a lot of future expected earnings for the same reason.
Replies from: austin-chen↑ comment by Austin Chen (austin-chen) · 2020-08-31T21:31:28.025Z · LW(p) · GW(p)
I hear this objection a lot but don't have a sense of how likely/how bad "permanent lung/brain damage" is -- do you happen to have any sources? I think this scenario is in the public conciousness because it's scary and newsworthy, not because it's common. Randomly guessing I'd say that permanent damage is meaningful in ~1% of all cases?
I tried to factor this already into my $1k - $10k COVID avoidance price, but I'd be happy to update on new data, and of course you might have different subjective valuations.
Replies from: catherio, catherio↑ comment by catherio · 2020-08-31T23:12:39.204Z · LW(p) · GW(p)
Also, don't forget to factor in "kicking off a chain of onwards infections" into your COVID avoidance price somehow. You can't stop at valuing "cost of COVID to *me*".
We don't really know how to do this properly yet, but see discussion here: https://forum.effectivealtruism.org/posts/MACKemu3CJw7hcJcN/microcovid-org-a-tool-to-estimate-covid-risk-from-common?commentId=v4mEAeehi4d6qXSHo#No5yn8nves7ncpmMt [EA(p) · GW(p)]
Replies from: austin-chen↑ comment by Austin Chen (austin-chen) · 2020-08-31T23:54:06.137Z · LW(p) · GW(p)
Good point, thanks.
Running the "microCOVID to $" conversion from the other end of the spectrum, the recommendation of 1% COVID risk = 10k μCoV to spend/year would suggest a conversion rate of $1 per μCoV (if your yearly discretionary budget is on the order of $10k/year).
I keep coming back to the "dollars conversion" because there's a very real sense in which we're trained our entire lives to evaluate how to price things in dollars; if I tell you a meal costs $25 you have an instant sense of whether that's cheap or outrageous. Since we don't have a similar fine-tuned model for risk, piggybacking one on the other could be a good way to build intuition faster.
Replies from: adamzerner↑ comment by Adam Zerner (adamzerner) · 2020-09-01T16:52:52.619Z · LW(p) · GW(p)
I keep coming back to the "dollars conversion" because there's a very real sense in which we're trained our entire lives to evaluate how to price things in dollars; if I tell you a meal costs $25 you have an instant sense of whether that's cheap or outrageous. Since we don't have a similar fine-tuned model for risk, piggybacking one on the other could be a good way to build intuition faster.
That's a great way to put it. And since the goal of the microCOVID project is behavior change (presumably), I think it's crucial to get the "have an instant sense of whether it's cheap or outrageous" part right. Without that I fear that only the most committed people would be motivated enough to change their behavior, but a lot of those people are probably being cautious to begin with.
Anecdotally, I was talking to my brother (not super committed) about it last night, and that data point supported what I'm saying.
↑ comment by catherio · 2020-08-31T23:09:42.962Z · LW(p) · GW(p)
Sadly nothing useful. As mentioned here (https://www.microcovid.org/paper/2-riskiness#fn6) we think it's not higher than 10%, but we haven't found anything to bound it further.
Replies from: rockthecasbah, William_S↑ comment by Tim Liptrot (rockthecasbah) · 2020-09-01T15:22:30.547Z · LW(p) · GW(p)
One way to bound the risk of long term consequences is to assume the long term consequences will be less severe than the infection itself. So if 1% of people in their 20's experience reduced lung capacity during infection, you can assume that less than 1% will have permanently reduced lung capacity. I have never heard of a disease which was worse after you recover than before.
I suspect that some people are hesitant to discuss the rate of long term consequences for young covid patients for fear of encouraging people not to social distance. But then the cost is a loss of trust between people and the information provider.
↑ comment by William_S · 2020-09-02T17:57:40.856Z · LW(p) · GW(p)
https://www.cdc.gov/mmwr/volumes/69/wr/mm6930e1.htm found that ~1 in 5 of 18-34 year olds with no underlying health conditions had symptoms 3 weeks later (telephone survey of people who'd been symptomatic and had a positive test).
Other discussion in comments of https://www.lesswrong.com/posts/ahYxBHLmG7TiGDqxG/do-we-have-updated-data-about-the-risk-of-permanent-chronic [LW · GW]
Replies from: rockthecasbah↑ comment by Tim Liptrot (rockthecasbah) · 2020-09-05T15:41:16.077Z · LW(p) · GW(p)
Interesting. The study discusses fatigue. Do we know if the fatigue is caused by reduced lung capacity or by the hormones/neuro stuff our body does to conserve energy while sick. If reduced lung capacity is a big part of that 1/5 I would update upward on permanent lung capacity rate.
comment by Richard Bartlett (richard-bartlett) · 2020-09-04T13:25:31.335Z · LW(p) · GW(p)
Thank you Catherio and friends for this incredible work, and the generosity of sharing it. It is super helpful
I have a question: I'm wondering how to reason about the nonlinearity of risk vs group size?
The model assumes risk and group size have a linear relationship, i.e. it is 10x more risky to do an activity with 10x more people.
I don't know how well this approximation holds. Yes, if there are 10x more average people at a party, then there are 10x more infected people. But I may not have meaningfully interactions (e.g. shared airspace) with them all.
I'm assuming at some point whether there's 5,000 or 50,000 people notionally included in the same activity, the risk to me does not increase.
So I'm curious to understand how significant this nonlinearity is. What thinking has already been done on this? What data do we have?
e.g. are there reports that show the upper bound of the number of people that were infected by participating in a single group activity?
↑ comment by catherio · 2020-10-22T21:24:20.620Z · LW(p) · GW(p)
Sorry to leave you hanging for so long Richard! This is the reason why in the calculator we ask about "number of people typically near you at a given time" for the duration of the event. (You can also think of this as a proxy for "density of people packed into the room".) No reports like that that I'm aware of, alas!
comment by Vaniver · 2021-12-23T16:17:12.091Z · LW(p) · GW(p)
I think microCOVID was a hugely useful tool, and probably the most visibly useful thing that rationalists did related to the pandemic in 2020.
In graduate school, I came across micromorts, and so was already familiar with the basic idea; the main innovation for me in microCOVID was that they had collected what data was available about the infectiousness of activities and paired it with a updating database on case counts.
While the main use I got out of it was group house harmony (as now, rather than having to carefully evaluate and argue over particular activities, people could just settle on a microCOVID budget and trust each other to do calculations), I think this is an example of a generally useful tool of 'moving decision-relevant information closer to decision-making,' a particularly practical sort of fighting against ignorance. If someone only has a vague sense of what things carry what risks, they will probably not make as good choices as someone who sees the price tag on all of those activities.
comment by Richard Bartlett (richard-bartlett) · 2020-11-24T14:49:51.289Z · LW(p) · GW(p)
I wanted to loop back to this post to say Thank You! for the microcovid project. I wrote up a case study of how we used it.
comment by GuySrinivasan · 2020-09-02T02:31:34.992Z · LW(p) · GW(p)
I used this today to get a quick estimate for the relative risks of an in-person doctor's consult, a 6 hour ER visit, and a round trip cross-country plane trip. Was very useful to have an easy way to get order-of-magnitude estimates. Thanks.
comment by remizidae · 2020-08-30T18:25:33.973Z · LW(p) · GW(p)
Selecting the scenario is duplicative of Steps 2 and 3. Should I skip those if I pick a scenario? Instructions are not clear.
I also think you should enable people to choose higher than average risk tolerance.
Replies from: remizidae↑ comment by remizidae · 2020-08-30T20:50:25.970Z · LW(p) · GW(p)
Couple other thoughts (cause this is fascinating, thanks!)
--The activity of grocery store shopping could be usefully expanded to shopping in any indoor store. Seems like all stores would be about the same level of riskiness if you control for number of people nearby (unless there are grocery-store-specific concerns?)
--The bar/restaurant distinction, and specifically rating bars as much higher risk than restaurants, is not convincing. I've heard people make this claim before but without explaining why. They might be thinking that going to a bar involves 1) more crowds, 2) more likely sitting at the bar, or 3) heavy drinking that clouds judgment. But while all of those things *sometimes* occur in bars, they don't always, and they very often occur in restaurants too. (Plausibly people drink on average more in bars, but that's not enough to warrant a dramatic increase in risk.) I suspect some of this is coming from people who either are unfamiliar with bars or have a certain Puritanical prejudice against drinking, leading them to think of bar-going as "more optional" than restaurant dining and therefore more condemnable. Obviously, both activities are optional, and there's no reason to judge the bar-goer more harshly than the diner.
Anyway, your numbers should be revised to reflect the reality that the key risk factors for a dining establishment are indoor vs. outdoor, and crowded vs. spaced. The type of liquor license the place has doesn't matter.
--This one is less feedback for the developers and more thinking out loud. A few people, like OP, are still on very high voluntary lockdown levels even now that almost every place has reopened. A few people, on the other hand, never really changed their habits that much because of COVID, or only did so when forced. The vast majority I expect are people who made significant sacrifices during the first month or two or three, but started to move back in the direction of a normal life when it became obvious that this pandemic was going to last a while. I wonder about the psychological effects it will have for the hardcore few to see groups 2 and 3 doing all sorts of things that the hardcore won't let themselves do. Just sitting at home and watching quasi-normal life going on around them, while they shut themselves out and self-flagellate about every grocery store trip. I'm not criticizing--but, guys, be careful of your mental health. If it makes you happy to track the risk metrics this closely, do it, but if it's making you anxious and amplifying tendencies towards scrupulosity, you don't have to do it!
Replies from: catherio↑ comment by catherio · 2020-08-31T19:37:32.820Z · LW(p) · GW(p)
"I've heard people make this claim before but without explaining why. [...] the key risk factors for a dining establishment are indoor vs. outdoor, and crowded vs. spaced. The type of liquor license the place has doesn't matter."
I think you're misunderstanding how the calculator works. All the saved scenarios do is fill in the parameters below. The only substantial difference between "restaurant" and "bar" is that we assume bars are places people speak loudly. That's all. If the bar you have in mind isn't like that, just change the parameters.
Replies from: remizidae↑ comment by remizidae · 2020-09-01T04:29:26.761Z · LW(p) · GW(p)
I suggest clarifying in the calculator how people are supposed to use the "scenarios" versus Step 2 or Step 3. Also, you suggest that the only difference between restaurant and bar in your model is volume of talking, but that doesn't seem to fit the result when I pick Step 2 and Step 3--the bar scenario gives me 10,000 microcovids, but the indoor place with loud talking option is only 9000. Also, why do you think all bars are indoor and involve loud talking? Weird assumption. Some bars are very quiet and empty, lots have outdoor seating nowadays.
Overall, I think you guys haven't quite figured out what your intended audience is. If you want to reach the general public, you'll need an easy to use calculator that does not smuggle in a lot of doubtful assumptions. Yeah, I understand you can download the spreadsheet and customize, but that option is for the nerds.
Replies from: Raemon↑ comment by Raemon · 2020-09-01T04:52:09.429Z · LW(p) · GW(p)
I think you're reading more into the bar thing than is intended. It's not meant to be a strong statement about all bars, it's just one of a list of examples to give you a sense of what different parameters look like.
I do think it'd be a bit of an improvement to make the bar dropdown more specific (ie. "Go to a loud, indoor bar"), but that feels more like a slight tweak than a major adjustment to target audience.
comment by spookyuser · 2020-08-31T18:44:58.586Z · LW(p) · GW(p)
This is incredible, thank you :)
comment by Raemon · 2021-12-22T22:31:53.836Z · LW(p) · GW(p)
I'm interested in someone reviewing the microcovid whitepaper, which I think is a good lens for evaluating this in the context of the LessWrong 2020 Review.
I'm separately interested in someone doing some cross comparison of microcovid.org the tool, with Peter Hurford's calculator or similar attempts.
comment by catherio · 2021-01-13T00:07:45.723Z · LW(p) · GW(p)
There is now a wired article about this tool and the process of creating it: https://www.wired.com/story/group-house-covid-risk-points/
I think the reporter did a great job of capturing what an "SF group house" is like and how to live a kind of "high IQ / high EQ" rationalist-inspired live, so this might be a thing one could send to friends/family about "how we do things".
comment by kindwolves · 2020-09-02T17:15:29.203Z · LW(p) · GW(p)
This resource is such a great contribution to COVID risk assessment. Thank you for this.
Are there any further resources that you might be able to point us to re: deciding on an annual household (or closed bubble) microCOVID budget?
comment by Mati_Roy (MathieuRoy) · 2020-09-02T05:39:47.677Z · LW(p) · GW(p)
suggestion: have an internal house market for the microCOVIDs
EtA: although this might create an incentive for some people to lower the microCOVID budget...