Posts

Book Review: The Righteous Mind 2022-01-20T21:42:00.883Z
Why is Covid Surveillance so Hard? 2021-12-28T17:41:06.383Z
Rapid Antigen Tests for COVID 2021-10-21T15:26:25.986Z
Interpreting Diagnostic Tests 2021-10-15T02:58:37.139Z

Comments

Comment by Tornus on Book Review: The Righteous Mind · 2022-01-20T21:52:39.348Z · LW · GW

As a side note, I run a thing that's like a book club but different and we're talking about The Righteous Mind on Saturday 1/22. We have room for a few more thoughtful people—feel free to message me if you're interested.

Comment by Tornus on How to End a Pandemic · 2022-01-03T22:54:09.473Z · LW · GW

Vaccine-required zones seem unworkable to me: ours is a highly connected society and it's common for a single household to have members who have jobs / school separated by many miles. Self-sufficiency is completely impossible in the modern world—the closest example is probably North Korea, but that's probably not a model we want to pursue.

There are also immense transaction costs here: there's no area where everyone wants (or doesn't want) to be vaccinated, so implementing this would require massive migration, with immense costs.

It seems to me you've hit on one of the most interesting and challenging things about Covid policy (at both a government and a household level): many of the usual libertarian-ish solutions don't work here, because of the difficulty of keeping one person's choices from impacting everyone around them.

Comment by Tornus on What are sane reasons that Covid data is treated as reliable? · 2022-01-02T18:03:32.487Z · LW · GW

I'm afraid I only have time for a short, partial response today. Short version: Covid surveillance is hard, and there's lots of noise in the data. But there are lots of smart people working hard on this, and in the aggregate we actually have a pretty good idea what's going on.

I'll address one of the questions you asked specifically:

So where are these numbers for variant spread coming from? Maybe hospitals do have special genetic tests and reliably do those? But then isn't there going to be a pretty strong bias based on the fact that these are only for people who are getting hospitalized?

In Washington, much of the variant prevalence data comes from UW, which sequences a subset of the samples they receive. This is a bit complicated: some samples are fully sequenced, and some are tested for S-Gene Target Failure, which is a faster, easier test that is a fairly good (but not perfect) proxy for Omicron vs Delta. The UW sequencing is a good but not perfect sample of what's actually happening in Washington. For details on this project, the person to follow is Pavitra Roychoudhury. Details vary, but there are multiple other institutions with largely similar programs.

More general answer: you're asking good questions. They are all important, and they're obvious to any smart person who thinks about the issue for a moment. Although I don't have time to answer them all, I assure you that the smart people working on Covid have thought of every single one of your questions, and have good answers to every single one. Many of the answers are in Zvi's excellent series of Omicron updates.

Comment by Tornus on Careful With Trailing Averages · 2022-01-02T04:12:28.718Z · LW · GW

Thank you. This helped me think more clearly about something we do often.

Comment by Tornus on Is Omicron less severe? · 2021-12-31T04:07:33.611Z · LW · GW

Zvi's Omicron summary is probably your best source of information:

Omicron probably milder than Delta (~50%) so baseline IFR likely ~0.3% unless hospitals overload, lower for vaccinated or reinfected.

Comment by Tornus on How should we adjust microCOVID estimates for omicron, boosters, testing? · 2021-12-30T15:16:17.268Z · LW · GW

Good questions—thank you for starting this conversation.

Your assumptions about testing seem reasonable, and hopefully we'll have confirmatory data soon.

I have with great regret stopped using microCOVID. A factor of 2 - 3 x risk multiplier seems reasonable, but I no longer entirely trust their transmission model. It's probably still more or less valid, but Omicron is a very different disease. There's some interesting data about it preferring the upper respiratory tract to the lungs, and about how Omicron particles behave differently in aerosols, that make me worry that transmission patterns may have changed in ways that are more complicated than a simple multiplier.

I'm hoping to see more data soon (and especially hoping that the microCOVID team will update for Omicron, although that seems somewhat in question).

Comment by Tornus on Rapid Antigen Tests for COVID · 2021-12-30T15:11:19.222Z · LW · GW

Excellent question!

My best guess: for detecting people who are infectious but asymptomatic, antigen tests will likely perform approximately as well with Omicron as they have with Delta. Because Omicron infections ramp up so fast, however, I'm reducing my guess for how long you can trust the results from 12 hours to 6 hours. (That is to say, if you tested negative this morning, you shouldn't assume that you aren't infectious this evening).

In addition to the data you cite, Abbott claims their testing shows no decrease in BinaxNOW effectiveness against Omicron. That would also be my prior. I'm curious what the FDA has found, although they've been coy about releasing details. I assume we'll see more data soon.

Comment by Tornus on Why is Covid Surveillance so Hard? · 2021-12-30T14:58:05.245Z · LW · GW

We could certainly have done much better (both before and during the pandemic), but unfortunately it isn't as simple as just giving IBM $100M. Any solution needs to fit into the vast array of other existing systems used for reporting lab results, managing medical records in hospitals, etc.

The US delivers health care in a very patchwork way, which has made the deployment of electronic medical records very slow and difficult. Strong, smart leadership at the top would help a great deal, but even in the best possible case, really fixing this problem would take many years.

Comment by Tornus on What would you like from Microcovid.org? How valuable would it be to you? · 2021-12-29T15:49:40.759Z · LW · GW

microCOVID has been a game changer for me and many people around me: the ability to get quantitative risk assessments radically improved our ability to efficiently spend risk. We recently stopped using it because of Omicron, and I'm very sad about it.

To me, one of the coolest things about microCOVID has been the proof of concept that a group of smart civilians can put together a useful tool that significantly shifts the efficient frontier for navigating Covid. That alone seems valuable to me, and I'd love to see the project keep going as a testbed for how to make similar projects succeed in future.

But, like most volunteer projects, it seems to be slowly sinking beneath the waves. I don't know what, if anything, could change that. A $50,000 grant from ACX? Providing an easier on-ramp for new volunteers? Some kind of Y Combinator for rationalist projects?

Comment by Tornus on San Francisco shares COVID data only when it's too late · 2021-12-27T15:12:28.158Z · LW · GW

I can't speak to San Francisco specifically. But if it's anything like many other locations in the US, the problem isn't malice or indifference: it's that generating this data is vastly harder than you realize. The politicians get the data the same time you do: as soon as it's ready.

Here's one tiny true example, from one part of the pipeline in one particular location. A substantial amount of data enters the system as faxes. The faxes go to a room full of National Guard, who manually enter the data into computers, from whence it begins a complicated process of validation and de-duplication before it enters the main pipeline. You can imagine that this system doesn't scale particularly well as case counts rise.

At a broad scale, what's happening is that an immense amount of data is trying to enter a legacy system that was designed for less than one percent of its current load. Some of the data comes from sleek modern hospitals with state of the art medical informatics systems. And some comes from computer illiterate rural doctors, and some comes from nursing homes that had never reported lab results before Covid, and some comes from employers who test their employees, and some comes from private labs, and some comes from sovereign tribes that have complicated data sharing agreements with the state, and...

If I can find the time, I might write a post explaining in more detail how surveillance data is generated and processed. But for now, I assure you this problem is incredibly hard. Update: here's the post

Important disclaimer: my opinions are mine alone and I don't speak for any government agency.

Comment by Tornus on How much Bayesian evidence from rapid antigen and PCR tests? · 2021-11-24T14:37:19.625Z · LW · GW

Rapid antigen tests at the door reduce risk by about 75%, assuming people are asymptomatic and you test each day if it's a multi-day event. I did a deep dive on antigen tests recently, if you'd like to see the data.

PCR is probably similar: they're much more accurate, but the data is more stale, which especially with Delta is a significant issue.

Comment by Tornus on Does it make sense to get 2 flu vaccines? · 2021-11-06T23:29:39.203Z · LW · GW

That is precisely the question, and I confess that I don't know the answer for certain. I think, though, that both factors are important.

The issue you're talking about is definitely a thing: influenza evolves rapidly enough that any given vaccine will become less effective over time simply because the dominant strain of the virus has drifted.

However, I believe it is also the case that the immune response drops off fairly quickly. I haven't found a definitive source (I confess that I didn't look hard), but the closest I came is this article, with this quote:

"My informal sense of the literature [is] that the suggestion is strong enough that if people could reliably get vaccinated the week or two before the flu season starts, they'd be better protected," Marc Lipsitch, PhD, a professor of epidemiology at Harvard University, told CIDRAP News. Lipsitch also penned a commentary on this study. "The more complicated thing is the trade-off between putting it off and not doing it at all," he said.

My interpretation of that is that he's talking about a benefit from getting the identical vaccine closer to the start of flu season, so that flu season hits while the immune system is at maximum activation.

Comment by Tornus on Does it make sense to get 2 flu vaccines? · 2021-11-03T13:45:54.988Z · LW · GW

That makes sense—it's also true that the efficacy of the flu shot declines over time (maybe 8% - 10% per month?), so there is significant concern about getting it too early. I could certainly see making an argument for getting one as soon as possible and a booster shot in the mid to late season. That's a single shot with a booster, technically, not a two shot series.

Comment by Tornus on Vaccine Requirements, Age, and Fairness · 2021-11-02T19:55:48.003Z · LW · GW

Taking the "wait" argument to its logical extreme, it seems to me one could argue not only waiting for kids to get vaccinated, but waiting until COVID rates are minimal, so immune compromised people can safely attend.

I don't think it's necessary or appropriate to take everything to its logical extreme, but it seems to me that if one is going to advocate waiting for one group but not another, it's important to clearly articulate the moral principle behind that distinction.

I'm not a dancer, but my instinct is that a "reasonable accommodation" model is appropriate here: there's a moral imperative to make events as accessible as reasonably possible, but not to cancel any event that isn't 100% accessible to every person.

Comment by Tornus on Does it make sense to get 2 flu vaccines? · 2021-11-02T04:22:49.999Z · LW · GW

I would doubt it—different vaccines provoke different immune responses, and each has a dosing schedule based on empirical evidence about what produces the best response. The fact that two doses are needed for an optimal response from the covid vaccine doesn't tell you much about any other vaccine.

It's possible that two vaccines would produce a slightly better response but they decided the cost/benefit didn't pencil out, and I could imagine that for some immune compromised people, getting two would be appropriate. But I'd stick to the recommended schedule absent strong reasons for doing otherwise.

Caveat: I have a strong layperson's understanding of vaccines, but I haven't looked at data specifically for the flu vaccine.

Comment by Tornus on Comparing Options For Safer Events · 2021-11-01T17:29:16.594Z · LW · GW

Really solid analysis. Regarding rapid tests:

A pretty important downside in many cases is that they're logistically complicated at a large event. The tests need to lie flat on a table or other surface for 15 minutes. Are you gonna have a giant table covered in tests? Do people come in to test and then go back out? Do they take their test out and perform it in their cars? These are solvable problems, but they can add a lot of complexity and crowding to the checkin area, which is already a problem spot at many events.

With that said, I'm a huge fan of tests for smaller events. Rapid tests let you get four times as much socializing for the same level of risk—depending on your risk budget, your financial situation, and your social ambitions, tests might (or might not) be a game changer for you.

Comment by Tornus on Indoor dancing is safe enough · 2021-10-31T15:20:51.105Z · LW · GW

OK, did some digging. The relevant source is table 2 from Peng et. al., Practical Indicators for Risk of Airborne Transmission in Shared Indoor Environments and their Application to COVID-19 Outbreaks.

They calculate the following relative risk rates:
Silent: 0.0012 (1x silent rate)
Speaking: 0.0058 (4.98x silent rate)
Shouting / singing: 0.0350 (29.91x silent rate)
Heavy exercise: 0.0817 (69.83x silent rate)

IMHO, you could may argue for a risk factor of 1/10 compared to heavy exercise (which is 7x the silent rate), but my gut is that 1/5 (14x the silent rate) would be more likely, and something like 1/3 (23x the silent rate) would be a better and more conservative choice

Comment by Tornus on Indoor dancing is safe enough · 2021-10-31T14:00:38.912Z · LW · GW

If you can afford them, the rapid tests are a great idea: microCOVID doesn't model them, but I believe they cut your risk by about a factor of 4.

Comment by Tornus on Indoor dancing is safe enough · 2021-10-31T13:52:27.392Z · LW · GW

Yes, I emphatically agree with this (as does my consultant, who is an epidemiologist who works on COVID full time).

I think one can reasonably argue about the details: loud vocalizations create different aerosol patterns than exertion, and off the top of my head I'm not aware of any really solid data on how the two would compare. But I think your numbers are low by at least a factor of 5, and a factor of 25 seems very plausible to me.

Also: you've selected surgical masks when doing the µCoV calculation. Will that actually be true? If most people wear cloth, thin or loose (which seems most typical here), that'll increase the risk by another factor of 4.

Comment by Tornus on Effectiveness of Moderna booster shot and mix-and-match? · 2021-10-26T14:23:06.210Z · LW · GW

Excellent data: thank you! Two things to keep in mind:

  1. The comment on page 5: the study was "Not powered or designed to compare between the groups"
  2. They're only looking at antibody levels (because those are relatively easy to measure), but there's a good argument that some of the differences between strategies will involve activation of B cells & T cells.

See also the limitations on page 33.

Comment by Tornus on Effectiveness of Moderna booster shot and mix-and-match? · 2021-10-26T14:06:29.239Z · LW · GW

Excellent question, and I think a lot of us are wishing we had more data on this—unfortunately, there is very little data so far. But here's my take:

  1. If you had J & J for your first shot, I think there's enough evidence now to say it's probably (p = 0.7?) better to get Pfizer or Moderna for your booster.
  2. If you had Pfizer / Moderna for your first two shots, my instinct is that J & J might be the better choice, because there's an argument from microbiology that mixing types might produce a more robust response.
  3. If you had Pfizer / Moderna and want an mRNA shot for your booster, I don't think it'll make much difference which one you get: they're very similar.

There's an argument to be made that absent strong reasons to do otherwise, it's best to follow standard practice (in this case, to get the same brand of booster as the original shots) simply because you'll be in a larger, better-studied cohort.

A couple of sources, such as they are:

“But something has really become clear: The mixing really is most impactful when you have a DNA/adenovirus vaccine first followed by the mRNA vaccine,” Gandhi said. WaPo

The study’s researchers warned against using the findings to conclude that any one combination of vaccines was better. The study “was not powered or designed to compare between groups,” said Dr. Kirsten E. Lyke, a professor at the University of Maryland School of Medicine, who presented the data. NYTimes

Comment by Tornus on How much should you update on a COVID test result? · 2021-10-18T15:50:48.919Z · LW · GW

Yes, accuracy in antigen tests seems to correlate very strongly with viral load (and presumably therefore with infectivity). This paper found 100% agreement with PCR for Ct 13-19.9 (massive viral load), all the way down to 8% agreement for Ct 30-35.

Ct (cycle time) measures how many amplification cycles were needed to detect nucleic acid. Lower Ct values indicate exponentially more nucleic acid than higher values, although Ct values are not standardized and can't be directly compared between testing facilities.

Comment by Tornus on How much should you update on a COVID test result? · 2021-10-18T15:41:42.767Z · LW · GW

Thank you for this! I have a few thoughts about antigen tests.

1: I'd recommend the BinaxNOW as the "standard" home antigen test in the US. Broadly speaking it's better studied, more accurate, cheaper, and more widely available than the others. Regarding data...

2: I think the best current source of general data on home antigen tests is this meta analysis from September. The results from multiple papers over the last year have been pretty consistent, but this adds a little more power to the numbers. They come up with:

Overall: sensitivity 68%, specificity 99-10%. Sensitivity for symptomatic individuals: 72%. Sensitivity for asymptomatic individuals: 52%

Sensitivity for Ct < 25: 94%, Ct > 3: 30%. (I'll be writing more about these results in a bit, but the short version is that this strongly supports the belief that test sensitivity depends strongly on viral load and will be highest during peak infectivity).

3: Two additional excellent papers are this one for subgroup analysis and this one for subgroup analysis and discussion of how user error affects accuracy.

4: Related to the above: accuracy seems strongly correlated with viral load, which strongly suggests multiple tests on the same individual at the same time would be highly correlated.

Comment by Tornus on Interpreting Diagnostic Tests · 2021-10-15T14:29:42.545Z · LW · GW

Yes, those are all excellent points.

I wrote this as a side reference for a deep dive on the BinaxNOW that's coming shortly, and it'll dig into the numerous, complex, and important issues affecting BinaxNOW accuracy. Short version: the accuracy varies substantially, largely based on viral load. And you're correct that repeated tests on the same individual will be strongly correlated.

And you've convinced me to change the example you cite: I'd gone with the first person for narrative consistency, but I'm shifting it to prioritize technical accuracy.