Covid 2/18: Vaccines Still Work

post by Zvi · 2021-02-18T12:30:00.789Z · LW · GW · 29 comments

Contents

  The Numbers
  Predictions
  Deaths
  Positive Tests
  Test Counts
  Vaccinations
  Europe
  Covid Machine Learning Project
  The English Strain
  The South African Strain
  #YouHadOneJob (to Appoint Someone To Fill)
  Vaccines Work
  Practical Model of Vaccine Effectiveness
  Vaccine Work So Well We Can Vaccinate More People
  Bubble Bursting
  No, Really, Andrew Cuomo Is The Worst
  New York Times Is Permanently Banned 
  Scott Alexander Predictions on Covid-19
  Metaculus Redux
  Vitamin D As It Ought To Be
  In Other News
None
29 comments

This week the CDC released new guidelines for schools. I’ve spun my analysis of that off into its own post. Scott Alexander also shared some good thoughts on Covid-19 in two new posts, and I discuss both of those, and how our models and predictions differ.

Also, as some combination of retaliation and its continued crusade against the evils of the tech industry and the notion of freedom of speech, the New York Times finally published its hit piece on Scott Alexander. I devote a brief section to it, the upside of which is that the paper is now permanently banned from this space. 

Oh, right, you came here to learn about Covid-19 news. The short news is great. Infections and deaths are down much faster than anticipated. The vaccine news is less great. The process continues to accelerate, but that acceleration was slower than expected, and there’s still lots of FUD about what vaccinated people can and can’t ‘safely’ do and little movement yet towards saner policy in other ways, although there are signs of moving in that direction. The new strains aren’t here in force yet, but it seems likely that will happen soon.

One potentially big thing we learned is the study from infected NBA players, which suggests that the English strain causes infections to play out over a longer period of time than the classic strain, with all the implications of that.
 

Another was that it got even clearer than it already was that the Pfizer and Moderna vaccines are highly effective after only one dose, and that First Doses First would be a vast improvement while we continue to have much less vaccine than we need.

The Numbers

Predictions

Results last week: 6.4% positive rate on 11.3 million tests, and an average of 2,752 deaths after adjusting for Indiana.

Prediction: 5.7% positive rate and an average of 2,450 deaths. Things should still continue to improve.

Result: 5.2% positive test rate on 10.4 million tests, and an average of 2,089 deaths. This is after removing Ohio’s adjustments that add additional deaths from previous months, similar to Indiana’s adjustments from last week. 

I’m straight up adjusting the results rather than listing both answers, because the adjustment makes my prediction less accurate. When adjustments like this make predictions more accurate, it seems necessary to list both results to keep oneself honest.

This is an extraordinarily good result. After several weeks of refusing to budge, deaths plummeted, and the drop in infections continues to be rapid. With a 20%+ week over week decline, we still aren’t fully ready for the English Strain, but it’s a great start and buys us time. 

This will be the first week that I am adjusting my prediction a bit to account for risk that the English Strain is starting to take over, but it won’t happen all at once. It would be very surprising not to see a continued drop.

Prediction for next week: 4.6% positive test rate and an average of 1,800 deaths.

Deaths

Date	WEST	MIDWEST	SOUTH	NORTHEAST
Dec 17-Dec 23	3826	5158	5131	3772
Dec 24-Dec 30	3363	3668	4171	3640
Dec 31-Jan 6	4553	4127	5019	4162
Jan 7-Jan 13	6280	3963	7383	4752
Jan 14-Jan 20	5249	3386	7207	4370
Jan 21-Jan 27	6281	3217	8151	4222
Jan 28-Feb 3	5524	3078	8071	3410
Feb 4-Feb 10	4937	2687	7165	3429
Feb 11-Feb 17	3837	2221	5239	2700

Once we take out the adjustment in Ohio, we see steady advancement across the board. 

Positive Tests

Date	WEST	MIDWEST	SOUTH	NORTHEAST
Jan 7-Jan 13	474,002	262,520	531,046	306,604
Jan 14-Jan 20	360,874	185,412	452,092	250,439
Jan 21-Jan 27	260,180	158,737	386,725	219,817
Jan 28-Feb 3	191,804	122,259	352,018	174,569
Feb 4-Feb 10	144,902	99,451	255,256	149,063
Feb 11-Feb 17	97,894	73,713	185,765	125,773

Once again improvement is steady all around, although slower in the Northeast and especially slower in New York, which I’m personally not too thrilled about. 

Test Counts

Date	USA tests	Positive %	NY tests	Positive %	Cumulative Positives
Dec 24-Dec 30	11,300,924	11.2%	1,303,286	6.0%	5.98%
Dec 31-Jan 6	11,649,640	13.3%	1,365,473	7.3%	6.45%
Jan 7-Jan 13	13,911,529	12.2%	1,697,034	6.6%	6.97%
Jan 14-Jan 20	14,005,720	9.7%	1,721,440	5.9%	7.39%
Jan 21-Jan 27	12,801,271	8.8%	1,679,399	5.3%	7.73%
Jan 28-Feb 3	12,257,123	7.7%	1,557,550	4.6%	8.02%
Feb 4-Feb 10	11,376,541	6.4%	1,473,454	4.1%	8.25%
Feb 11-Feb 17	10,404,504	5.2%	1,552,555	3.5%	8.41%

As a point of comparison, these rates are similar to what we saw late in October. 

Also worth noting that we’ve never seen a rolling 7-day average below 4%, so we could get that to an all-time low within the next two weeks. Exciting stuff.

Vaccinations

This was a deeply disappointing week on the vaccine front, with the 7-day average declining over the last few days, and on net almost no progress week over week. Our rate of progress has stalled out. Weather and the holiday I am sure contributed to this, and increased supply is still in our future, but it’s still disheartening. 

Europe

Overall news is good, but nowhere near as good as in America.

Covid Machine Learning Project

As with other sources, the infection numbers look good but the vaccine numbers are disappointing. He has us at 27.7% infected as of February 4, versus 27.2% a week earlier. 

The English Strain

The NBA once again delivers us reliable data on the things we care about.

Clustering means that this can’t be presumed to be a random sample, but 7/65 for December and January is such bad news it could be seen as potentially turning the corner into becoming good news again. If we were at 10%+ new strain over that period, then that means at least 20% by the end of January, and it’s now two weeks later than that which should take us at least close to 50%. 

If we interpret that as the new strain about to become impactful and take over, it’s bad news. But, if we interpret it as the new strain is already here in force then it gets more ambiguous and could even be good news! If the new strain is 25%, or even 50%, of new cases, and cases are still going down, then that’s amazing. It means we’ve somehow managed to do so much additional prevention work that we’re still making rapid progress. So that would be pretty great. 

The timing of the transition between strain, however, wasn’t what was being studied here at all. Here’s the paper. Instead, the paper is studying the timing for when people are infectious with each strain, using the frequent NBA testing to generate a robust data set. That’s pretty great, since it informs all sorts of decision making.

These charts seem super useful, with red being the English strain and blue being classic:

Note, before we go any further, that these are in some ways highly atypical cases. NBA players are not representative of the population, and these could be highly correlated in various ways. It’s entirely possible that these results don’t mean much because of those issues, and it would be super nice if we could run similar tests on a population that wasn’t so exceptional and unusual.

Together, these charts tell a story of the typical infected person with the classic strain being acutely infectious for a brief period, whereas the new strain makes them sick and infectious over a longer period. This would then presumably also be the reason the new strain is deadlier, although I don’t have a good understanding of why it has that effect and wouldn’t have been surprised if this pattern had gone along with a less deadly strain rather than a more deadly strain. 

This means that the infection cycle (the serial interval) of the English strain is longer than that of the old strain. That means that it will grow slower, relative to its increased infectiousness, than the old strain. If we weren’t doing any mitigation at all, it’s possible that the English strain could have a higher R0 than the old one yet end up not becoming (at least at first) the dominant strain, because the old one would grow faster. This means that we should expect the English strain to arrive in numbers somewhat slower than its level of infectiousness would otherwise indicate, buying us more time to prepare, but also means that if the strain arrives in force at a given time, we should estimate it being a bigger jump in infectiousness than we would have previously since it will have had fewer cycles in which to get that far.

This also means that if the English strain is a major factor, and this data is accurate, quarantine periods will have to get longer. Whatever was being done before won’t be enough, and anything that would be enough now was previously overkill. 

There also won’t be one number to represent the increase in infectiousness, because the relative R0 will differ based on people’s behaviors. 

Due to the origin of the sample take all this with a bunch of salt, but it’s a bunch of interesting updates. 

The South African Strain

It is clear that antibodies from our current vaccines are less effective against the South African strain. The question is how much less effective, and what that translates to in practice in terms of protection. When one shows antibody response levels, it is traditional for a reason to use a logarithmic chart. The immune system doesn’t produce exactly enough antibodies, it does its best to massive overkill when that makes sense.

Also, there’s the continuous risk that one will confuse the different measures of ‘effectiveness’ and reach a nonsense conclusion by doing math that isn’t valid. I worry about the interpretation of things like this but trust my readers to handle it: 

There are plenty of people who read that and interpret it very incorrectly as ‘whelp, we were 95% protected, but this is two thirds less protection, so that’s about 1/3 protected.’ That would be rather alarming, but it’s not the case. Cutting antibody effectiveness by 2/3rds is going to reduce vaccine protection, and the ‘no evidence’ statement here is obvious nonsense the way such statements usually are, but we also have no reason to expect a 2/3rds drop in antibody protection levels to translate to anything like a 2/3rds drop in how effective the vaccine will be at preventing infection or death. 

Here’s the chart from the preliminary report, click through to get their explanations:

#YouHadOneJob (to Appoint Someone To Fill)

President Biden continues not to have named his new head of the FDA (WaPo). The two top candidates continue to be Janet Woodcock and Josh Sharfstein. 

It seems from a brief investigation that Sharfstein’s primary mission would be to stop evil drug companies from potentially earning a profit. Such villains would need to be stopped immediately. On the plus side, he has earned the ire of anti-vaccine groups, and that has to count for something.

Janet Woodcock is the current acting commissioner, and thus directly responsible for our failure to approve vaccines. You gotta love this being front and center on her Wikipedia page: 

In the context of opioids that certainly does not seem great, but we do need that same energy now in other matters, and also there’s this:

Not that it’s working all that well, or anything, based on what we observe, but that doesn’t mean it couldn’t be much worse. At least Janet Woodcock is moving on the margin to produce more innovation and get things to move faster, as opposed to stopping things and slowing them down. So if those are the only choices, I’d have no choice but to go with Janet Woodcock.

Vaccines Work

(And yes, you should know this already.)

Somehow, we constantly get articles and new studies that are framed as surprising, saying that yes, the vaccines that we know work, work. 

And thus, ‘No evidence’ for the obvious conclusion all the Bayesians reached weeks or months ago becomes evidence that counts in newspapers and journals. 

Which is important! Even when one trusts, one should verify. Even if one did not need to verify, one can still learn more and in more detail. Every little bit helps.

It’s best not to get too worked up about the failure to use words like ‘confirms’ and instead call such results ‘game-changing.’ What matters now is that our beliefs, and what we’re willing to do with those beliefs and say out loud, have now converged. 

Yet, even when the write-ups clearly point to the correct response (first doses first), it still pains me to see things like this…

I say this points to First Doses First because otherwise this isn’t a game-changer in pursuit of herd immunity. If you always give everyone their second dose two weeks after the first, this new finding at most puts you two weeks ahead of schedule. The only real ‘game-changer’ would be to use this to do First Doses First.

(In good news, there’s reports that the CDC is in technically confidential discussions about recommending a move to First Doses First.)

And here we are, back to Very Serious Person framing, complete with ‘may’ and ‘partial’:

Another similar article on the topic is here, with the same framing that this is news.

The problem is, we all want to directly measure infectiousness, since it’s genuinely unclear exactly how much protection against this various vaccines offer at various points:

But how are you going to do that? The only practical option that would actually work, that I can think of, would be a challenge trial, which is super illegal, so the next best thing anyone’s come up with so far is what this study did, which was to measure viral load in the infected. 

The result of 75% lower average viral loads in vaccinated people, conditional on infection at all, reinforces our previous understanding. First, it confirms that, conditional on infection at all, vaccinated people are less infectious even after one dose. Second, it confirms that severe disease, hospitalization and death should be much reduced in vaccinated individuals, again after only one dose, even conditional on infection. 

Since one dose also does a remarkably strong job of preventing infection, we can now multiply these effects. The only downside is that, with less viral load and less serious infection, those who are vaccinated are more likely to have a case and be undetected, so the “% effective” number on infected-at-all is going to be lower than the “% effective” number measured by those who test positive. Testing often happens in response to a reason, and also sufficiently light cases generate more (technically false) negatives.

Practical Model of Vaccine Effectiveness

Basically, my current model is that the vaccines all improve your chances at each step of the illness, cumulatively. This fits the data and also my understanding of immunology.

Think of being infected kind of as: 

1. First you get exposed and you roll to see if you get infected at all.

2. Conditional on getting infected at all, roll to see if you get mild symptoms. 

3. Conditional on getting mild symptoms, roll to see if you get moderate I’m-sick-this-sucks level symptoms.

4. Conditional on getting moderate symptoms, roll to see if you get a severe case.

5. Conditional on severe, roll to see if you need hospitalization.

6. Conditional on hospitalization, roll to see if you die.

If you pass a roll, you recover, and then you roll again based on how bad it was to see if you get some form of Long Covid. The further down the chart you get before you stop, the more infectious you are for longer, ratios unknown. And the worse your case, the higher the viral load you pass to someone you infect, which makes their chances at each stage worse as well.

Being young and healthy helps you at each step, and so does vaccination, as does being exposed in step 1 to a low initial viral load, with the difference that vaccination helps relatively more at steps 1-2, and being young and healthy is relatively more effective at stages 3-6, and I don’t know where viral load helps more, but it all helps at least some at all steps.

When we measure vaccine effectiveness, different studies are measuring different places on the chart – most are checking how many people test positive as primary endpoint, which is closest to #3, but others test everyone periodically, which is more like a mix of #1 and #2 (if you’re barely positive, the test won’t be positive for as long and might miss it entirely). We also measure #5 and #6 and report those, because people notice they’re important.

What we notice in every vaccine study is that #6 comes out to 100% protection, and #5 at least comes damn close. Two weeks ago there were 5 vaccine studies released, quite large, with zero hospitalizations and zero deaths in any of the treatment groups. As far as I have seen, for every vaccine anyone is considering offering to anyone, there has not been a single death from Covid-19 in the treatment group of any vaccine trial. 

Call the dominant strain in America the ‘Classic’ strain to contrast it with the South African, Brazillian or English strains. 

My current prior on Pfizer/Moderna vs. Classic is that they’re something like 90% vs. infectiousness (step #2 or so), 95% vs. moderate symptoms (#4), 99%+ versus death. To the point where we’ve fully vaccinated several percent of the population across many countries and have one report of a fully vaccinated person dying of Covid-19, versus what would otherwise be over a hundred per day. 

Similarly, J&J was measured at 66-72% vs. Classic as the headline number, but is clearly 95%+ and plausibly 99%+ against hospitalization and death, we don’t know the full number because again zero cases in the treatment group. 

The other vaccines follow similar paths. They all offer strong protection versus hospitalization and death.

Even in the worst case, where the AZ vaccine is not so effective against the SA strain in terms of infection and transmission, it’s more effective than nothing at all even against infection and transmission, doesn’t block getting a second vaccine later, and it would still prevent a lot of deaths. So the SA decision to not use AZ only makes sense if they can trade their AZ supply for someone else’s supply of a different vaccine, which could still be a win/win.

You want to know how effective the Pfizer vaccine is? This effective:

It’s too early to draw firm conclusions about impact on the IFR (infection fatality rate) from this alone, since my prior for the baseline number is something like 0.2%-0.6%, which means we’d expect 1-3 deaths by now if they were detecting every case from 544 infections, or a few times that if we assume the majority of infections continue to be missed. The hospitalization rate is similar to the overall USA hospitalization rate for cases, but the American death rate is about half our hospitalization rate, so even if hospitalized but vaccinated patients are somewhat less likely to die than hospitalized unvaccinated patients, I’m guessing the hospitalization threshold in Israel is lower. 

Also worth remembering that Israel focused on vaccinating its elderly. 

Winning all fifteen coin flips does happen by accident, but rarely on one’s first attempt. Combine that with winning all the coin flips in all the vaccine trials. 

Or to put it another way, vaccines are so good that mild cases after vaccination are clickbait news items (hence I won’t direct link):

Or, in summary, this:

This study covers our own results in practice, and also finds the vaccines highly effective (pdf). Numbers here were actually somewhat disappointing compared to what we’ve seen elsewhere. Sharing the chart below because the data is the thing, but note that these are very small samples in terms of number of infections in many cases, so don’t take the exact numbers all that seriously.

Once again wasn’t sufficiently powered to measure everything we want, but once again reduced severe outcomes, and a zero on deaths.

Vaccine Work So Well We Can Vaccinate More People

Shout it from the rooftops (study). Moderna half doses illicit robust immune response in healthy adults, including those over the age of 55.  

This is a huge boost to our vaccine supply, if only we are willing to take it. Given the physical mechanisms it isn’t quite a doubling of Moderna doses, but it’s close. 

A letter to the editor reanalyzing the Pfizer data on the first dose only. 

I wouldn’t want to hang my hat on a sample of size n=29, but the pattern keeps repeating itself, and the math isn’t remotely close. 

There is no lower hanging fruit available than this.  

The logistical hurdles are many. What would it mean if we put so many doses into each vial? Well, good news, the FDA only took a few weeks to give their sign-off on that one:

It’s pretty great when there’s a painfully obvious thing to do, you hold it up for weeks, then you allow it and everyone gives you credit for the huge boost provided by your approval.
 

Celebrate, good times, come on, also look at those reporters doing math:

Presumably the 20% number comes from a 40% increase in Moderna vaccine doses, and Moderna being about half of our supply. That implies that there will still be a shortage of vials, and that if we could find a way to produce more vials we could deliver more vaccine doses. We may not be quite the can-do country we used to be, but ‘produce more vials faster’ still seems like an ask that should be within our reach. 

In good news, France is recommending only one dose for those previously infected.

The data seems very clear that if you’ve already had Covid you need at most one vaccine dose.

Linked studies are here, here and here. The real question is whether such a person needs the first dose of vaccine, or whether the prior infection was sufficient. As I told a commenter last week, my position on that is that there’s no real downside to getting the first dose so you should get it, but I wouldn’t fight for a scarce appointment and go way out of my way if I’d already been infected, and for now let them go to those who need them more.
 

A proposal to change our method of vaccine administration. Huge if true and implemented, definitely won’t be implemented in time to matter even if true. Thus did not do the research to determine if true, but good brainstorming so figured I’d pass along.

Bubble Bursting

What to make of this story from the conference organized by Peter Diamandis? 

I’d encourage comparison to this analysis of what a Covid test tells you, at Aceso Under Glass. 

Diamandis attempted to create an immunity bubble via PCR testing.

Masks were encouraged between venues, but not required. 

There was one positive test prior to day one. That person was not admitted, and participated virtually. All other test results were negative, but afterwards…

If you go from testing negative one day to highly infectious the next day, at least one of those tests almost certainly gave the wrong answer, even if you’re only or primarily interested in infectiousness.

He links to this article, noting that people with Type B or AB blood are more likely to test positive than those with Type O blood but not more likely to get sick, and that he is Type O. But a lot of people are Type O, so interpreting this as ‘tests don’t work on Type O’ would be saying they don’t work on almost half the population, which unless you knew a particular person’s blood type would cash out as straight up ‘doesn’t work.’ 

Sounds like everyone is going to recover, which is great. Concrete specific data is always highly useful, and it’s great to have a detailed report. What can we conclude from this?

Masks work, and not using masks was a mistake, but I think the mask data here is being misinterpreted. If one group was isolated in one area and none of them got Covid, and another group was in a different area and many of them did get Covid, the presumed explanation is that the isolated group, even if large, didn’t have any Covid cases to begin with, and they wouldn’t have caught Covid even without masks. Again, that doesn’t make the masks unnecessary or not a good idea, but it could easily have gone the other way.

Mostly what I see is a badly designed protocol. 

You can’t have a bubble without an initial quarantine period.

Tests are great for reducing risk. Testing at the rate this group tested, if practiced by the general population, would end the pandemic in a month or two.

But the requirement for a bubble isn’t reducing risk. The goal is to all but eliminate risk, and this protocol obviously didn’t do that given the timing involved. Rapid tests don’t reliably test positive before symptom onset. 

A 14-day quarantine period before starting is definitely sufficient. A 7-day period before starting, combined with multiple tests on everyone, would in practice be good enough for a bubble of this size. A 0-day quarantine doesn’t make any sense and won’t get it done. 

This stuff is complicated, and people are not explaining it well. As a result, even super smart people like Peter Diamandis are often getting it importantly wrong. 

Again, that doesn’t mean that what they did wasn’t highly useful. The protocol helped by catching one infected person, and on average I would expect it to stop a large majority of potential outbreaks in such a group. Missing someone (or multiple someones) required having bad luck. 

What it doesn’t do, under current conditions, is provide enough safety to form a true bubble where people can ignore transmission risk between those in the bubble. It doesn’t let you ignore masks, distancing and ventilation entirely. 

No, Really, Andrew Cuomo Is The Worst

The Washington Post is among many calling for the removal of Cuomo from office. It seems clear now that not only did Cuomo put Covid-19 positive nursing home residents back into nursing homes where they could infect other residents, he also covered up the resulting deaths. 

New York Times Is Permanently Banned 

It is a great sorrow to see the paper I grew up thinking of as the paper of record, with the motto “All The News That’s Fit To Print,” engage in such brazen libel and retaliation, insinuation of guilt by association so brazen it should be used as an example in textbooks, and condemnation of the very idea of freedom of speech. 

Due to its recent remarkably disingenuous retaliatory hatchet job hit piece on Scott Alexander, together with the paper’s growing general amount of malice (as Eliezer points out it’s not obvious that this piece was even unusually disingenuous for the NYT), The New York Times is now permanently banned from my blog. There will be no links of any kind to NYT and I will not speak to anyone associated with the paper or provide them any information for any reason, other than to inform them of this decision and its cause. I will share information from NYT sources if there is no alternative, without linking.

As further proof of its principled opposition to free speech, The New York Times took this position to its logical conclusion, and also warned us this week about the nefarious Clubhouse and its dangerous potential for ‘unfettered conversation.’

If you’re reading this, you almost certainly read Scott (if not, that’s a serious mistake that you should fix, so go do that now, I’ll wait.) Remember that this is how they conduct themselves in the arena you know. Why would you think they do any differently elsewhere?

For now, I am not extending this ban to avoiding information from individual NYT reporters on Twitter. A remarkably large number of people providing information on Twitter turn out when you check to be NYT reporters. I could be persuaded that I should take this step, and block all of them on principle. 

Beyond that, and urging everyone else to follow suit (or at least to cancel any subscriptions and not talk to their reporters) on pain of me thinking less of you, I consider the matter closed.  

Scott Alexander Predictions on Covid-19

Scott Alexander is out with a post on Covid-19. I provided feedback on an earlier draft, and think both versions were quite good. When writing on a weekly basis, it’s easy to forget to step back and look at the longer term picture more broadly. I’ve done it somewhat, but haven’t focused enough on the longer term, so this is a welcome reminder. Scott is doing what so many do not, creating a physical world model full of gears, trying to figure out how those gears function, and thus figure out how things work and what is likely to happen. Then he even makes probabilistic predictions. It’s what you’d expect from the consensus number one pick. 

Let’s take a look at the predictions:

The threshold for this prediction is not so high. Right now we are more than 50% down from peak in terms of positive tests and hospitalizations, so this wouldn’t require us to get anywhere near the previous peak. The prediction also requires that the new peak be in March or April; technically if it’s in May then this resolves to no, no matter how high the peak. I still expect this to happen, because math, because the threshold is low, and because while we’ve exceeded expectations in vaccination we likely haven’t exceeded them by enough to make a March deadline once you account for all the lags.

The most promising thing is that the fourth-wave narrative has  become fully accepted by the Very Serious People in advance, which is plausibly why we’re seeing such strong declines in cases now – the control system may be partially working to act on the future rate of growth rather than the current one.

With us at R0~0.83 at the moment, we’re definitely not there yet, but we’re not that far from there especially if the English strain isn’t fully 50%+ more infectious and is only 30-40% more infectious.

Combine all those effects including the timing involved, and I’m at something like 60% here if measured in the 7-day average positive test count. 

Given this says ‘or new strains with similar dynamics’ this seems hard to avoid. A majority could end up being not that many cases, but that doesn’t change what we have to do to prevent case growth, nor does it change the odds here. Our containment plans for the new strains are terrible, and we’re seeing convergent evolution towards things that would count. I think this is low and I would be closer to 80%. A full year is a long time.

I’m interpreting this purely on an individual risk level. I worry that our knowledge generation systems and our vaccine approval systems are so broken that we’ll never ‘know’ for sure the way this question implies we will know, but let’s say that this question is asking about our best guess looking back on 2021. I am much more optimistic here. AstraZeneca is the one that might fail this test against some strains, but if I hadn’t seen Scott’s number I’d have been at least around 85% for this, because 80% reduction isn’t that high a threshold. Given Scott looked and said 55%, I’ll revise that down to 70% on the basis of there being multiple ‘all’ clauses in this, and it being dangerous to get too optimistic about claims that are worded like that. 

Note that this threshold is an order of magnitude or more stronger than the one that says vaccinate anyone anywhere with anything you can get, at any existing price, and I predict that all vaccinations listed here will have been worth getting for the general population regardless of which strains are dominant with a probability of 95%, except insofar as it prevents people from being given access to another vaccine soon thereafter (e.g. if you can get Moderna next week or AZ right now, but the government won’t let you have both, it makes sense to wait). The remaining 5% mostly involves the vaccines somehow interfering with each other and some being vastly better than others plus widely available relatively quickly.

I’m interpreting this prediction as being agnostic as to the framing involved, and assuming it resolves to Yes if and only if the CDC issues guidelines saying many of those previously vaccinated should get an additional shot some time in 2022. With that wording, I’d be around 50% for this, so rough agreement. The difference is more that I don’t think this is that likely to portent a big problem, and more likely to be a similar annoyance to the flu shot, where it’s totally worth doing but not a big deal for healthy people if you end up forgetting.

The long term mutation rate of Covid-19, and how impactful those future mutations are likely to be, is unknown, although we can guess. I agree with Scott that this is the big physical unknown for long run outcomes, but within the range of plausible answers I think our choice between plausible decisions as a civilization matters far more. 

The link Scott provides here is very welcome news. The FDA seems to be on the verge of getting this one at least mostly right. 

The probability here depends on exactly how formal this has to be. If it needs to be an explicit announcement of such a lane for a particular company or group of companies, then I think 50% is a good prediction. If all that’s required is a general understanding by Pfizer or Moderna and those observing them that they could get approval within three months in a future emergency, then I think this is closer to 80%. 

I’m optimistic here because there’s an existing example of this process (the flu) that everyone agrees upon, and because this involves essentially repeating a similar thing to an already done thing, which is what organizations like the FDA deal with best, and also because of how terrible it would look (and be) to hold this up in that situation were it to arise.

I’m going to be less confident on this prediction, and bring it down to 80%, because 10% is not a lot of people, and it wouldn’t take that big a cultural shift to exceed that rate of mask wearing semi-permanently even if there wasn’t a strong physical need for it.

The core question to be decided is, are we going to emerge with a new appreciation for life into a new roaring 20s, or are we going to do to adults what we’ve already done to childhood, and extend life-crippling restrictions indefinitely because of a recalibration of risk and new cultural norms? Will we bring another promise to Mother Earth with a bounty and mercy, or will we hide from her within walls of stone at the barrel of a gun?

It’s great that moving forward we’ll have improved our infrastructure for providing remote services and doing video calls and such, but it would be a supreme tragedy if we let our way of life permanently end in a new wave of disease paranoia. That outcome seems entirely plausible but also entirely preventable. 

There will be rock concerts and music festivals once again. There will be more than there have ever been, because there have to be.

You Gotta Fight For Your Right to Party.

All your other rights as well. Covid could well end up being used as a framework for the denial of our basic rights. If elites can break the rules in an emergency they will create an emergency in order to break the rules, and thank you for coming to this Ted Talk. It could also do the opposite, and embolden people to not listen to elites the next time they try something like this, given their performance record. 

We need to go several steps further than reversion to the old norms. As I discussed in the School Daze section, we need to take this opportunity to undo previously instituted insane safety concern trolling, and let life exist. Covid-19 safety is a group effort where we must contain spread, so safety concerns are at least understandable, but the overarching reach of most such concerns are about blameworthiness, liability and people’s inability to do or intuitively understand or appreciate statistics and math. Free the children. Also free everyone.

Metaculus Redux

I got a bunch of pushback from people involved with Metaculus on my comments on Metaculus from last week. There was a lot of discussion in the LW comment section, and some talk on Twitter as well. I continue to be very happy Metaculus exists (and that Good Judgment Project exists) which is why I am willing to discuss them and why I am sad that they are not better. I’ve written extensively about prediction markets, and these aren’t prediction markets. They also aren’t entirely not prediction markets, of course, but it’s true that they’re different and it’s not obvious the incentives can or should be the same as in prediction markets themselves.

If you are motivated to make predictions in such places, and are putting in the time such that your predictions are net helpful, then you are doing a public service, so please keep doing that. And a big thank you to those helping run and improve such places, even if I don’t agree with all your decisions and don’t choose to participate.

My other request for such places is that I find them next to impossible to usefully navigate for the questions I care about, whether I’m looking to view results or make predictions, and more and better curation and organization of the markets seems high value. So does ensuring that the most important questions exist and are in good form, and get the extra attention they deserve. 

At some point I hope to write up my thoughts in more detail.   

Vitamin D As It Ought To Be

New study claims that it confirms that Vitamin D is very important for Covid and is highly effective even when administered at the hospital. Alas, it looks like the methods and statistics they used were highly suspect to say the least. They didn’t randomize patients, they did the treatments in some wards but not others, then didn’t even assign patients randomly to wards, then didn’t cluster their errors. 

That doesn’t change the fact that everyone should be supplementing Vitamin D, and it’s plausible I should be talking about this way more often lest people forget. The new data isn’t compelling (at least, it isn’t in the form it’s been presented) but it’s also definitely not evidence against Vitamin D. 

Then there’s Scott Alexander’s latest article on the subject, Covid/Vitamin D: Much More Than You Wanted To Know, which is in some sense a very accurate title (as in, the first paragraph tells you what you wanted to know, take Vitamin D regardless), but in another sense not true at all, since if anything he can tell us less than I would want to know, since I want to know more than is known, again even though it doesn’t directly impact behavior today. His thoughts definitely fit right in with ‘Vitamin D study looks good but turns out to be deeply flawed and thus it tells us nothing new.’ 

I understand where Scott’s analysis is coming from here, but I don’t agree with it. I do acknowledge that it is possible that all the gigantic correlations are a big coincidence that results from intermediation from some combination of otherwise poor health, other sunlight effects and socioeconomic status, and failing to control for such matters. It’s possible, sure, despite the effect sizes in the ‘damn check out this chart’ area. I also think there’s a bunch of dialectic thought going on here, where it’s effectively about Whose Study Is Right and so when two disagree we have it out see whose was better, or something, with negative findings needing to be defeated in a righteous battle. I also think Scott is going with a big “yeah, yeah, the boy is crying Vitamin D again and I’m not about to go chasing after this non-existent wolf yet again.’ 

That’s making his prior a lot different than mine, perhaps entirely fairly. If I was informed I’m making an epistemic mistake, my first guess would be that I need to better appreciate the degree to which there’s been wolf-crying and the dynamics involved, and that this should be moving my estimates.

Scott makes three probabilistic assessments (they’re not predictions exactly) at the end:

Tyler Cowen notes that he is even more skeptical than Scott. 

The first curious thing here is the matching 25% numbers. I can imagine Vitamin D helping if taken as a regular supplement while not helping, or helping less or only helping when administered early or the right way, in the hospital. I can’t imagine it working in the hospital, but not working as a regular supplement. That would blow my mind. These two numbers being identical thus suggests Scott doesn’t see it that way, and in particular that he’s thinking that if it doesn’t work in a hospital (or doesn’t work in a hospital for any given reasonable dosing method) it also doesn’t work as a supplement. 

I still think Vitamin D supplementation in advance of infection is a favorite to significantly decrease the risk of getting Covid and give that maybe a 60% chance. I’ve been persuaded that it’s not obviously true and that taking Vitamin D definitely doesn’t mean you can laugh off Covid risk or anything. I find it reasonably plausible that if you try to make up for a deficiency in the hospital, it’s too late to do much about it, and the chance drops to maybe 45%. 

Can you hear Eliezer Yudkowsky in the background saying “Bet! Bet! Bet!”? I hope you can!

The problem is that we both agree on the correct course of action – give everyone Vitamin D supplements now, and give every Covid patient more intense ones in the hospital – because there’s so little downside to doing that. And I believe we both agree that this issue probably isn’t going to be definitely settled, regardless of who is right. If Vitamin D supplementation doesn’t do anything about Covid-19, some studies will keep finding correlations because they aren’t controlling for enough of the right things. If Vitamin D does do something, a lot of studies will still be deeply flawed because scientists and doctors mostly run flawed studies and there’s a lot of correlations to dodge here and people trying to prove a point are especially bad about controlling for such things properly, and other studies will lack sufficient power or use the wrong protocols or control for things twice or whatever and end up finding no effect. There will never be a challenge trial or other definitive answer.

Maybe if there were a particular future study happening where we could both see the protocol, we could bet on what that study will find. That sounds like it might work, and if someone has a good candidate, please share. 

Because we agree on the course of action, in many ways the more interesting questions are epistemic ones. How should one evaluate the evidence in this situation? Which things should move our priors how much? An in depth discussion of that with an eye on general principles of evaluation of evidence seems like it would be generally useful.

I’d also note that the third line says equal or outweigh, not outweigh. The word equal either is or is not doing work here. Does this include the case where Vitamin D supplements do nothing for most people but are at least marginally helpful if you’re sufficiently deficient (42% of Americans are deficient, Google says)? If so, then I find this number strangely low and wonder why it isn’t starting with a 9 (or at least an 8) even if we ignore Covid entirely. If the word equal isn’t doing work and the question is about whether this is worth the cognitive opportunity cost of taking up an action slot that could be used for something else, then I could accept 75% conditional on the Covid-related skepticism being very reasonable.

A final note is that no one is denying that Vitamin D deficiency is very highly correlated with bad Covid-19 outcomes. The world in which Vitamin D supplementation doesn’t help is the world in where there holds some combination of (A) Vitamin D supplementation doesn’t provide Vitamin D in a useful way, (B) Vitamin D levels are a proxy for other sunlight effects, (C) Vitamin D levels are a proxy for age or other health effects or (D) Vitamin D levels are a proxy for socio-economic status. 

Thus, if you go to the doctor and they measure your Vitamin D levels as sufficient, that definitely is very good Covid-risk news for you personally. If they measure your levels as insufficient, that definitely is very bad Covid-risk news for you personally. Thus, there’s a weirdly Calvinist perspective on all this that one could take, that I do not endorse, where those with high Vitamin D are The Elect who are low risk and are saved, even if one is saved via an entirely different method. Again, not endorsing that, but I do find it amusing and interesting to think about.

In Other News

Monthly reminder that I have a Patreon if you have the desire to give me money to support my writing. While I find this motivating, I do not need the money even a little bit. Contributing earns you my silent thanks to you, but that’s it. If you want to contribute enough to matter, contact me privately to minimize the costs involved.

We’re finally doing a challenge trial. Even at this late date, it’s super valuable for its results, and even more valuable for the precedent it sets:

Stimulus plan wants to spend $500 per person on distribution alone, and The Grumpy Economist asks why it costs so much. For all things vaccines I prefer to ask why we are spending so little rather than pinch pennies. One can reasonably raise an eyebrow here, but vaccine distribution is so valuable that the concern should be corruption and theft. If the money is effectively being stolen that would be bad. If the money is being used inefficiently but with any good effect at all, such as by massively increasing payments to those who give the shots, I’m totally fine with that. Cost benefit analysis checks out.

This piece by Zeynep on how to think critically and interpet official communication is fantastic. Not for those who are old hats at this, but great for those getting up to speed.

Some unusually good non-Covid content recently on LessWrong: Anna Salamon observes that PR is corrosive whereas reputation is not [LW · GW], and Eliezer Yudkowsky discusses the concept of Cheerful Prices [LW · GW], the amount of money that would cause one to happily do a thing.

There’s an online event coming up on February 25 about pandemics, biowarfare and the future of human health called Going Viral, if you’re interested you can register at the link, and they asked me to pass it along.

I am curious what changed Dr. Fauci’s timeline here, I haven’t heard an explanation of what caused the update:

The alternate hypothesis, of course, is that Fauci is telling people the timeline he thinks will cause the most responsible behavior rather than his actual estimate (or, in other words, lying.)

Medical students are at risk more from socializing than from treating patients, in one study in Denmark. This matches priors.

Your periodic reminder (in this case from Bryan Caplan) that bioethicists are mustache-twirling villains and without them we wouldn’t have a pandemic. Also, did you know that if you want to know if your experiment is allowed, you have to ask bioethicists and find out their answers? That sure sounds like an experiment on human subjects. Shouldn’t we ask bioethicists if that’s permitted before we ask bioethicists if that’s permitted? 

Your periodic reminder that rapid tests would end this pandemic and could be had en masse for about $5 each, but we don’t allow them to be sold, so what few exist languish in places where the people involved don’t want them. 

Thread about how we measure the effectiveness of flu vaccines.

Story about a private service to help with vaccine distribution in Oregon. Sadly not about a way to spend more money to get more vaccines, but still helpful.

In WaPo, Alex Tabarrok tells us how to get on with it and vaccinate faster via some of the usual completely obvious minimal-downside strategies we should definitely do. Nothing new here.

It’s fun to think about what might be good to do with the J&J vaccine if we could do whatever we anted, but that doesn’t make such ideas remotely practical or plausible. 

Department of it could always be worse: A current hypothesis that China is holding off on vaccinating more widely until it can have sufficient supply for the whole country. There’s concerns about equity, and then there’s full on anti-escapism. 

Biden team announced vaccine deliveries sooner than they were officially previously expected:

That is of course excellent news, although how good the news really is depends on what our true previous expectations were and how much this updates those expectations. Biden’s goal of 100 million doses in 100 days was never in doubt, and it seems plausible that all the official timelines have been chosen in a similarly conservative fashion to ensure they can’t fail. Underpromise and overdeliver. 

Germany doesn’t have much vaccine supply, but is managing to actually target its most vulnerable with what supply it does have. Comments suggest Sweden is doing this as well. I too am curious to see how the case fatality rates shift a month from now in such places versus our own.

FDA refuses to authorize rapid tests, so we ship 60mm of them overseas. Sigh. 

We ran an experiment to see people’s willingness to pay for less crowded planes (MR linking to WSJ), and it was low. The implication is that people’s willingness to pay to avoid Covid risk may be low in general, and we should consider that (while keeping in mind that much risk is not private). One complicating factor is that the people who value Covid safety a lot mostly either aren’t flying at all, or think the plane itself is mostly safe. I’ve noted before that the ventilation of the planes themselves seems rather good, and I’d be more worried about the taxi and getting through the airport, and also I haven’t flown at all anyway. I also expect a psychological effect where once a person decides they must fly or is willing to fly, often they don’t properly factor in marginal risk when making decisions, because they’re in the context of a bigger baseline risk and people’s brains don’t handle that very well. 

Even more than that, I think that the way we shop for plane tickets is designed to cause this result. Everyone goes to an internet site and everything is sorted by price, price and price, with filters for airlines and times and stopovers to the extent you use those. But mostly the whole system is about getting the best price, which gives you visibility and makes you the default choice on multiple levels. I don’t see why plane safety via less people on the plane should be any different – people won’t think about it or won’t be aware of the differences, or won’t be willing to actively give up cheaper prices to get the thing even if they’d also never make the opposite trade, and so on. So I think this is interesting and worth pondering, but doesn’t prove as much as one might think.

A thread arguing the UK should go for Covid elimination, and that such suggestions have effectively been censored and considered non-options. It uses a lot of the ‘no evidence’ lines of reasoning, although in ways that are less terrible than normal. I think the way you get to zero is first you vaccinate and then you go for zero rather than locking down first, but yes everyone should absolutely go for zero Covid by the end of the year by getting the contact tracing systems ready for that. I’m actively confused that this isn’t obvious.

Let’s check in on our model of Dr. Fauci, and see how it does on some events from last year:

Weekly Tom Friedman update for additional perspective, nothing new here.

Regulatory hold-ups are worse than you thought. Ada Poonawalla of the Serum Institute of India reports that they have 70 million doses that exist but can’t be shipped because the countries that have purchased them haven’t approved them, and they are already four months old and expire in April. It doesn’t sound like he’s optimistic that, if the countries involved still haven’t approved them, he’ll be able to redirect those vaccine doses to other places. He’s already making 70mm doses a month, so one would hope he’d be able to ship the older doses out and replace them with new ones, so at least nothing spoils unused, but I no longer count on such sanity to prevail. Also worth noting that Ada got the majority of his funding from the Gates Foundation, so they did end up helping build useful capacity in at least one case. 

Part of this ‘vaccine nationalism’ seems obviously good to me. If every country decides it needs its own manufacturing sufficient to make everything it needs, that seems great, because then we might have something approaching enough overall manufacturing capacity. If everyone says ‘no exporting vaccines until we have enough for everyone domestically’ that’s not the ideal distribution mechanism but it is highly motivating for capacity, which in the long term matters more. Whereas restrictions on importing vaccines, requiring one’s own regulatory hurdles everywhere, is pure downside. Even if you want to ‘support’ your own candidate, why not simply pay for the vaccine doses later from your own factories even if you end up not using or exporting them? The amount of pound foolishness, from almost all angles, is staggering. 

Not Covid, but we should be rather worried about our failure to reliably keep the lights on. Literally. 

Non-Covid Weight Loss Drug Update takes a look and finds the results most promising, against a backdrop of historical money incineration that suggests continued need for caution. 

Non-Covid Civilization Watch: Large parts of Texas continue to be without power under dangerously cold conditions. Now there are reports of lots of people losing water as well. Explosions shut off power to large parts of Portland. California’s power grid is increasingly unreliable each year. We should be alarmed about this. It is perhaps the best concrete test of whether our civilization can maintain its physical infrastructure, it can’t be faked, and it is very much not going well. Any given incident can usually be explained away, extreme weather events have always caused temporary power loss, but when it happens more often on larger scales for longer periods, we need to ask ourselves what’s going on. 

Thus, we must ask ourselves: What still works?

29 comments

Comments sorted by top scores.