($1000 bounty) How effective are marginal vaccine doses against the covid delta variant?

post by jacobjacob · 2021-07-22T01:26:26.117Z · LW · GW · 72 comments

This is a question post.

UPDATE: the bounty has now been awarded as follows: 

===

Since I got vaccinated I’ve started working in-person, going to restaurants, travelling and whatever. But the delta variant might change this risk calculation. Some of the same buzz that got covid right early on, the first time around, is now buzzing about delta potentially being very bad.  

So I’m exploring ways of responding to that. This time around I’d rather stand up and fight [LW · GW] than lock myself in a house for a year. And so this question explores one possible approach. 

There’s evidence that two vaccine doses are more effective than one (though there might be much higher diminishing returns than is commonly acknowledged). I’ve also heard that past infection confers marginal immunity even in the presence of vaccination. Further, as far as I understand, the 2-dose schedule for many vaccines is more “a common thing that we happened to test in the big trials” rather than “the dose level at which marginal benefit = marginal risk”. (And giving your population more than two doses is probably much harder logistically.)

Hence: getting more than the standard 1-2 vaccine doses might be a way to protect against delta. 

I live in California, and we currently seem to have a large vaccine surplus. I heard a vague rumour about a friend basically just walking into a pharmacy and getting a 3rd shot. There’s also RADVAC, which you can make yourself. 

So, if there is supply... how worthwhile is it to get more vaccine doses? Should you get a 3rd, 4th, … or even more? How does getting a further dose of the same vaccine compare to getting the first dose of a different vaccine?

I’m posting a bounty of $1000 for answers that change my mind on this question (maybe increasing to many times that if this proves valuable). 

The ideal thing I want would be a graph with the x-axis showing # doses, and the y-axis reduction-vs-control of the following four parameters: 

(Answers are of course impacted by the combinatorial search space of dose spacing / dose number / dose size / vaccine type / various demographics, but I won’t make any special restrictions here for now)

I will pay at least $1000 for answers that help me get clarity on this, split at my full discretion in proportion to how useful I find the answers. I’ll pay out the bounty on a rolling basis as answers come in; there is no deadline. In case this proves fruitful and there seems to be useful marginal work, it’s possible I will increase the bounty a lot (i.e. up to many thousands of dollars), or reach out to work with some answerers as contractors.


 

Answers

answer by Connor_Flexman · 2021-07-25T13:52:26.753Z · LW(p) · GW(p)

Tl;dr: I think we can vaguely guess a 1.5-2x risk reduction from a third shot, maybe pushing 2-2.5x if you get it close to 6 months after the first; for those with less immunity after two shots, it might be more like expected 3-6x, though it’s unclear if one can discern who these are. If you do get a third shot, you should definitely try to get a different kind, as others have said, though unclear if you want to sacrifice efficacy for this.

---------

First, to establish an expected effect size, how good are the first two shots? The first Pfizer/Moderna takes you to about 80% protection, or a 4/5 risk reduction. The second takes you to 95% efficacy, a 4x risk reduction. To compare, Delta variant takes Pfizer from 95% back to 80, so is a 4x risk increase. So most likely, the third vaccine is not going to be as efficient as the second vaccine. So we expect less than a 4x effect; realistically, maybe I expect something like 2x at the outset since secondary boosters are common in immunology and third boosters aren't. I wouldn't be that surprised at seeing 3x or 1.25x though. (Note that this is all about transmission, for a shot of the same type as your first two; probably slightly stronger effect than this in mortality, and mixed vaccinations I'll get to at the end).

There's no released data, so we have to get creative fast. The best empirical evidence I know of comes from this study on how much vaccinations boost your neutralizing antibody (NAb) titers after already having COVID. (NAbs are by no means all of your immunity, but they’ve been shown to correlate surprisingly well with vaccine effectiveness. Hilariously, there are nonetheless papers claiming we have some evidence that NAbs can CAUSE infections to be worse—while this may be true, all I can do is plead for them to make a better case that this isn’t bog-standard Treatments Correlated with Harm [LW · GW].)

Anyways, this study shows limited effect from a second Pfizer vaccine dose if the patient had already contracted COVID! It’s useful because very roughly speaking, getting COVID gives similar titers as a vaccine dose—and COVID plus one dose is similar to two doses, so looking at COVID plus two doses is a good approximation for a third dose. This gives us some limited evidence that a third dose won’t be that effective.

Notably though, it did significantly increase the titers of the least-protected! This seems like a decent reason to get a third vaccine if you are vulnerable. To put very rough numbers on it, my guess is that for the vulnerable it would cut risk by a factor of 3, whereas the group as a whole only gets a factor of about 1.5.

  • Calculation: Getting a second dose cuts risk by 4 as we’ve already said; on this graph the first to second shot gives a jump in titers by about 3 logs, whereas it looks like the vulnerable have their titers go up about 2.5 logs from the third shot, so I’ll give this a ~3-fold reduction. The group as a whole only goes up about 1 log on the third-dose-equivalent, which is maybe a factor of 1.5 protection. Both of these figures match well with our original bound of finding less than a 4x average reduction, and quite well with my prior of about 2x!

As a second creative line of evidence, could we figure out anything from the claims from Pfizer that their third dose increases neutralizing antibody (NAb) titers by by 5-10x?

Is 5-10x a lot? Unfortunately, a major hurdle is that every vaccine appears to measure NAbs using different assays, with no translation to efficacy. I really hope I’m mistaken here, because otherwise it seems like every report of increasing titers by X-fold is almost completely meaningless, e.g. this otherwise-nice comparison. But I don’t think I am.

I tried going through other Pfizer studies to see if maybe they at least used a single assay across the board. One Pfizer study says a factor of 1.5 difference in unnamed geometric titers for a strain meant roughly nothing. Another shows about a 4x factor in log2PRNT50 difference between Delta and wild-type, which corresponds to about a 4x decreased efficacy as we know from elsewhere. But this can’t be the same as the above-stated titer, because that would mean a 5-10x reduced risk from a third shot, which is way higher than our bounds. It’s possible that if we escape the log though, this becomes a 16x titer difference for 4x risk reduction, which is at least in the ballpark and would imply a 2x risk reduction from the third shot. I think this is on shaky ground though and I don’t place much stock at all in it, even if it happens to pop out the magic number.

-----

A very important thing here that a few people mentioned: combating waning vaccine efficacy is important. Many studies show this effect, but I’ll cite my favorite that I linked earlier. It predicts about a 2x risk increase (or effectiveness decrease) over 6 months. Another study shows variable waning from natural infection that roughly accords with this, at least insofar as NAb titers from this or similar papers can be interpreted.

There are two questions here: first, would a third vaccine reverse this waning effect? There aren’t data to confirm this yet, but surely the answer is yes—this is how other vaccine boosters work, and all theory points to this being the case. The second question is whether the third vaccine’s natural effect, as described early to be ~~1.5x normally or ~~3x for the vulnerable, would fully stack on top of a waning-reversal. I think the answer is probably no, but with partial effect, closer to fully stacking than not. Part of this answer comes from the idea that the longer you wait between doses, the more effective they are. In total then, I expect a booster after 3 months to give you about 2x risk reduction, and 4x for the vulnerable.

Last, I completely agree with the other people bringing up that getting a different third dose seems better than the same (though the same shot seems still worth it for many, as above). I wish I had gotten my second dose of a different kind—this is significantly because of “original antigenic sin”, the tendency to get antibody lock-in toward whatever strains you experience early. I expect different vaccines as boosters will help somewhat with this problem, though I really can hardly speculate quantitatively about this—all I’ll say is I would be unsurprised to see that taking this strategy with the right vaccine might get you 4x instead of 2x from a third shot. (However, you’d need both an effective vaccine and for it to be substantially different than the first one or two—and I don’t know if Moderna/Pfizer fit that bill. Perhaps I can research this in the future, or how much effectiveness to sacrifice for difference, though it sounds hard and by default I won’t since I don’t think it’s all that decision-relevant.)

Two last thoughts: if Elizabeth is right in the comments about a possible low limit for mRNA vaccines, that’s a major downside. If it weren’t for that, it would almost surely be worth it to get a 3rd shot for most people by the calculus, since Delta is growing so fast and has so little morbidity for the vaccinated that there isn’t much incentive to keep it out of young and healthy circles. (Cf about 4 days of expected lost life from Delta while vaccinated, and getting a third shot would save you 2 expected days at a cost of two days lightly sick. But if there's much cost in the vein of what Elizabeth mentions, that washes out.)

comment by Ethan Perez (ethan-perez) · 2021-07-28T14:33:34.015Z · LW(p) · GW(p)

Do you (or others) have a sense of whether it'd be better to take Moderna or J&J after two doses of Pfizer? (Or whether to take Pfizer vs. J&J after two doses of Moderna)

Replies from: Connor_Flexman, Connor_Flexman
comment by Connor_Flexman · 2021-07-29T03:22:35.241Z · LW(p) · GW(p)

No, but I would like to know. The two relevant variables are that mRNA is more effective, which we can sort of quantify, but non-mRNA is probably more different, which I don't know how to quantify. Currently I view them as roughly equivalent, but an even cursory glance at what was in the mRNA vaccines vs non would potentially be quite helpful.

comment by Connor_Flexman · 2021-08-07T21:30:13.344Z · LW(p) · GW(p)

I have a tentative answer! Some cursory googling makes me think that J&J also just replicates the spike protein in you, the same way Pfizer/Moderna do. This means it's just strictly less effective. Then you'd want to just do the Pfizer/Moderna one that you haven't yet—unless Elizabeth's comment about limited mRNA vaccine doses is decision-relevant, which I still haven't looked into.

comment by jacobjacob · 2021-08-09T07:29:46.047Z · LW(p) · GW(p)

I'll pay $425 for this answer, will PM you for payment details. 

answer by JenniferRM · 2021-07-22T05:14:11.266Z · LW(p) · GW(p)

EDITED: Connor's comment below [LW(p) · GW(p)] landed directly in a place where I was ignorant and I've updated to think the right answer is "defer to him on the details of immunology until I read a lot more". 

I feel like what I wrote was basically adequately hedged given my ignorance, and maybe it is interested to read for having some of the picture, but not all of the picture, but if you're just saving your attention for super high value things you can probably just skip it? I get kind of rant-y towards the end about the lack of a proper public health system. Maybe if you like the rant-y bit this is still worth reading for that?

...

My current personal mechanistic model is that for "a given challenge" (that occurs in a window roughly 6-21 days large, with variation between people and also between challenges for the same person in different states of health), people either have a germinal center response, or not. 

If not, then on a second attempt they might have a response again, or not. This is "in general". I've seen studies where they did really good followup with a population for a measles vaccine, and some patients managed to play along with followup 5 times, and only the 6th vaccine caused their immune system to finally begin testing positive for the desired antibodies.

However, my current personal mechanistic model has huge error bars, and numerous asterisks, with warnings about how immunology is a blacker art that computer security, because it basically IS computer security, but for programs written in nucleic acid that are billions of years old, with millions of new revisions spammed into existence every second by a mad god. I can't read the code (at least not most of it, and not easily). I just expect it to contain devilish tricks because: it would be surprising if it didn't.

Under the simple "response or not" model, the reason they did "two tries without bothering to check in between for a response" for the current Pfizer/Moderna design is that this "double tap" protocol ups the official "chance of having the response from the one official protocol" and basically jukes their numbers, so they can brag about a 90% efficacy "if you do it right" and this makes the narrative and the medical protocols and everything simpler. 

If I was rich enough to have a private doctor (I'm not, and I don't even have "a doctor" as such because of moves and paperwork and dealing with insurance paperwork constantly being so so so exhausting) then I could totally see just paying "one competent personal doctor" to acquire and administer one vaccine dose, test for response, then doing another dose only if the antibody test is negative... but also doing a third if the antibody test after the second didn't show that it worked, and so on.

However, under the "response or not" model this binary response is for a specific epitope (or epitopeS if several antibodies form for different surface bits of different proteins... I'm not personally clear on how the immune system picks exactly what folded protein surface part(s) to pick as a target, or why) and so if you have DIFFERENT vaccines that target DIFFERENT surface patches on the same set of proteins (or variants of proteins) then for a specific later challenge by an environmental exposure probably the best fitting antibody for that exposure reacts the strongest and leads charge by the immune system to fight off the invading virus particles and their initial first couple generations of viral babies [LW(p) · GW(p)].

My current working model might be wrong. If it is right, then the best vaccines would target lots of different epitopes selected from numerous possible genetic variants and we would generate and ship them in maybe close to realtime, as a sort of "human culture and institution mediated meta immune system". 

This is how a properly designed public health system would actually work. But the US does not have a public health system basically? It just has a pharmaceutical and medical licensing monopoly maintenance and legal challenge immunization system, that prevents incumbents in the medical industry from being competed into low margins and high consumer health surpluses :-(

If our country was well run, we would already have a delta booster designed and available that could trigger the formation of these new kinds of antibodies, and the reason we don't is because the government is "either evil or incompetent" and the private industry isn't allowed to innovate around this barrier to technologically solving technologically solvable problems. ("FDA delenda est", but that's neither here not there.)

(Maaaaybe someone HAS already designed such boosters, but the FDA and/or CDC threatened them with angry looks and being put on some kind of professional blacklist if they ever admitted this in public on the second go-around because the PR was so bad for them when it came out that the first mRNA vaccine was designed in early 2020?)

Basically, my model is that RaDVaC or other novel vaccine designs COULD improve your response to VARIANTS by increasing the breadth of covid epitopes that your body can recognize and respond to, but that if a given vaccine design triggered a response in you already then another one is unlikely to help much more...

...however I have genuine model uncertainty. Maybe a second exposure causes the specific form of specific immune responses to somehow be stronger or different in a way that I've never seen a clean and clear explication of? Some anecdotal evidence suggests that double-dosed people are safer than single-dosed people, but I've never seen anyone report anything like this with a coherent followup about how or why such anecdotes might occur more (or less) in "different worlds" where (1) single-dosed people are less likely to have had a germinal response at all vs (2) the second dose materially changes the nature of the immune response.

I'm interested in learning a better model, if my model is wrong.

Another thing to point out is that having more antibodies is called "being allergic to things" when the thing you have antibodies for something that isn't actually bad. And it is called an "autoimmune disease" when the antibodies recognize something that naturally is already part of you. Acquiring new antibodies is not totally risk free. All biology is half black magic and full of exceptions because evolution is deep and twisty. Please please please be careful!

If you want to give me the $1000 for this writeup, please give it to johnswentworth [LW · GW] instead.

comment by Connor_Flexman · 2021-07-25T11:06:14.641Z · LW(p) · GW(p)

I think almost all the evidence points in the opposite direction, unless I’m drastically misunderstanding something, which does occasionally happen.

First and foremost, the idea of having a binary seroconversion dependent on germinal center response seems highly contraindicated at best. There are a billion studies showing that single doses of vaccines give some antibody response, and then a second dose gives far more (often 2-3 OOM). For example, 1, 2, 3, 4, etc. This is the point Lanrian seemed to be making, which I think pretty immediately disproves the hypothesis. 

This isn’t just COVID either—many vaccines have this pattern of giving boosters to increase antibody count. And not only does it increase count, secondary responses vastly increase antibody affinity and produce different antibody types, e.g. the primary response is more IgM whereas secondary response produces more IgG and IgA (the latter aiding especially in mucosal immunity). [Citations for this can be found on pgs 413-414 of the Janeway immunobiology book, and I can maybe link pictures.]

On this note: higher doses of vaccine straightforwardly give higher antibody levels from a nonzero baseline, inconsistent with a binary response [1].

Further, amount of NAb present in the infected scales with disease severity—it isn’t a binary, which you’d expect to see if the main correlate of immunity was a single threshold of GC response. [1 shows correlation with severity, 2 “great titer variability, 3 amazing paper showing different neutralization levels translates well to efficacy in vaccines, plus waning titer and loss of efficacy over time.] 

A different important departure from a binary condition: a substantial number of cases exhibit rapid waning of NAb levels over 6 months to a negligible concentration (and cases fall all over the spectrum for how fast they wane) [this great Lancet study]. Presumably this could be easily overcome with another vaccination, as asked.

Another datapoint against is that it doesn’t explain partial vaccine efficacy. How would a vaccine protect you from mortality but not from symptoms, unless it mattered where on a spectrum you landed? (Obviously there are ways, but they all strain credulity.) It also wouldn’t fit with the fact that vaccines can result in NAb titers that are universally high in a group, but still leave them getting occasionally infected [I’ve lost this source but it was obviously relevant for Delta strain, though many people produced no or little NAb for Delta which was an important distinction].

Last, even if we were totally dependent on a binary GC response, we could probably still modulate that with more introduction of antigen! My understanding of GC responses is that they get initialized in part through other cells like CD4+ T cells, the concentration of which also correlates with different doses of vaccine, presumably causally.

Replies from: JenniferRM, Connor_Flexman
comment by JenniferRM · 2021-07-30T04:36:42.787Z · LW(p) · GW(p)

I just read this tonight and it is fantastic. You know more immunology than me by a lot! <3

I'll be editing my answer to leave the substance of what I wrote so posterity can see me being dumb but willing to make an educated guess, but defer to your answer at the top with an explanation :-)

comment by Connor_Flexman · 2021-07-25T11:55:28.431Z · LW(p) · GW(p)

The relevant graphics on booster shots:

comment by Lukas_Gloor · 2021-07-22T06:41:49.254Z · LW(p) · GW(p)

I feel like your model doesn't explain why getting the 2nd dose of the vaccine after 8 weeks instead of 4 weeks increases efficiency. I think this is the case, and if so, it suggests that the 2nd dose adds something on top of the first one, falsifying your assumptions. 

Replies from: Joe_Collman
comment by Joe_Collman · 2021-07-22T14:35:07.851Z · LW(p) · GW(p)

It would explain at least a slight efficiency increase: presumably [some collection of factors] (SCoF) influences whether there's a response or not. A priori you'd expect a smaller correlation of SCoF with SCoF-after-8-weeks than with SCoF-after-4-weeks.

Presumably the actual impact is larger than this would predict (at least without a better model of SCoF).

comment by Lanrian · 2021-07-22T20:14:26.182Z · LW(p) · GW(p)

Huh, I'm pretty surprised by this model. Why do you think it's correct?

Here's an image of some measure of people's antibody responses from page 9 of this paper, where the first set of points is people's response 5-6 wks after dose 1, and the second set of points is people's response 2-3 wks after dose 2.

It looks like people who get an antibody response to the first dose still get a much improved response from a second dose. And there's no sign of a bimodal responses to any of the doses. Is that consistent with your model?

Also, the way vaccines can protect from severe disease without protecting from infection seems to suggest that there's more than a binary question of response/not-response.

Replies from: JenniferRM
comment by JenniferRM · 2021-07-23T02:07:18.093Z · LW(p) · GW(p)

You found a neat paper! Thank you!

"MY WHY" (for my admittedly simply model)... Is... Basically... Uh... a thing I experience often is that I have a sort of "models inside of models" expectation that seems rarely to be applied in practice even by so-called experts? 

For example during Feb/Mar 2020 there were people talking about how "X% of patients are asymptomatic" but they were making these assertions based on single snapshots of infection cohorts in whom the epidemic was actively moving. So some of the people (like on a cruise ship) might have been PRE-symptomatic (because maybe checking for symptoms 14 days later would turn something up) rather than A-symptomatic (as a general property of their entire disease course). People were using the terminology willy nilly, and no one was tracking any of it precisely... almost no one was thinking of it like they were getting momentary glimpses of a sort of state machine or advertising funnel or something, where each person might get a slightly different ride along a slightly different path AND ALSO might be at a different step along whatever ride they will end up having taken.

Another very common failing is that people think that a group mean implies a group median. If there are bimodal responses to something, and then a group summary is given... is the group's denominator over "all treated" or "all who were treated and had a followup confirming an adequate response"?

BOTH of these errors have a common cause of "assuming homogeneous efficacy and assuming competent followup at the clinical level" and in my experience neither of these assumptions are strongly justified. They constantly fail, and people are constantly acting surprised about it. 

Failures of followup are ALSO why we couldn't get people properly quarantined at the beginning of this disaster. 

Often patients DO NOT WANT to have "the system" FOLLOW them.

The study you linked to seems to have somewhat solved "the standard problems with the 'lost to followup' state" that is the bane of so many time course studies. The design certainly seems to take the followup process very much into account (and I couldn't find drop out rates from skimming or ^f and so maybe literally no one dropped out):

Every patient who completes the protocol will have visited the clinic at least 4 times within a window smaller than 15 weeks, which is pretty good followup! <3

Something I'd like to call attention to here... in the paper you link the Extended Track had Bleed1 data from the extended cohort group, where they got (in some sense) to see how many people seroconverted from just one dose by week 5 or so...

Within the extended interval cohort, antibodies were detectable in 91% (62/68) at the first timepoint, at 5-6 weeks after the first vaccine, but this rose to 100% 2-3 weeks after the vaccine boost.

Recall that up in the abstract the paper summarizes the key result thusly:

Results: In donors without evidence of previous infection the peak antibody response was 3.5-fold higher in donors who had undergone delayed interval vaccination.

Suppose, hypothetically, that instead of 91% of people having "a seroconverting response" on the first shot it was only 28% of them?  

(This would be almost understandable. The youngest person in that study was 80 years old! The whole study is on a group whose immune systems should be assumed to be decrepit and fragile from the raw fact of great age.)

Then if the second exposure brought this up to 100% seroconversion "somehow", and the seroconverted "antibody levels" were gaussian (log normal?) among the seroconverted and 0 among the rest...

...Then that bimodal response could directly and cleanly justify claiming "antibody response was 3.5-fold higher" in some very fuzzy and general way (because 28% x 3.5 = 98%)

MY NORMAL EXPECTATION is for people to communicate in a fuzzy and general way :-(

The graph you included as a supporting claim was, I think, just the B panel from the totality of Figure 2 which is nice in many ways. Color coded! The horizontal axes are mostly aligned! Nice!

All the data from all the cohorts, basically? Not just Panel B.

Note that in Panel A the two timepoints give basically the same levels of antibody response, with maybe some hint of a slow decline, but also overlap with Panel B's separated ranges. Some in Panel A went up?? Weird. Probably stuff goes down (and sometimes up?) over time, in general?

The data in Panel A therefore seems consistent to me that "eventually" there is some roughly normal and acceptable level of "vaccinated at all, in an essentially bimodal way" that two doses reaches faster than typical? 

This is what the two dose shot is designed to do in my mind: get ALMOST ALL of the patients (because of herd immunity benefits) to the state of "CLEANLY SEROCONVERTED" with the LEAST amount of measurement and need for followup (because followup is really hard).

Bleed2 of the standard group is "10 weeks post standard dose2".  There is no Bleed3 for either group out all the way at week 21.  That third data collection event would be "10 weeks after dose2 for the extended group" and thus sorta comparable to the standard group's Bleed1?

My hunch is that extended Bleed3 would show a decline from the extended Bleed2 measurement... 

...maybe this prediction is a crux?

I could also imagine those slow risers in the standard group would STILL be going up by week 21?

Basically, I suspect that antibody levels eventually go down EVENTUALLY (over months and years), but also have some "sensitivity to dynamics over a timecourse" (which is probably not showing up here, not because it didn't happen, but because it wasn't measured).

I don't know. My error bars are wide.

...

Also it would have been great to measure antibody levels for everyone on week 3, as Bleed0? More of the dynamics would be visible I think, and it would help characterize (and separate?) various members of the standard group in terms of seroconversion status before the second dose?

...

Basically, immunological science is more of an art than a science. It has a gazillion moving parts created under extreme adversity. Also, humans underestimate the difficulties of just doing the thing over and over.

People don't get that "seroconversion" IS A THING. Sometimes it just doesn't happen. That's often the most important practical fact. Many bad vaccine designs end up with a "vaccine" whose seroconversion rate is non-zero but so low as to be impractical.  This meta-analysis for measles (trying to find a relationship between age and seroconversion) shows numerous things that were tried in the clinic that had less than 50% rates for some kids. 

The Pfizer/Moderna/mRNA design is weirdly effective from my perspective here. To take a second dose of a weirdly effective thing... uh... I guess? Sure. If empirically that works for this disease maybe it could or would help somehow in a way that eventually could be made sense of.

So if I have to pick ONE THING to assume about vaccine status at the INDIVIDUAL level, it will be "seroconverted or not" and then after knowing the answer to that, I assume "the rest will be very very complicated".

I freely admit this is a simplified model. I just also think that any specific additional mechanistic second step to the modeling effort is likely to explode the combinatorial space of patient states to worry about, and yet also be a TINY epicycle, and the first of MANY epicycles.

The thing I'm asking for is: what's the best second epicycle to add? What is the mechanism? If someone is already seroconverted, what would you measure to detect "that their mechanistic biological state is not ALREADY in the configuration that you'd be hoping to cause to improve via the administration of a third dose"?

And in the meantime: DELTA MIGHT BE MUTATING AROUND THE CURRENT VACCINE DESIGN and so a new design aimed at the new epitopes just obviously seems like it would address the main uncertainty in a central way. 

RadVac for Delta is something I might pay for, and maybe something I might want to try to make on my own? Searching a bit: it looks like mixed third dose trials are already starting, so... <3!

A third dose that's "the same as the first two" doesn't interest me. The second one doing what it does seems to be empirically real, but I don't know why the hell it empirically gives the results it does.

Replies from: Lanrian
comment by Lanrian · 2021-07-23T20:13:09.935Z · LW(p) · GW(p)

On your "WHY", you seem to be presenting reasons why other people not believing your model shouldn't count as strong evidence against it. Which is all fair. But I'm still curious for positive evidence to believe your model in the first place. Maybe this would be obvious if I knew more biology, but as it is, I don't know why I should place higher credence in your model than any other model (e.g. the one at the bottom of this comment, if that counts).

...Then that bimodal response could directly and cleanly justify claiming "antibody response was 3.5-fold higher" in some very fuzzy and general way (because 28% x 3.5 = 98%)

As far as I can tell, "antibody response was 3.5-fold higher" just means that, on average, people in the extended dosing schedule had 3.5x more antibodies. I can't tell whether you interpret it in some other way, or if you think this is a misleading way to describe things, or if you're making some other point...?

The graph you included as a supporting claim was, I think, just the B panel from the totality of Figure 2 which is nice in many ways.

Yup!

The data in Panel A therefore seems consistent to me that "eventually" there is some roughly normal and acceptable level of "vaccinated at all, in an essentially bimodal way" that two doses reaches faster than typical? 

Ok now I'm confused.

  • Do you think that all people on these graphs have reached a "normal and acceptable level of 'vaccinated at all, in an essentially bimodal way' "?
    • If so, do you not think that there's any important immunity difference between a single-vaccinated person around 1-10 on the graph, or a doubly-vaccinated person around 1000-10000?
  • Or if you think that only some of the people on this graph are immune, where do you think the line between immune and not-immune should be drawn on these graphs? (The distribution seems to be fairly continuous everywhere, to me, so it seems arbitrary to draw the line anywhere.)
  • Or if you think the important immunity difference isn't captured by antibody-levels, what is it about?
  • And re "that two doses reaches faster than typical"; are you implying that the single-dosed people's antibody response would've kept increasing beyond the 5-6 week mark and eventually gotten as high as the doubly-vaccinated people? That seems unlikely to me. (Other than maybe the few people where their antibodies did increase, but I'm happy to ignore them until I understand the most normal response curve better.)
My hunch is that extended Bleed3 would show a decline from the extended Bleed2 measurement... 

Agreed.

The thing I'm asking for is: what's the best second epicycle to add? What is the mechanism? If someone is already seroconverted, what would you measure to detect "that their mechanistic biological state is not ALREADY in the configuration that you'd be hoping to cause to improve via the administration of a third dose"?

Here's one suggestion:

1. The more antibodies you have, the less probability of getting sick, the less probability of getting severe disease, etc.

2. More vaccines increases the number of antibodies you have.

3. Therefore you want to have more vaccines.

I would've thought (1) to be fairly uncontroversial? And the linked study seems to provide good evidence for (2) when going from 1 to 2 doses, increasing antibodies by roughly a factor of 100. And of course adding more vaccines will eventually stop adding more antibodies. But right now I don't have any reason to believe in a big difference between going from 1->2 vaccines vs going from 2->3 vaccines (other than 2 vaccines being the general standard). So I wouldn't be surprised if taking a 3rd vaccine could increase your antibodies by another order of magnitude.

Maybe you think this doesn't provide enough of a "mechanism"? Biology being complicated, I'm very happy to take empirical data for what it is, and make extrapolations even if I don't know what the mechanism is. Personally, I also don't feel like I have any more mechanism for "vaccine have a fixed probability of causing antibodies if you don't already have them, otherwise they don't do much" than "vaccine typically increases antibodies by a lot regardless of whether you have them or not". So when the evidence clearly indicates the latter, I will definitely believe it.


And yeah, also, if someone has the option, I agree that it seems probably better to get a different vaccine than the same vaccine again!

Replies from: JenniferRM
comment by JenniferRM · 2021-07-30T04:33:07.472Z · LW(p) · GW(p)

I sort of tapped out because "very long posts with an explosion of quotes" is a smell for me, but I wanted to continue because other indicators suggest "teaching and/or learning in good faith" <3

Finally posting now because of a big update from elsewhere...

On your "WHY", you seem to be presenting reasons why other people not believing your model shouldn't count as strong evidence against it. Which is all fair. But I'm still curious for positive evidence to believe your model in the first place.

For me, evidence happens at the point of measurement. Then often measurements are summarized in language by people who don't think clearly, or worry about standard misinterpretations of simple measurements... so careful reading is sometimes required just to acquire evidence able to distinguish between models.

So for me, the default is to need to think about mechanistic timecourse evidence through the screen of "how it was confusingly explained to me" by people who often aren't worried about mechanistic timecourse dynamics. 

I kinda don't care if people don't believe my model, I just want my models to get better over time... and I'm happy to explain them to people, and I like teaching... but if people don't believe me, then it is their tragedy that they believe false things, not my tragedy. (Conversely, people teaching me things is awesome!)

But to make my models better I don't just import other people's posterior believes about how a mechanistic system works, but rather see if my own model can "round trip" through my best guess of the raw data that they observed in a specific situation.  If people have bad reasoning, then their posteriors are even less safe to import than otherwise...

FWIW, just tonight I got around to reading this cousin comment by Connor [LW(p) · GW(p)] and it swiftly tipped me over almost entirely. Three doses... might work? Sure.

I already thought there were empirical reasons to think it, so for me I think the key words in Connor's post started somewhere around:

And not only does it increase count, secondary responses vastly increase antibody affinity and produce different antibody types, e.g. the primary response is more IgM whereas secondary response produces more IgG and IgA (the latter aiding especially in mucosal immunity). [Citations for this can be found on pgs 413-414 of the Janeway immunobiology book, and I can maybe link pictures.]

The filter I have I think, is that I want to hear about mechanisms when it comes to biological theories. 

I'm not saying button mashing doesn't work. That plus "copy the winner" is how most actual technical innovation occurs and scales in practice most of the time. Its fine <3

But... a HUGE filter that avoids adding broken bits to my general reasoning capacities is whether someone can offer keywords that connects their proposed mechanism to ALL THE OTHER MECHANISMS in physics and chemistry and evolution and all of it.

Gimme a word like "IgA" and I can find my way to new and helpful parts of the truth mine! I can round trip it through general science, and so on.

I have paragraphs and paragraphs of text from my first attempt at a response, trying to explain "I don't know and neither do you (but politely and at length)".

They are deleted from this response. Maybe "two people debugging epistemics in the face of ignorance" is useful somehow for something, but I'm not attached to it. I could PM it maybe if you care?

Practical upshot: empirically more doses has worked, and now I have heard some "new magic mechanism words" from Connor, who seems to me to clearly knows his shit backwards and forwards and also seems to be in tentative favor of a third dose :-)

Maybe interesting: my main argument AGAINST a third dose is part of why I thought it might be smart to give single doses as fast as possible several months ago. Now that like... "mechanisms are mechanically different (giving more than just lots of IgM)" I feel like I learned enough to even notice errors in past thinking? 

But also... weirdly(?) this same body of empirical results says that the second reaction works BETTER after ... <missing mechanism that somehow is time dependent> has had 12 weeks to <do something> instead of just 3 weeks?

F-ing immunology, man. Its crazy.

comment by johnswentworth · 2021-07-22T22:50:52.006Z · LW(p) · GW(p)

Thank you!

comment by jacobjacob · 2021-07-22T21:28:07.930Z · LW(p) · GW(p)

I'll pay at least $150 $100 for this, might increase later. And yes, it will go to John if he accepts it. 

answer by SoerenMind · 2021-07-22T18:36:01.011Z · LW(p) · GW(p)

According to one expert, the immune system essentially makes bets on how often it will face a given virus and how the virus will mutate in the future:

https://science.sciencemag.org/content/372/6549/1392

By that logic, being challenged more often means that the immune system should have a stronger and longer-lasting response:

The immune system treats any new exposure—be it infection or vaccination—with a cost-benefit threat analysis for the magnitude of immunological memory to generate and maintain. There are resource-commitment decisions: more cells and more protein throughout the body, potentially for decades. Although all of the calculus involved in these immunological cost-benefit analyses is not understood, a long-standing rule of thumb is that repeated exposures are recognized as an increased threat. Hence the success of vaccine regimens split into two or three immunizations.

The response becomes even stronger when challenging the immune system with different versions of the virus, in particular a vaccine and the virus itself (same link). 

Heightened response to repeated exposure is clearly at play in hybrid immunity, but it is not so simple, because the magnitude of the response to the second exposure (vaccination after infection) was much larger than after the second dose of vaccine in uninfected individuals. [...] Overall, hybrid immunity to SARS-CoV-2 appears to be impressively potent.

For SARS-CoV-2 this leads to a 25-100x stronger antibody response. It also comes with enhanced neutralizing breadth, and therefore likely some protection against future variants. 

Based on this, the article above recommends combining different vaccine modalities such as mRNA (Pfizer, Moderna) and vector (AZ) (see also here). 

Lastly, your question may be hard to answer without data, if we extrapolate from a similar question where the answer seems hard to predict in advance:

Additionally, the response to the second vaccine dose was minimal for previously infected persons, indicating an immunity plateau that is not simple to predict.

comment by jacobjacob · 2021-08-09T07:29:02.143Z · LW(p) · GW(p)

I'll pay $50 for this answer, will message you for payment details. 

answer by Dorikka · 2021-07-24T07:39:57.885Z · LW(p) · GW(p)

Depending on when you got your second mRNA dose, the Israeli data suggests there is significant vaccine decline in vaccine efficacy after 3 months (see  https://www.gov.il/BlobFolder/reports/vaccine-efficacy-safety-follow-up-committee/he/files_publications_corona_two-dose-vaccination-data.pdf ) This does not of course indicate whether a third dose would restore efficacy (presumably by reigniting the immune response in some way) - I suspect it would, however. No need for prize money if relevant - just resharing a link that was already in Forbes https://www.forbes.com/sites/roberthart/2021/07/23/pfizer-shot-just-39-effective-against-delta-infection-but-largely-prevents-severe-illness-israel-study-suggests/?sh=ca1146c584f1

comment by Ethan Perez (ethan-perez) · 2021-07-24T22:42:27.833Z · LW(p) · GW(p)

That data is very helpful and clear, thanks for sharing! Pfizer has some initial findings that suggest to me that a third dose would fix the declining efficacy issue: https://cdn.pfizer.com/pfizercom/2021-07/Delta_Variant_Study_Press_Statement_Final_7.8.21.pdf?IPpR1xZjlwvaUMQ9sRn2FkePcBiRPGqw

"Pfizer and BioNTech have seen encouraging data in the ongoing booster trial of a third dose of the current BNT162b2 vaccine. Initial data from the study demonstrate that a booster dose given 6 months after the second dose has a consistent tolerability profile while eliciting high neutralization titers against the wild type and the Beta variant, which are 5 to 10 times higher than after two primary doses."

This seems to make third dosing sound like a pretty good idea, definitely 6 months after the second dose, and probably as early as 3-4 months after the second dose (when the Israel data shows efficacy starting to decline). Curious if others disagree!

comment by Connor_Flexman · 2021-07-25T13:06:32.352Z · LW(p) · GW(p)

(Heads up that this is where the Delta variant is widespread though, so this is probably exacerbating the natural effect of waning immunity. Not sure of the exact timescales though)

answer by Ethan Perez · 2021-07-28T00:53:00.194Z · LW(p) · GW(p)

It might be worth posting this question to Metaculus to see what the crowd-consensus view is there. I'd post a question myself there, but I'm not sure I have enough expertise to define the question and resolution properly (but it seems like you or others might).

answer by rhollerith_dot_com · 2021-07-22T07:08:50.410Z · LW(p) · GW(p)

My guess is that you have generated your own hypothesis (namely, more than two doses will work better than 2 will). I almost always leave medical hypothesis generation to the medical experts with the result that my job is to choose among the hypotheses and to tailor my choice to my situation. Of course I consider hypotheses by contrarians and alternative health-care providers, which number in the high dozens or hundreds of hypotheses for Covid.

(Experts have proposed a 3rd shot, but that would be a shot of a new vaccine, not one currently being given to people.)

comment by MichaelLowe · 2021-07-22T08:50:10.515Z · LW(p) · GW(p)

This does not seem to be the case in Turkey, where they are right now handing out third doses of existing vaccines to people.

Replies from: Randaly
comment by Randaly · 2021-07-22T23:22:33.052Z · LW(p) · GW(p)

This, and the Bahrain/UAE cases, seem more likely to be driven by concerns about whether/how well the Chinese vaccines work?

comment by thjread · 2021-07-22T09:31:38.607Z · LW(p) · GW(p)

There are lots of clinical trials of third doses of the same vaccine (some completed and some still in progress, see e.g. https://www.bmj.com/content/373/bmj.n1659 https://cdn.pfizer.com/pfizercom/2021-07/Delta_Variant_Study_Press_Statement_Final_7.8.21.pdf ), and it's likely that the UK will offer third doses of the original Pfizer vaccine to everyone over 50 or especially at risk, some time towards the end of this year.

Replies from: neel-nanda-1
comment by Neel Nanda (neel-nanda-1) · 2021-07-22T12:02:51.239Z · LW(p) · GW(p)

it's likely that the UK will offer third doses of the original Pfizer vaccine to everyone over 50 or especially at risk, some time towards the end of this year.

Interesting, do you have a source for that?

Replies from: pseudobison
comment by pseudobison · 2021-07-23T07:57:13.697Z · LW(p) · GW(p)

https://www.bbc.co.uk/news/health-57667987 - Hard to say what's "likely" with this government, but it's what the Joint Committee on Vaccination and Immunisation has advised

Replies from: thjread
comment by thjread · 2021-07-31T08:31:06.713Z · LW(p) · GW(p)

Also we've ordered an extra 60 million Pfizer doses for the booster program: https://www.gov.uk/government/news/uk-secures-extra-60-million-pfizerbiontech-covid-19-vaccines

72 comments

Comments sorted by top scores.

comment by Elizabeth (pktechgirl) · 2021-07-22T23:22:46.557Z · LW(p) · GW(p)

On the cost side: I've heard concerns that due to the delivery mechanism there might be a total number of mRNA shots (for any disease) you can use per lifetime, and side effects seem to increase per dose suggesting individuals might have a lifetime limit for any particular disease as well. If there is a lifetime limit, you want to think about how to allocate those across boosters and variants, and getting a third identical shot could be a huge waste.

Replies from: habryka4
comment by habryka (habryka4) · 2021-07-22T23:24:21.043Z · LW(p) · GW(p)

Huh, that's really interesting. Do you have any sources around this? That sure could introduce some big shifts in the EV.

Replies from: pktechgirl
comment by Elizabeth (pktechgirl) · 2021-07-22T23:36:46.233Z · LW(p) · GW(p)

The polyethylene glycol thing is discussed elsewhere on this page and I think they remember more specifics than I do. I do expect this problem is solvable, but you wouldn't want to run out of mRNA shots before that happens.

The lifetime limit per disease is based on:

  • General knowledge of the immune system
  • Most people found the second shot worse than the first, although there are exceptions
  • My own response to the second shot was exceptionally bad, after a normal first shot (like, 99th percentile, still seeking medical treatment for it bad. I don't regret getting the shot because I expect I also would have been an exceptionally bad covid case, but it does affect my cost/benefit calculation around further shots)
  • In the course of seeking treatment for my long term side effects, hearing from medical providers that they're seeing a lot of people in the same boat. There's no denominator on this of course, but it doesn't seem to just be me.
Replies from: Sodium
comment by Sodium · 2021-07-28T20:11:53.046Z · LW(p) · GW(p)

Could you send a link to the discussion/ any other source on this?

comment by Josh Jacobson (joshjacobson) · 2021-07-22T14:37:26.201Z · LW(p) · GW(p)

My post from 12 days ago: https://www.lesswrong.com/posts/8RYxQrKegKMDGHcvo/for-some-now-may-be-the-time-to-get-your-third-covid-shot [LW · GW]

Replies from: jacobjacob
comment by jacobjacob · 2021-07-22T21:26:51.850Z · LW(p) · GW(p)

Thanks for signal-boosting, I had missed this. I'll pay at least $300 for it. (The fact that it already had been written 12 days ago seems like a point in its favour!)

Replies from: joshjacobson
comment by Josh Jacobson (joshjacobson) · 2021-07-23T14:35:22.410Z · LW(p) · GW(p)

That’s awesome, thanks!

comment by Owain_Evans · 2021-07-22T11:43:57.346Z · LW(p) · GW(p)

I'd like to see investigations of the following:

1. Secondary attack rates for Delta for pairs of people who are fully vaccinated with Pfizer. (I'm guessing this is relevant to someone in the Bay, as most of the people you have close contact with for more than 30 mins are fully vaccinated).
 
2. Long Covid rates for the fully vaccinated (both Delta and other variants). 

3. Using genetics or other personal information to estimate risk of severe disease or Long Covid. (e.g. Blood type was relevant to severe disease risk for original variant.) 

4. How to get access to new vaccines designed for Delta (or for "universal" Covid). Most likely route is clinical trials. Maybe some countries will approve earlier than others (e.g. UK, UAE). 

5. Aside from being more infectious, is the guidance for avoiding infection from Delta any different? E.g. surfaces vs aerosol.

Replies from: SoerenMind
comment by SoerenMind · 2021-07-22T18:40:31.974Z · LW(p) · GW(p)
  1. FWIW I've seen multiple double-mRNA-vaccinated people in my social circles who still got infected with delta (and in one case infected someone else who was double vaccinated). Two of the cases I know were symptomatic (but mild).
Replies from: ethan-perez, jacobjacob, brp, Lanrian
comment by Ethan Perez (ethan-perez) · 2021-07-28T00:40:29.428Z · LW(p) · GW(p)

I also know of 5+ cases of symptomatic COVID among double-vaxxed people in the bay area (including one instance where most people in a group house of ~6 people got covid). These are also relatively healthy individuals in their 20s

comment by jacobjacob · 2021-07-22T21:26:39.555Z · LW(p) · GW(p)

Do you know which, if any, risk-reducing precautions they were following? 

Replies from: SoerenMind
comment by SoerenMind · 2021-07-25T16:06:42.076Z · LW(p) · GW(p)

Some standard ones like masks, but not at all times. They probably were in close or indoor contact with infected people without precautions.

comment by brp · 2021-07-23T13:10:21.234Z · LW(p) · GW(p)

Immune response is generally associated with age and lifestyle. What can you tell us about those factors?

Replies from: SoerenMind
comment by SoerenMind · 2021-07-25T16:08:33.777Z · LW(p) · GW(p)

Age around 30 and not overweight or obviously unhealthy

comment by Lanrian · 2021-07-22T19:45:00.015Z · LW(p) · GW(p)

How many asymptomatic? And how did people know of them?

comment by JonCB · 2021-07-22T03:47:25.487Z · LW(p) · GW(p)

This is 100% anecdotal and has next to no details (so i wouldn't consider it a contender for any part of the bounty) however this seems relevant (assuming it's true) :-

https://www.businessinsider.com.au/delta-variant-infected-all-unvaccinated-guests-at-a-sydney-party-2021-6
 

I hope that helps.

comment by ChristianKl · 2021-07-22T04:36:05.997Z · LW(p) · GW(p)

Some of the same buzz that got covid right early on, the first time around, is now buzzing about delta potentially being very bad.  

Chris Martenson is one of the people who got it right back in January and his latest video is quite interesting: 

The tl;dr is that while Delta has a significantly higher r data from Israel suggests that this doesn't translate into increased deaths. This suggest that the virus is less harmful. Unfortunatley, there's no good long-COVID data. 

I'm not 100% sure that Chris is right here but I think it's a good talk to hear for orientation.

I live in California, and we currently seem to have a large vaccine surplus. I heard a vague rumour about a friend basically just walking into a pharmacy and getting a 3rd shot. There’s also RADVAC, which you can make yourself. 

I strongly suspect that RaDVaC offers good additional protection. RaDVaC gives you mucus immunity while the other vaccines likely only start protecting you once the virus hits area besides mucus. 

The fact that RaDVaC also offers targets that are different from the spike protein helps provide immunity if all the vaccines that we have that target the spike protein lead to the virus mutating the spike protein to evade immune response (which is likely partly what we are seeing with Delta).

RaDVaC also has the advantage that if you want to target the spike protein you can actually target the sequences that match the latest spike protein.

Unfortunately, we don't have good data for RaDVaC. According to a facebook discussion one person involved with RaDVaC said that RaDVaC is heavily cash constrained. Connecting them to funders might be high leverage in a world where the official vaccines for some strange reason don't get updated to the latest virus strains. 

How does getting a further dose of the same vaccine compare to getting the first dose of a different vaccine?

If you take the same vaccine too much (the same meaning the same adenovirus or both using polyethylene glycol) you get an immune response against the adenovirus or polyethylene glycol which means your immune system kills some of the vaccine before it gets active. A polyethylene glycol immune response is also something that you generally don't want. Unfortunately, it's not clear what "too much" means here. It might be 4 it might also be 100. It might be different from person to person.

I had a 30 minute conversation with my doctor (whom I know privately as well) about whether the increased side effects of the second dose for Pfizer are due to polyethylene glycol and he was quite certain that this isn't the case but did say that if we start to give the mRNA more regularly we have to look at whether polyethylene glycol resistence is a significant problem.

Next to vaccines there are other more dakka actions available. To not reiterate the more political ones here, air quality is important. If you plan on meeting in your home with other people in the coming months get an airfilter if you don't already have one and run it on max when guests are there. At work you could also partition your company to buy airfilters. Arguments about how air pollution reduces cognitive performance might be worth making at work.

The ideal thing I want would be a graph with the x-axis showing # doses, and the y-axis reduction-vs-control of the following four parameters: 

That sounds to me like something bad to seek because such a graph would likely be mostly madeup. You likely want to understand on all the assumptions that go into the graph and not take any graph at face value. 

Replies from: Lukas_Gloor, jacobjacob, jacobjacob, brp
comment by Lukas_Gloor · 2021-07-22T07:19:40.246Z · LW(p) · GW(p)

Chris Martenson is one of the people who got it right back in January and his latest video is quite interesting

I watched the video and I think it hard to convey just how bad it is. The guy is also annoyingly smug (the first ten minutes of the video are just him reading headlines and saying the word "dangerous" in a mocking voice) while being wrong about his important claims.

The tl;dr is that while Delta has a significantly higher r data from Israel suggests that this doesn't translate into increased deaths. This suggest that the virus is less harmful

Vaccines work great at preventing severe disease and deaths. Obviously one of the highest-vaccinated countries will have fewer deaths now – that says nothing about how deadly the Delta variant is in an unvaccinated population.

And then he goes on to talk about how there are more vaccinated people aged >50 in the UK hospitals than unvaccinated people, and he thinks that's strange or even implies that it could mean something bad about vaccine efficiency.  He doesn't understand base rates (9/10 people over 50 are vaccinated in the UK!). 

Doing the analysis right, we see that Delta is probably significantly more deadly than Alpha, which itself was more deadly than the original variant. We also see that Delta affects children more.

(Edit: I initially thought he's also wrong about not factoring in that the majority of Delta variant cases in the UK haven't had enough time to run their course, but I think the chart he used actually factored that in and was considering cases with a known outcome. So I deleted a paragraph above – one fewer mistake than I initially thought!) 

Replies from: ChristianKl
comment by ChristianKl · 2021-07-22T08:41:18.880Z · LW(p) · GW(p)

Obviously one of the highest-vaccinated countries will have fewer deaths now – that says nothing about how deadly the Delta variant is in an unvaccinated population.

I don't think vaccinations resulting in fewer deaths explain how the death rate in Israel stays relatively constant while the case rate goes up.

One thing worth noting here is also that just because both the strains in the UK and Israel are classified as Delta doesn't mean that there can't be differences that are clinically relevant.

Replies from: Lukas_Gloor
comment by Lukas_Gloor · 2021-07-22T09:18:17.287Z · LW(p) · GW(p)

The weekly average of deaths went from roughly zero, to one, to two. That's going up. Soon it will be at three or four if things continue like that. That looks like normal growth, deaths always lag behind surprisingly much when case numbers are growing exponentially. It also took the UK a really long time to from 7 weekly deaths to even just 10, but now the weekly average is above 50. 

Replies from: ChristianKl
comment by ChristianKl · 2021-07-22T10:35:04.786Z · LW(p) · GW(p)

From my perspective it seems relatively hard to know what's happening. I'd love to have some prediction for the Israeli numbers in 2-4 weeks, so that I can then update to believe your thesis or Martenson. Do you feel confident to make some prediction in confidence intervals of where you believe the numbers will be?

Replies from: Lukas_Gloor
comment by Lukas_Gloor · 2021-07-22T11:45:34.251Z · LW(p) · GW(p)

Scientific consensus opinion is that Delta is more deadly, not less. The guy in the video says otherwise, but his specific arguments are flawed. This isn't difficult to check/verify: He didn't factor in that the death rate is automatically lower in a vaccinated population.

Without the evidence he thought he had from the UK data, all that's left in support of his position, that Delta is less deadly, is this: "Intuitively, deaths are climbing really slowly in Israel compared to case numbers."

By itself, intuition about deaths climbing slowly doesn't seem like anywhere close to a good reason to question expert consensus. (It also wouldn't make me think that Australia may have its own less deadly subtype of Delta.) 

I feel like if your plan is to wait 2-4 weeks to check my predictions against reality, you're losing time unnecessarily. There's enough info here to update sooner. 

Maybe we have different intuitions about how weird the slowly climbing death rate is. I think it's not weird at all – it's always gone this way.  You can compare the situation in Israel now to how things were in the UK throughout June, see the charts here, specifically comparing how the rise in case numbers came early than the rise in death numbers. 

Or, to give another example, back in early 2020, people kept pointing out that the case fatality rate in South Korea seems really low, and that this means the virus isn't that bad. But that was just the lag from cases to deaths, and after South Korea's outbreak went beyond its peak and people had time to die, the case fatality rate went up by multiples! 

Regarding predictions: Israel only has 1k cases and they do extensive testing, so there's not a lot of underreporting. This makes predictions a bit difficult because the variance is high. Probably we'll see at least 4 average daily deaths in three weeks. But most importantly, and most confidently, I'll say that as long as cases keep rising, deaths will eventually go up as well – that's the prediction of the model that Delta isn't weaker.

I have a good track record on Covid predictions. I won the first big forecasting tournament on Covid on Metaculus [EA(p) · GW(p)] and got 3rd in the second installment of the tournament. I live in the UK and therefore have a headstart (except vs. people in India) on following Delta developments closely. I'm also a bit addicted to virus news and spend 2h per day on that.

Replies from: ChristianKl
comment by ChristianKl · 2021-07-22T14:37:59.227Z · LW(p) · GW(p)

Scientific consensus opinion is that Delta is more deadly, not less

The phrase scientific consensus seems to me quite ill-placed in a state of evidence like this. Scientific consensus is usually something that takes years to build for a hypothesis. At the moment it usually gets used to manipulate people to treat positions that come out of a very political process has having to do something with science.

I think the phrase should only be used when there's at least a meta-study that looked at the literature and found consensus.

That said, what makes you believe that this is a "Scientific consensus opinion"?

I feel like if your plan is to wait 2-4 weeks to check my predictions against reality, you're losing time unnecessarily. There's enough info here to update sooner. 

Losing time suggests that there's an action I'm currently not taking that I should be taking. It's likely that the same action I take now is twice as expensive in terms of COVID-19 risk in a month (given COVID numbers where I live)

Replies from: Lukas_Gloor
comment by Lukas_Gloor · 2021-07-22T19:16:15.469Z · LW(p) · GW(p)

It's really not that difficult to ballpark IFR estimates (especially now that everyone knows that there's no giant iceberg of asymptomatic cases). The Delta variant has been around for a while. You can play word games but it's fairly obvious what I mean by "scientific consensus." I mean that >100 media articles I've read in the last couple of months, from various mainstream UK sources, I've never seen anyone seriously entertain the hypothesis that the Delta variant is less deadly than previous variants. 

I'm not saying to blanket endorse the media's perspective on what's the scientific consensus. That would be strawmanning me. I'm saying that when you claim that the consensus is wrong (edit: or that it very well might be), at least have more of a reason than "Intuitively, those numbers look too low on the chart." At least have an argument for what, specifically, the mainstream experts are getting wrong. The video doesn't have that since it fails to pass the Ideological Turing test for people who are concerned about the Delta variant. 

Losing time suggests that there's an action I'm currently not taking that I should be taking. It's likely that the same action I take now is twice as expensive in terms of COVID-19 risk in a month (given COVID numbers where I live)

Having accurate beliefs can be beneficial in unanticipated ways. There's no point in needlessly delaying epistemic updates. I'm saying that there's something off about how much credibility you give to confident-sounding contrarians on youtube (with some credentials, admittedly, though not that it matters). (Esp. once some of their core arguments get debunked.) 

Again, I'm not making a blanket argument about contrarians always being wrong. I'm making the specific argument that contrarians who are right don't tend to make easily visible mistakes. 

Replies from: ChristianKl
comment by ChristianKl · 2021-07-22T19:59:08.947Z · LW(p) · GW(p)

I mean that >100 media articles I've read in the last couple of months, from various mainstream UK sources, I've never seen anyone seriously entertain the hypothesis that the Delta variant is less deadly than previous variants. 


That's journalistic consensus and calling it scientific consensus is an insult to the scientific project. For society it's very costly to confuse academic discourse with journalistic discourse. There's a huge cost for the public trust in science from confusing the two. Making the public case that while there are a bunch of media articles suggesting that climate change doesn't exist there's a scientific consensus that it clearly exists wasn't an easy sell. There's huge collateral damage from equating the two for propaganda purposes. 

The decision of what mainstream journalistic sources print depends on editorial policy and is not based on asking a representative sample of scientists in a field. 

Chris Martenson was telling people to get masks in January 2020. That might make him 4-5 months ahead of the journalistic consensus on getting the mask question right. In general the journalistic discourse on COVID-19 is so bad that people who open windows in public transportation get angry looks for it because the journalist failed to tell people that opening windows and getting airflow is very useful for reducing COVID-19 risk. 

Replies from: Lukas_Gloor
comment by Lukas_Gloor · 2021-07-22T20:07:05.966Z · LW(p) · GW(p)

You're right, I think I phrased my point poorly. What I should have said is "If there's no reputable or intelligent-seeming person making some claim for a long time, and then comes along some lone contrarian, that contrarian better make a good impression – otherwise it probably doesn't make sense to invest a lot of time steelmanning their claims and digging into a hypothesis that wouldn't even have occured to us without this person." 

Maybe you've seen more people talk about Delta deadliness concerns being overblown. But based on my media diet as well as based on all the conversations I've had with EAs and rationalists about this, the view that Delta might be a less infectious (but more contagious) variant has never come up.

It's true that media consensus isn't great to go by, as we've seen with all the instances Zvi documents in his post (mask wearing, possibility of lab escape, etc.). 

Replies from: ChristianKl, Lukas_Gloor
comment by ChristianKl · 2021-07-22T20:48:01.975Z · LW(p) · GW(p)

The UK had a political fight about whether or not to end lockdown and what you read in UK mainstream sources likely reflects how the outlets editorial position stands in the fight. 

I do think that having a good track record is what makes people credible and rather listen to people who got COVID-19 right at the start then people who don't and who published crap about masks don't working because they were to lazy to do real journalism and critically report. 

If we take the Guardian as an example, what possible reason can there be to not publish a single article about patient zero despite Huang Yanling being recognized as important in the NIH letter to the EcoHealth Alliance? Outlets that act that strange should not be trusted.

Replies from: Lukas_Gloor
comment by Lukas_Gloor · 2021-07-23T07:05:07.943Z · LW(p) · GW(p)

I feel like we got sidetracked. 

I do think that having a good track record is what makes people credible and rather listen to people who got COVID-19 right at the start

I got Covid right at the start (among others) and I posted above that my track record is winning the largest prediction tournament at the time.

So let me repeat what I cared about conveying in this discussion, one last time: 

The person in the youtube video you linked to may have gotten Covid right in early 2020, but so did hundreds of people (but maybe not thousands). Out of the set of people with a good track record on Covid, this guy is now pushing an extreme minority position. In theory, he could have been right with that. But he's wrong because his arguments are bad in an easily verifiable way. Once someone's core arguments for a fringe position (fringe in reference to the best sources we'd want to listen to here, not fringe with respect to the media) get discredited, there's no reason to continue treating the fringe position as though it still has a high chance of being right. By that point, we must be prepared to say "This guy went off the rails."

All I wanted to convey is that it makes no sense to continue holding a person's specific opinion in high regards (good track record or not) when the opinion is highly contrarian* and just had its core arguments refuted. By continuing to argue as though the guy might still be right, you were employing a type of epistemology which, to me, seems doomed. I get the impression that you not only distrust the media consensus, but any consensus seems worthless to you when you see a single confident-sounding expert who stands out as having gotten something right when others had gotten it wrong. I think that's too strong of an update, because lots of people got things right, and some of them may still be completely nuts and bad at reasoning, and we can spot that by checking things against a mental reference class of "consensus among the people we hold in high esteem."

*Again, I'm talking about contrarian with respect to the sources we'd want to listen to. That's a subjective reference class, but since we're both on this site and value track records, our takes on this may not be crazily different. It is my highly confident impression that <3% of Lesswrongers with high karma, and <3% of people who got Covid right in the early days, think that the Delta variant is less deadly in unvaccinated population than the original variant. 

comment by Lukas_Gloor · 2021-07-22T20:16:48.048Z · LW(p) · GW(p)

BTW, it's am minor point, but I feel like the media is biased to fail to identify new variants as more deadly, because of racism concerns. Every single media article about the South Africa variant said "there's no evidence it's more deadly." They were saying this before there was enough time to know with confidence (and "no evidence" was technically false because there were anecdotal reports of children being more affected). 

comment by jacobjacob · 2021-07-22T21:35:53.512Z · LW(p) · GW(p)

I'll pay at least $75 for this comment. If nothing else, alerting me to RaDVaC's funding gap is clearly worth that much. I think it offered some interesting considerations beyond that. E.g. the search term polyethylene glycol seems useful, though I haven't looked into it much at all and definitely don't have strong models of that domain. 

(I also think the fact that this comment bundled together a lot of different arguments and considerations caused the karma to take a downward hit.)

comment by jacobjacob · 2021-07-22T05:09:03.942Z · LW(p) · GW(p)

According to a facebook discussion one person involved with RaDVaC said that RaDVaC is heavily cash constrained.

Sounds like a state of affairs that should not be allowed to persist. Very interested in more details/screenshots if possible without violating any privacy norms -- I'll send you my email in PM.

Replies from: ChristianKl
comment by ChristianKl · 2021-07-22T05:39:30.745Z · LW(p) · GW(p)

It was a public discussion was on Robert Wiblin feed. Given that they are actually searching for funding it feels like a good utilitarian idea to quote here (if someone thinks it shouldn't be quoted just tell me):

Me: Is funding a problem holding RaDVaC back? If so, it might be worth making the case for EA funds going to RaDVaC on the EA-forum or seeking a grant from OpenPhil. I expect that it would be possible to raise high six figures or low seven figures for RaDVaC by seeking EA donations.

...

Alex Hoekstra:

Christian Kleineidam funding is very much a bottleneck for us. We've been ~99% focused on the science for the last 15 months, to the detriment of much else.

I gratefully welcome any advice on getting some additional fuel in our organizational tank; we're all volunteers so far and it's become increasingly clear over time that building open-source vaccine developer kits is worthy of a full-time commitment. There's so much more we need to do.

Somehow the draft I wanted to post as an answer encouraging writing an EA forum post never made left my Evernote draft and is still unpublished. 

I don't have the best overview over how to go about convincing EA donors to fund RaDVaC but I see the potential of RaDVaC has highly worthy of funding. If anybody reading this has the ability to connect the RaDVaC guys with funding I expect that to be very positive.

Replies from: jacobjacob
comment by jacobjacob · 2021-07-22T05:53:49.106Z · LW(p) · GW(p)

I have some good leads, will check in with them tomorrow.

(If I stop working on this/don't make any progress I'll post about that here, so as not to make this funding gap erroneously appear filled.)

Replies from: jacobjacob, ChristianKl
comment by jacobjacob · 2021-08-16T06:31:19.472Z · LW(p) · GW(p)

Following up: as a result of this thread, radvac will likely get a $100k donation (from a donor who was not considering them before). This does not fill their funding needs however, and they're looking to raise another $300k this year. 

For any interested funders, PM me and I can share detailed call notes. 

comment by ChristianKl · 2021-07-22T06:12:31.590Z · LW(p) · GW(p)

I'm very happy to hear this. 

comment by brp · 2021-07-23T13:17:23.269Z · LW(p) · GW(p)

The last time CM was mentioned on here I looked up his old videos about Fauci. In a video from Sept 2020 he made a very confident claim that NYC had already achieved 70% infections and thereby herd immunity. That turned out to be untrue:

https://www.lesswrong.com/posts/xEFfbEMFHhtgseKz3/covid-6-10-somebody-else-s-problem [LW · GW]

So my prior for him is now skewed toward "pundit" rather than "honest inquirer."

Replies from: ChristianKl
comment by ChristianKl · 2021-07-23T14:30:39.044Z · LW(p) · GW(p)

That does make me update towards trusting him less. The video which I previously watched where mostly good with a few irrelevant factual errors.

comment by adamzerner · 2021-07-22T01:49:13.226Z · LW(p) · GW(p)

I've thought about doing something similar with posing a question and offering a bounty for responses to the extent that they change my mind. I haven't pulled the trigger yet. I don't know why. Kudos to you for doing so!

Replies from: Joe_Collman
comment by Joe_Collman · 2021-07-22T02:31:51.962Z · LW(p) · GW(p)

In any context where good faith isn't to be expected (which I'd hope doesn't apply here), bear in mind that there are exploits.

Replies from: jacobjacob
comment by jacobjacob · 2021-07-22T03:12:02.377Z · LW(p) · GW(p)

I'm excited about mechanism design in this space. Like, if you have a prediction market (or forecasting question with a good aggregation algorithm), you can sort of selectively throw out pieces of information, and then reward people based on how much those pieces moved the market.  (And yes, there are of course lots of goodhart-y failure modes to iron out to make it work.) 

In this case I'm not going to be quite so formal. I don't have that strong of an initial view, so it might often be more of rewarding "provided a very useful write-up" than "provide a compelling counterargument to a thoroughly considered belief". 

comment by avturchin · 2021-07-25T08:10:44.087Z · LW(p) · GW(p)

I was told by one of researchers that the risk side is accumulation of “wrong antibodies” which may eventually target own tissues as autoimmune diseases. Any new shot increases this small risk. This is more true for complex vector vaccine like AZ, as they trigger generation of antibodies not only to carrier but also to vector. Anyway, I already got third shot of a vector vaccine.

comment by Douglas_Knight · 2021-07-23T01:30:29.950Z · LW(p) · GW(p)

I’ve also heard that past infection confers marginal immunity even in the presence of vaccination.

What do you mean by "marginal"? The most common meaning is "low," and if that's what you mean, you really need to know it's false. But I guess you mean "additional." That's a pretty weird usage, though, because infection isn't a margin that is easy to control, particularly if it comes before the vaccine (it would make more sense to switch the words "infection" and "vaccination"). There are lots of studies trying to measure how much protection infection + one shot gives. The most interesting is one that claimed that it provided substantial protection not just against sars-2 variants, but even against the quite distant sars-1. I'm not sure anyone has tested against delta, but protection against sars-1 is a pretty good substitute.

So why is this combination so broadly useful? One possibility is that actual infection is so different from vaccination that the body responds broadly, to encompass both. This possibility doesn't give much practical advice, although mixing vaccines would provide a little diversity. Another possibility infection is just like a vaccination, but came much earlier and the long interval (8 months) produced broad immunity. Indeed, memory cells diversify over time. If this is the mechanism, then an additional dose could be valuable, not because it is additional, but because it is so much later. The time between doses might be more important than the number of doses, particularly for broad immunity, ie, for variants. Since the UK already has a longer vaccine interval than the rest of the world, they should be better protected against delta, so their success so far might not be such good news for the rest of the world.