Participating in a Covid-19 Vaccine Trial #3: I Hope I feel Worse Tomorrow

post by ejacob · 2021-03-05T00:02:50.407Z · LW · GW · 2 comments

Contents

  Did I get the Placebo? Using Bayes' Theorem
  Estimating Likelihood Ratios
  Conclusion
None
2 comments

Today is four weeks to the day after my first injection, and I received my second at my appointment this morning. The process was very similar to my first visit: I entered and met first with someone who led me through paperwork and asked a series of screening questions, then had a brief medical exam, got my shot, waited 30 minutes in case of any acute reaction, and left. Notable differences this time:

• The entire process seemed less repetitive and went by faster (both subjectively and objectively). I was also filling out fewer forms than last time and wasn’t getting any orientation so this makes sense.
• I was told that my follow-up meeting “had to” be scheduled for two weeks from now by order of Novavax. This is sooner than was specified in the study design. I suspect they have enough data already to show that it works so they are planning to close the site.
• In the middle of my health exam, someone rang a Nurse Call button and the doctor ran out of the room. A few minutes later the Nurse Practitioner from last month came in and finished up. The building we were in is also used a clinic for World Trade Center survivors and first responders (and probably other health care I don’t know about) so it’s possible that this was totally unrelated, but the call would imply that someone had a reaction to the injection.
• There were specific instructions to inject the 2nd dose in the opposite arm from the first dose. I am right-handed so I used my left arm last time, which meant I had to use my dominant hand this time. The second round is more likely to have side effects though, so I wish I had known this to begin with. By word-of-mouth from a medical professional (my mom), apparently some people have reported pain in the first injection site after getting their second dose. So maybe they are looking for that.
• Doors were mostly kept open by default instead of closed by default. The first person asking me health-related questions even asked me if I wanted to close the door.

When I received my shot the NP mentioned that most people don’t have side effects from the vaccine. He also said the side effects seem to have a similar profile as the Pfizer and Moderna vaccines, namely mild flu-like symptoms for a day or two. This was not exactly supported by the literature I ended up reading.

Did I get the Placebo? Using Bayes' Theorem

I would like to have a better idea of my chances at having received the real vaccine. The experiment randomized 2 people into the trial group for every 1 person in the control. This gives a prior odds of 2:1.

There are two pieces of evidence: my symptoms after the first injection (none), and my symptoms after the second injection (to be observed over the next day or so). These are obviously not independent. I need to know how much more likely I am to experience side effects from the real injection than the placebo.

After reading a few unhelpful press releases, I found my way to medrxiv.org and entered NVX-CoV2373 into the search bar. I found a few papers which seemed likely to have the information I was looking for.

Estimating Likelihood Ratios

This first study was a phase II investigation of dose response; in simpler language, they were giving people different amounts of vaccine and seeing what happened. They also broke their study population into two age cohorts (18-59 and 60-84). Younger patients were on average more prone to react at all stages and doses. Data are not available but there are graphs (Fig. 2) and some statistics in the text. All of the numbers here are read off of graphs and are approximate.

“Local” adverse effects from the vaccine include pain, swelling, tenderness etc. at injection site, while “Systemic” adverse effects include fever, nausea, or malaise. The study had one placebo group and two pairs of dosed groups; each pair used a different dose size in their vaccines. Within each pair of dosed groups, one group received two doses and one group received a placebo shot for their second dose.

At least one local adverse effect was reported by:

• 15% of placebo participants on the first shot, 10% on the second shot
• 50% of the 5-microgram dose group participants on the first shot, 70% on the second shot
• 65% of the 25-microgram group participants on the first shot, 80% on the second shot

For the second shot, about 10% of participants who received a placebo that round reported local adverse effects, whether or not they received a real vaccine for their first dose.

Systemic adverse effects were reported by:

• Roughly 40% of people in all groups after the first shot
• 20% of people receiving the placebo for their 2nd shot
• 50% of the 5-microgram group after their 2nd shot
• 60% of the 25-microgram group after their 2nd shot

For younger participants, the gaps between placebo and treatment groups was a bit larger - younger people reacted to vaccines more often - so I’m tempted to bump all of those figures up a bit in my own calculation (I am in my 20s).

The second study I read had a very similar design, but participants generally reported more symptoms than in the first one I discussed. I suspect whichever method they used to ask participants about their symptoms was more sensitive - for example, more than a third of the placebo group reported a headache, which counts as a systemic adverse effect - so these numbers might be inflated. Even if this is true, there’s almost no reports of fever from any of the participants, which surprised me. Looking at the other systemic effects, it looks like fatigue, malaise, and muscle pain were each all about two or three times more common in the treatment groups than the control group.

Finally, I read this meta-analysis which apparently has done most of the work for me:

For the meta-analysis, we separated the adverse events based on vaccine vs. placebo injection as reported by individual studies. In general, we observed there was an increase in total adverse events for subjects with low dose vaccine injection [OR: 2.86; 95% CI: 1.90-4.29, P < 0.00001]. Especially, the local reactions were significantly enhanced in subjects with low dose vaccine groups [OR: 2.07; 95% CI: 1.07-4.00, P = 0.03]. However, the systemic reactions were no significantly changed between vaccine and placebo groups [OR: 1.28; 95% CI: 0.67-2.43, P = 0.46].

The doses used by the Novavax PREVENT-19 trial are 5 micrograms, at the lowest end of the doses used in these papers. The analysis of high-dose vaccines yielded modestly higher odds ratios (there was a significant increase in systemic reactions for high-dose participants vs placebo) but the same general picture.

Integrating all of this evidence in a reasonable way is not a trivial problem, but they don’t seem to disagree too much - the odds ratio for having local adverse effects seems to be about 1.5-4, and the odds ratio for systemic adverse effects a little lower than that. The systemic reaction doesn’t seem to be very diagnostic and it’s probably correlated to the local reaction in ways I don’t understand and am uneasy guessing about. So to a first approximation I will just update on the presence or absence of local adverse effects.

Conclusion

At the time I am publishing this, it’s been about 6 hours since the injection. I have a slight pain my right arm and general malaise, but not to such a degree that I am sure I’m not imagining it. Hopefully I feel worse tomorrow!

If I do, I’d update my posterior odds to be 2:1 times 2:1 = 4:1. I’d be about 80% confident I received the real vaccine.

If I don’t, I’d update my posterior odds to be 2:1 times 1:2 = 1:1. I’d be about 50% confident I received the real vaccine.

Since I’m young, and younger people tended to have vaccine reactions more reliably, these could probably be treated as (very rough) lower and upper bounds, respectively, e.g. I would be at least 80% or at most 50% confident I received the real treatment depending on the case. In the unlikely event I wake up with crushing fatigue and nausea or aching all over, I reserve the right ignore these estimates and throw myself a party.

2 comments

Comments sorted by top scores.