Covid 9/10: Vitamin D

post by Zvi · 2020-09-10T19:00:01.664Z · score: 103 (36 votes) · LW · GW · 36 comments

Last week: Covid 9/3: Meet the New CDC

Imagine there is a simple, cheap, safe and effective solution for Covid-19. 

The solution is something known to be safe. It is widely available for reasonable prices. Any patents have long expired. It is something that people need and benefit from anyway. It’s probably worth doing without the pandemic. It just happens to also have a dramatic effect on Covid-19. 

You might think that once the solution was discovered, everyone would shout it from the rooftops. There would rapidly be studies to confirm the solution if it was even considered ethical to not give the solution to everyone. Production would kick into high gear. The pandemic would soon be over. 

Or, if you’ve been paying attention, you might think that our civilization is so dysfunctional, so inadequate, that none of that would happen. That for no particular reason, or for reasons we’ll get into later, the whole thing would end up mostly being ignored. We’d carry on with all the same arguments, all the same deaths, all the same economic devastation, putting all of our lives on hold. 

That the world you would see would not look much different from our own.

That cynical view looks right. 

The solution has quite possibly been found. We were talking about it, including in the rationalist community, back in February.

Everyone’s mostly ignoring it.

The solution we’re talking about, of course, is Vitamin D. 

Are we certain or even highly confident this is the whole ballgame? No. Of course not. 

We’re not a functional enough civilization to figure this one out in half a year. But we are exactly functional enough of a civilization to start to notice this as a potential solution, and to have run one tiny study that showed dramatic results. If it’s not a dramatic real effect, it’s either taxes or fraud, and I don’t think it’s taxes.

So that’s the headline this week.  

I don’t want to oversell this – it’s still possible this is all a false alarm and there’s nothing to see here, because we dropped this ball so utterly that the first study just came in and it’s tiny. But at this point I’d be very surprised if this isn’t, at a bare minimum, a gigantic piece of low hanging fruit.

Here’s the thing. 

I think about the world in which Vitamin D is a huge deal, and eliminating Vitamin D deficiency would make the pandemic harmless enough that we could mostly let it burn. I think about the world in which Vitamin D matters almost not at all, and if we pushed on it we’d be wasting our imperial focus points for nothing but a few less broken bones and other minor assorted benefits to the otherwise deficient. 

Aside from a few study results and statistics, those words look almost identical. People’s behaviors look the same. So even if it turns out D is useless, that in no way lets anyone off the hook.

In the meantime, for those who tl;dr the later sections, please generously supplement Vitamin D until further notice and get others to do so as well. It’s a freeroll with a huge upside.

There’s a few other minor things as well. In other let’s-not-solve-this-problem news are some vaccine and plasma developments. We also have some school related matters to discuss. 

First, let’s run the numbers.

Positive Test Counts

DateWESTMIDWESTSOUTHNORTHEAST
July 16-July 221175065779726522120917
July 23-July 291102196790324066726008
July 30-Aug 5910026446221294523784
Aug 6-Aug 12930426193118848621569
Aug 13-Aug 19808876338415699820857
Aug 20-Aug 26675456654013232218707
Aug 7-Sep 2550007540112741421056
Sep 3-Sep 9472737243910640821861

 

Nothing at all surprising here.

Deaths

DateWESTMIDWESTSOUTHNORTHEAST
July 2-July 88945591503761
July 9-July 1513805392278650
July 16-July 2214696743106524
July 23-July 2917077004443568
July 30-Aug 518317194379365
Aug 6-Aug 1217386634554453
Aug 13-Aug 1915768504264422
Aug 20-Aug 2615037453876375
Aug 27-Sep 212457593631334

 

Previous trends continue, with a large dropoff in the West. Labor Day weekend may be part of this, as Tuesday’s count was highly suppressed. If there is a small bounceback next week, it should not be alarming. In any case, slow but relatively steady progress seems to be taking place as expected.

Positive Percentages by Region

PercentagesNortheastMidwestSouthWest
7/23 to 7/292.54%5.51%12.32%7.99%
7/30 to 8/52.58%7.26%12.35%6.68%
8/6 to 8/132.30%5.67%14.67%6.98%
8/13 to 8/202.06%5.62%9.41%6.47%
8/20 to 8/261.86%5.78%9.93%5.88%
8/27 to 9/21.87%6.37%9.38%4.78%
9/3 to 9/91.97%6.02%8.48%4.13%

 

Continued improvement in the South and West. Midwest pulls back a bit from last week’s increase, probably not a sign of a real decline yet. THe Northeast number isn’t great. Overall, a mixed bag.

Test Counts

DateUSA testsPositive %NY testsPositive %Cumulative Positives
July 2-July 84,468,8508.2%429,8041.1%0.93%
July 9-July 155,209,2438.4%447,0731.1%1.06%
July 16-July 225,456,1688.6%450,1151.1%1.20%
July 17-July 295,746,0567.9%448,1821.1%1.34%
July 30-Aug 55,107,7397.8%479,6131.0%1.46%
Aug 6-Aug 125,121,0117.3%502,0460.9%1.58%
Aug 13-Aug 195,293,5366.2%543,9220.8%1.68%
Aug 20-Aug 264,785,0566.0%549,2320.8%1.77%
Aug 27-Sep 25,042,1135.5%606,8420.8%1.85%
Aug 27-Sep 24,850,2535.3%547,6880.9%1.93%

Positive test rates continue to decline despite slowly dropping test counts. The upside of seeing low test counts is that it means a given percentage of positive tests is less scary and likely reflects less cases, because those who do get tested are on average higher priority. 

Governor Cuomo would often say that test percentages were low despite large numbers of tests, because he’s innumerate and a blowhard. In this clip he says he ‘reopened the whole economy,’ which he didn’t, and he can’t understand why the infection rate has stayed low. It’s ‘inexplicable to the experts.’ They said ‘it’s going to go above 1% and it could get to 2%.’ 

And then what, you moron? You thought it would double one and then stop? As if by magic? What the flying fork? 

In other words, he took actions he expected to lead to the virus to come back, and can’t figure out why it didn’t. Hint. It’s herd immunity plus heterogeneity (new study on that this week!), and people making responsible private choices, you forking idiot. 

Why oh why don’t people trust the experts?

Please. Please. Please. Do not hold this up as an example of a good leader. Do not consider this man for higher office. Being more responsible than the orange man is not the standard.

Again, everything here is as expected. It’s clear that things are improving slowly, and that our testing capacity has leveled off because no one cares to improve it.

Vitamin D As It Ought To Be

What is going on with Vitamin D?

For a long time, we have known that not being deficient in Vitamin D is important. Like many vitamins, if you don’t get enough, it’s not good. A lot of people are deficient. This study from Europe leads the abstract off with this: 

Vitamin D deficiency (serum 25-hydroxyvitamin D (25(OH)D) <50 nmol/L or 20 ng/mL) is common in Europe and the Middle East. It occurs in <20% of the population in Northern Europe, in 30–60% in Western, Southern and Eastern Europe and up to 80% in Middle East countries. Severe deficiency (serum 25(OH)D <30 nmol/L or 12 ng/mL) is found in >10% of Europeans.

There’s a reason they put Vitamin D into the American milk supply. Google says that the American ‘deficiency’ rate, defined as less than 11 nanograms/ML, is around 11%. This study says the rate of ‘deficient’ Vitamin D, which they define as under 50 nanograms/ML, is way higher than that:

According to data collected between 2005 and 2006 by the National Health and Nutrition Examination Survey (NHANES), insufficient vitamin D levels were found in 41.6% of the 4495-individual sample size. Race was identified as a significant risk factor, with African-American adults having the highest prevalence rate of vitamin D deficiency (82.1%, 95% CI, 76.5%-86.5%) followed by Hispanic adults (62.9%; 95% CI, 53.2%-71.7%) [3]. Additional risk factors for vitamin D deficiency that were identified included obesity, lack of college education, and lack of daily milk consumption [3].

(The most relevant threshold for Covid-19, as suggested by correlational studies, is plausibly in the middle between those two, around 30).

As is often the case, even if Covid-19 is not a consideration, we are in need of More Dakka. In theory you can overdose on Vitamin D, but it’s something that can only be done on purpose with extreme supplementation. Quantities as high as 10,000 ICU/day (50k-100k per week) seem to be safe over extended periods.

For a long time, there have been studies trying to show that Vitamin D helps to prevent or treat various conditions. The verdict has mostly been negative. Lots of trials for various things showed that Vitamin D had no effect. A few did show some effects, but we’ve learned to be suspicious that publication bias plus random effects explain such results. When I look at the net result, I see improvement in respiratory infections and in a few other places, with an emphasis on helping those that are deficient. It seems clear to me that there’s substantial benefits to supplementation when deficient, especially severely deficient. It does not seem clear that there is benefit to supplementation for those that are not otherwise deficient. But of course we don’t get outdoors much these days, and in California doing so would be dangerous and probably wouldn’t get you much Vitamin D – the sky is kind of the wrong color. So given the lack of downsides, supplementation seems like a freeroll already, but not a panacea.

The problem is that Vitamin D has been somewhat of a hype magnet. Many, including one person I know, claim it helps with pretty much everything quite a lot and it’s vital to immune response. Which might be true, but the results so far are not encouraging. This makes the Responsible Person headline to not listen to the hype and not get excited, rather than to point out that the odds still favor using it. Which is a pretty good microcosm of many parts of the general Covid-19 situation. 

It’s clear that correlational studies of Vitamin D with disease find a strong link. Those without sufficient Vitamin D are at higher risk of pretty much everything. Hence the checking of Vitamin D as a solution to pretty much everything. 

The going explanation is that causation at least partly runs in the other direction. If you are in poor health, you’re unlikely to get enough Vitamin D. It’s also plausible that those who supplement tend to have already been healthier as well. Old people with limited mobility are especially likely to be deficient. There are other causal pathways as well. 

That’s also the central feature of the skeptical interpretation of Covid-19 results with regard to Vitamin D.

For a while, we’ve had data that suggests Vitamin D deficiency might play a role, potentially a large role, in who dies or gets severe illness from Covid-19 and who recovers. Some examples that were quoted on LessWrong [LW · GW] were:

– A study in Indonesia found that out of the patients that died from COVID-19, 98.9% of them were deficient in vitamin D, while only 4% of the patients with sufficient vitamin D died.

-A study of patients in New Orleans found that 84.6% of the COVID-19 patients in the ICU were deficient in Vitamin D while only 4% of the patients in the ICU had sufficient levels of Vitamin D.

-A study in the Philippines found that for every standard deviation increase in vitamin D people were 7.94 times more likely to have a mild rather than severe COVID-19 outcome and 19.61 times more likely to have a mild rather than critical outcome.

Those are dramatic differences. They are often phrased in a way as to prevent extraction of the most meaningful information, but the effect sizes are too big for that to be that big a deal here. If we combine these numbers with our ranges for the baseline rate of deficiency above, it seems like a really, really big deal.

The alternate explanation, other than outright fraud which seems in context rather unlikely, is that this is correlation rather than causation. Being sick or prone to get sick means you become deficient, being deficient doesn’t make you get sick. Or maybe both have a common cause. Vitamin D could be a proxy for ethnicity or age, or something else.

Pause for a minute. Assume that this is fully reverse causation or common cause. What would that mean?

It would mean that if you got tested for Vitamin D without changing how you supplement or how much sunlight you get, and you have sufficient levels, you are at very low risk! 

So what if it’s correlational? It still counts for your purposes. Take the New Orleans numbers. If about 50% of people in the USA have sufficient Vitamin-D levels, and only 4% of ICU patients (and presumably deaths from Covid-19 would be similar) do, then being in the 50% with sufficient levels should lower your risk by roughly 92% from baseline, through some combination of getting infected less often and being at lower risk once infected. Instead of something like a 0.3% risk of death, you’re looking at more like 0.03% risk of death. Combine that with being under 50 years old, and even if there’s some double-counting there to correct for, you gotta ask. Unless you’d put someone vulnerable at risk, why are you letting another day of your life go by not living it to its fullest? 

Maybe the other long term risks respond less dramatically. We don’t know, because somehow we have no data on them. Precautionary principle still somewhat applies. Sure. But look at what we’re giving up.

Now we have a full RCT (although not a blind one), and the effect size is gigantic. They treated hospital patients with or without calcifediol (effectively D3), and used ICU admission as the end point. There were 50 patients in the treatment group. None were admitted to the ICU. There were 26 patients in the control group. Half of them, 13 out of 26, were admitted to the ICU. So 13/26 vs. 0/50. Two of the control group died, none of the treatment group.

Do whatever complicated calculations you want. This wasn’t luck.

Possibility zero is that this was luck. This wasn’t luck. 

Possibility one is that this is real or mostly real. If that’s true, everyone supplements, patients get mega-dosed, and we can probably mostly resume our normal lives. This pandemic is no longer worth preventing.

Possibility two is that this is fraud. Always consider the possibility of fraud, especially when the effect size is this big. If anything, I’d like to think a fraud would choose results less dramatic than this, but the type of person who commits fraud doesn’t usually also carefully choose results to be believable or hard to disprove. Cheaters gonna cheat cheat cheat cheat cheat. 

Possibility three is doctor bias. This was suggested to me, since ICU admittance is a human judgment call. But look at the effect size. To get this effect size primarily from bias would mean that ICU admission criteria are meaningless and/or completely ignored. This is no less of an outright fraud, except lives are being risked and damaged for the fraud. I find this implausible.

Possibility four is publication bias or something, but that would require a ton of other studies we don’t know about that we’re rather confident didn’t happen, and also it would be luck. This wasn’t luck.

Possibility five is that this has some relation to them also giving everyone hydroxychloroquine and azithromycin. They did this to both control and treatment groups, considering it the ‘standard of care.’ There’s a large group convinced hydroxychloroquine has been proven useless and another large group that thinks it has been proven effective and is the standard of care, and I’m not going to get into that right now. It was suggested to me that it’s possible that Vitamin D only worked here because of the additional treatment, but the other observations suggest to me that this isn’t true.

I basically see this as a ‘either real or a fraud’ situation. I don’t know how to evaluate the chances of fraud.

This paper makes what I see as a convincing case that eliminating Vitamin D deficiency, at a minimum, should be a high priority. The paper thinks this is so well established for various reasons that further studies would not even be ethical. I think it’s almost never unethical to run tests to confirm your hypothesis when people weren’t otherwise going to get the treatment, because that’s ridiculous, but that’s “medical ethics” for you.

Our “medical ethics” is so screwed up that there is a very thin line between ‘allowed to do this study at all without being “unethical” to treatment group’ and ‘can no longer do the study without being “unethical” to control group despite them doing what they would have otherwise done anyway,’ so you need to grab your opportunities where you can get them.

Thus, hospitals are seen as being in active defiance of guidelines regarding plasma treatments, because they want to run experiments to find out if and how effectively it works, rather than turning on a dime from ‘this isn’t approved’ to ‘this is mandatory’ based on a highly politically motivated announcement.

I am supplementing 5000 IUs of Vitamin D daily. This is not medical advice because nothing I ever write has ever been or ever will be medical advice, “for legal reasons,” but I don’t see any substantial downside to that level. Again, it’s a freeroll. The more people that supplement, the better I’ll feel about it.

That’s the personal, practical side of this. 

On a broader level, replication is the name of the game. We should be doing more and bigger RCTs as soon as possible, and either confirm this is real (and figure out the real effect size) or find out it was a fraud. Test both the megadose for patients, and the regular dose for regular people, although that will take longer to sort out. 

The reasonable alternative would be to make the megadose the standard of care on the spot, given the effect size, decide that RCTs are no longer ethical, and see what happens. I think that’s a mistake, we need more data and it should only take a few weeks to get it, but I would understand it.

The unreasonable alternative is what we are doing. Which is nothing. But it makes sense that we’ve chosen that path. Of course we chose that path. 

I’ll conclude with some thoughts on that, but first, In Other News:

The Perfect as the Enemy of the Good

AstraZeneca Covid-19 vaccine study is put on hold because one person had an adverse reaction that might or might not be related to the vaccine. One. Out of thirty thousand. With neurological symptoms that seem on their face to be something entirely unrelated. Who is expected to recover. 

Now, somehow, the trial is on indefinite pause (it’s unclear what this means, but presumably no new participants and potentially no new doses either, so it could potentially invalidate the whole thing) and the entire vaccine is in jeopardy. 

Bloomberg’s news summary put it this way:

While AstraZeneca Chief Executive Officer Pascal Soriot told a private group of investors the precise diagnosis of the adverse event remains unclear, it could represent a serious setback for efforts to get a Covid-19 vaccine to the world quickly: If AstraZeneca’s review finds the adverse event is related to the shot, all the doses it has already manufactured will be thrown away, Collins said.

In other words, if one person has an adverse event linked to the vaccine, the vaccine is considered useless. All doses will be destroyed. 

If the vaccine ever does anything seriously bad, it’s no good. That’s it. Game over, man. Game over.

You. Fail. Statistics. Forever.

This is utterly insane. 

We could be more like Russia, which already is going ahead with its vaccine without waiting. Its results continue to look promising: Immunogenic in all 76 volunteers, with good neutralising antibody and cell-mediated response. No serious side effects recorded. 

Or meet somewhere in the middle. As Tyler Cowen points out, we could easily already have a vaccine ready at trivial additional cost.

Instead, well, whoops.

Football! Football! Football! Football! Football! Football! Football! Football! Football! Football!

Woo-hoo!

We have football. Ergo, we might have peace.

The NFL returns this weekend. That’s excellent news. So too does much of college football. That’s also excellent news.

So too, in some stadiums, do a lot of the fans. That’s… not as excellent.

I’ve argued repeatedly that not only is football urgently needed, it is not substantially riskier for its players than it was in 2019 – worries about concussions and other injuries are a much bigger deal than Covid-19 risk, as the newly realistic discussions of the NFL make very clear. For college, being on the team is likely safer than the same kid would be as a regular student, as you get better testing and better protocols and less parties. 

None of that applies to fans attending games. Fans at football games are rowdy, completely out of control, yelling at the players and with each other not only because it is fun, but intentionally to create strategic noise. They’re drunk before they get to the stadium and then they keep drinking. The idea that such people could ‘socially distance’ or act ‘responsibly’ is obvious nonsense.

And yet we read about things like North Dakota State University allowing 10,000 fans into the stadium. Many other college stadiums are doing so as well. 

The problem with stupid people with bad motives doing the right thing for the wrong reason is that the next thing they do for the wrong reason is often going to be the wrong thing. 

Our problem of dumb people organizing themselves around being dumb is not football. It’s college.

School Hard With a Vengeance

That is not fair either to football or to college. 

I try to be fair to school. They don’t make it easy. 

This week, we have a student suspended for the display of a toy gun on his zoom video during a virtual class. Think about that. A toy gun, in a video shot, and he got suspended. 

No, wait. That’s the first version of the story I heard. Here’s the full one (WaPo). They sent the cops to his home. He now has a permanent record that he ‘brought a gun to school.’

We have regulations calling for punishment if children are not wearing shoes. For a zoom class. In their own house.

Colleges are charging students tens of thousands of dollars per semester, then kicking students out on the first offense for violating social distancing guidelines that actual everyone will violate from time to time, and keeping the north of $30,000 per semester tuition money. That they often have students take on non-dischargeable loans to pay. As one Twitter account put it, one dean who had just seen Charlie and the Chocolate Factory had an idea. 

Other schools are requiring students to be on campus and pay five figures for a tiny dorm room, because otherwise they won’t be able to do their single lab or other in-person required course, which would make them fail the whole semester. 

Many schools are taking away the scholarships of students who refuse to come on campus for safety reasons, and would prefer to wait things out remotely.  

Then these schools put out studies that call their students ‘an enigma’ because they only are willing to pay half of their outrageous tuition for virtual classes that don’t allow them to party and otherwise do what they came to college to do, but then when they do go to college, they do what they came to college to do, or in academic parlance, ‘engage in activities that present significant barriers to holding in-person classes.’ 

It’s a mystery!

We are home schooling our son. We are constantly pressured from all sides to torture him as if he were being tortured in a real school, because at some point in the future we will need to send him to a real school to be tortured, and if we don’t torture him exactly the same ways now, he won’t be prepared. 

Oh, and this one is my new personal favorite. Iowa Covid-19 rules say if you are close to a Covid-positive person for 15 minutes you need to quarantine… so an Iowa school is shuffling the kids’ seats every 12-14 minutes

We think this belongs in r/funny instead of r/childabuse:

If you’re wondering why we see headlines like Coronavirus: Teens’ anxiety levels dropped during pandemic, study finds, think about it and you’ll figure it out.

Every day it becomes more and more clear what school is for and what it is about. The answer is most definitely not learning reading, writing or arithmetic. 

Let that boy go outside. Maybe he’ll get to be a kid, and soak up some Vitamin D.

Imagine the Epilogue

An interesting exercise is to imagine that we find out that Vitamin D is the solution, it really does reduce risk from Covid-19 by 90%+, and by the end of the year life has mostly returned to normal, with some vulnerable people continuing to take precautions but everyone else accepting that life is never fully safe. 

What happens in that world? How do people react? What behaviors change? How do they judge? Who rises? Who falls?

Take a moment and model what you think happens.

My guess is that the primary reaction is righteous fury. 

We put our lives on hold for six months? And all we had to do was take a vitamin pill? That we mostly should have taken anyway? And all our best people, all our “health experts,” all the “science” we listen to, couldn’t figure that out?

Hell hath no fury. People would believe what they were told even less. Next time, they’d turn to a bunch of quacks hawking miracle cures, even more than they are already inclined to do that – and they’re pretty inclined to do that as it is.

In order words, everyone with authority and power pisses the public off and gets kicked in the nuts, and what little institutional credibility still exists is destroyed. Whoever gets to push the solution officially gets some of that back, but probably not too much. Yes, you found the solution eventually, but what about that whole time before? 

Good luck getting any kind of cooperation on anything, ever again, from pretty much anyone.

There would also of course be great relief, as we rapidly get to return things to normal. There would of course be a chorus of Very Serious People saying that even now we can’t return to normal, and we need to wait for a vaccine, and so on and so on. They might even be listened to and inflict massive damage, both general economic and reputational upon themselves. 

The whole thing would be looked back on as a farce, our civilization as pathetic. 

So maybe there’s upside to the whole thing. If we see ourselves as a pathetic farce that can’t do things, we can work on fixing that. It’s a lot better than what seem like the popular alternative positions: Either fooling ourselves into thinking that we’re great and can still do great things without having to fix things, or being fooled into thinking that we’re irredeemably evil and need to no longer have nice things because we don’t deserve them, and should promptly abandon all our core values and our prosperity. 

In any case, for now on the margin, it would be wrong not to consider this all very good news. We have a lot more of a locus of control over our lives and our fate than we did last week. Use it.

36 comments

Comments sorted by top scores.

comment by Timothy Johnson (timothy-johnson) · 2020-09-10T21:04:41.397Z · score: 10 (8 votes) · LW(p) · GW(p)

Sid Meier seems to have used "More Dakka" as a foundational principle in designing the Civilization series.

https://www.pcgamesn.com/sid-meiers-memoir-civilization

"Efficiency is the goal, which means many iterations, but also getting as much information as possible out of each iteration. One of my big rules has always been, “double it, or cut it in half.” Don’t waste your time adjusting something by 5 percent, then another 5 percent, then another . . . ​just double it, and see if it even had the effect you thought it was going to have at all. If it went too far, now you know you’re on the right track, and can drop back down accordingly. But maybe it still didn’t go far enough, and you’ve just saved yourself a dozen iterations inching upward 5 percent at a time."

comment by rockthecasbah · 2020-09-12T00:51:36.007Z · score: 0 (4 votes) · LW(p) · GW(p)

Bro, it's a great comment, but idk if this is the time for it. There's important shit happening here.

comment by Ben Pace (Benito) · 2020-09-13T20:40:37.853Z · score: 9 (6 votes) · LW(p) · GW(p)

Two things:

  • I've taken vitamin D daily for a few months of the pandemic, will get back to taking it. I've also just told my mum she needs to be on it. Thanks for this post, and pushing me to get back on it.
  • You said we were on it back in February, which is right. It was discussed on LW in the coronavirus justified practical advice thread, where the top answer was Connor Flexman recommending Vitamin D [LW(p) · GW(p)]. It's still a little early, but if the effects are confirmed, we will definitely be celebrating Connor for getting it so right for the right reasons early on, and for telling the rest of us. (His comment is why I started taking it.)
comment by ShemTealeaf · 2020-09-10T19:22:02.992Z · score: 9 (6 votes) · LW(p) · GW(p)

Regarding the Vitamin D study, it doesn't seem like it was placebo controlled. Given the lack of placebo control, I'm not sure how it could be double blind. There were also a number of risk factors where the treatment and control groups had significant differences, most notably diabetes (present in 2.5x as many patients in the control group). If you combine the differences in the characteristics of the groups with the lack of placebo control and blinding and the "fuzziness" of ICU admission criteria, that could start to explain the effect without any deliberate fraud.

Of course, it's still an impressive enough result that we absolutely need to be doing further research. I just think we should be cautious about how much weight we put on a single tiny study with unclear controls and blinding.

comment by rockthecasbah · 2020-09-12T00:48:49.350Z · score: 11 (6 votes) · LW(p) · GW(p)
There were also a number of risk factors where the treatment and control groups had significant differences, most notably diabetes (present in 2.5x as many patients in the control group).

Firstly, two risk factors were more common among the treatment groups: <60 years of age, immunosuppressed & transplanted. Secondly, 3 treatment group patients (6%) were diabetic and 5 control (19.23%) were. Let us take the most generous assumptions for your position, and say that the 3 patients with diabetes in the treatment group did not require ICU and that the 5 in control group all required ICU. This is a strong assumption (aka unlikely).

With these generous assumptions, the study results are now that 1/47 patients in treatment required ICU and that 8/21 in control required. The p value remains .0001. *In order to achieve a p<.05 the lack of blinding/fuzziness would must have failed to send 16 of the 46 treatment group members to the ICU.* That is still not likely without deliberate fraud.

comment by Rafael Harth (sil-ver) · 2020-09-12T07:03:06.930Z · score: 11 (8 votes) · LW(p) · GW(p)

I think it is a really bad idea to disincentivise medium effort comments that point out problems with important studies. I don't want it to be a requirement that you know statistics before you get to question a study.

comment by rockthecasbah · 2020-09-12T12:49:45.557Z · score: 10 (5 votes) · LW(p) · GW(p)

You’re right, thanks for that. I removed that sentence and changed the tone a bit.

comment by ShemTealeaf · 2020-09-14T15:15:56.215Z · score: 3 (3 votes) · LW(p) · GW(p)

I apologize; I made an error in my original comment. I was actually referring to high blood pressure rather than diabetes. 15 out of the 26 people in the control group had high blood pressure, which is greater than the number of people who needed ICU care. Using your (maximally generous) assumptions, we would have zero non-hypertensive patients from either group needing ICU care.

Firstly, two risk factors were more common among the treatment groups: <60 years of age, immunosuppressed & transplanted.

Absolutely true, but the overall risk factor prevalence was still significantly higher in the control group. Furthermore, I'm not sure if all risk factors are created equal. Regardless, the overall point is that the two groups had significant differences in important characteristics.

*In order to achieve a p<.05 the lack of blinding/fuzziness would must have failed to send 16 of the 46 treatment group members to the ICU.* That is still not likely without deliberate fraud.

I think it's more likely that they sent a few of the control group members to the ICU unnecessarily. If you figure that the difference in risk factors between the two groups accounts for a couple of the extra ICU cases, the placebo effect accounts for another couple, and unnecessary ICU admission accounts for another couple, it brings the P-value up pretty dramatically. I'm not statistically literate enough to know how to properly adjust for those factors and get an exact number, but it doesn't seem to require deliberate fraud.

Just to be clear, I still think that there is probably at least some sort of real effect here. I'm just advocating caution in interpreting the results of a tiny study with clear flaws. I don't really understand why there was no placebo control or double blinding, and that makes me more suspicious that there are other flaws that I'm not educated enough to notice. For example, the way that they describe the ICU admission criteria suggests that the presence of a comorbidity is itself a factor for ICU admission. If that's the case, the differences in the risk factor numbers become even more important.

comment by rockthecasbah · 2020-09-21T16:13:30.878Z · score: 1 (1 votes) · LW(p) · GW(p)

https://twitter.com/AlvaroDeMenard/status/1304399452816519169/photo/1 - probability of reproduction from one forecaster for DARPA SCORE study. The distribution is bimodal because the lower hump is p=.05 and the upper hump is some lower p-value like p=.01 or something. It looks like even the higher p values have reproduction rates at .8 to .9 . This updates toward Shem's skepticism. Even though the p value is very small, reproduction rate is still stuck at .85 for good studies. Since this study has the problem's shem pointed out, we might expect a reproduction probability lower at like .75. So a likelihood ratio of 3:1.

I could imagine people with knowledge of the subject having sufficiently lower priors.

comment by Mitchell_Porter · 2020-09-11T04:14:32.710Z · score: 5 (4 votes) · LW(p) · GW(p)

"risk factors... notably diabetes"

This was my thought - that vitamin D deficiency here might be a proxy for ill health in general, with other conditions being the true risk.

comment by rockthecasbah · 2020-09-11T23:53:48.774Z · score: 1 (1 votes) · LW(p) · GW(p)

Bro, if diabetes is the common cause the RCT would have failed.

comment by IlyaShpitser · 2020-09-11T16:43:28.621Z · score: 7 (10 votes) · LW(p) · GW(p)

"Unless you’d put someone vulnerable at risk, why are you letting another day of your life go by not living it to its fullest? "

As soon as you start advocating behavior changes based on associational evidence you leave the path of wisdom.

---

You sure seem to have a lot of opinions about statisticians being conservative about making claims without bothering to read up on the relevant history and why this conservativism might have developed in the field.

comment by rockthecasbah · 2020-09-11T23:51:31.650Z · score: 6 (4 votes) · LW(p) · GW(p)

Zvi has repeatedly cited a paper arguing that the FDA kills more people by preventing effective treatments than it saves by preventing bad treatments. Not having followed that link personally, the results suggest that pub health statisticians are miscalibrated in expectation.

One explanation for why pub health statisticians are miscalibrated in a causing-death-by-inaction direction is that they are punished for deaths caused by action but not deaths caused by inaction. I’m this model, conservatism (aka miscalibration toward inaction) is a result of the lived and publicized experiences of people in the field. This seems a great explanation for both the experts norms and the paper results.

Are you contesting that statisticians are miscalibrated in expectation in the utility they cause?

I think the “Bailey to your Motte” is that people are bad at predicting who they might infect, so this advice could lead to greater deaths. I think Zvi could have phrased it more carefully. But the broader point needed emphasis, that we are loosing so much for something we could fix. And that fix might not be so hard. That point is more important than quibbling one darn sentence.

comment by IlyaShpitser · 2020-09-13T15:58:22.764Z · score: 6 (10 votes) · LW(p) · GW(p)

A whole paper, huh.

---

I am contesting the whole Extremely Online Lesswrong Way<tm> of engaging with the world whereby people post a lot and pontificate, rather than spending all day reading actual literature, or doing actual work.

comment by Periergo · 2020-09-13T21:26:54.070Z · score: 4 (5 votes) · LW(p) · GW(p)

which literature do you recommend?

comment by df fd (df-fd) · 2020-09-11T06:58:20.308Z · score: 7 (4 votes) · LW(p) · GW(p)
– A study in Indonesia found that out of the patients that died from COVID-19, 98.9% of them were deficient in vitamin D, while only 4% of the patients with sufficient vitamin D died.
-A study of patients in New Orleans found that 84.6% of the COVID-19 patients in the ICU were deficient in Vitamin D while only 4% of the patients in the ICU had sufficient levels of Vitamin D.
-A study in the Philippines found that for every standard deviation increase in vitamin D people were 7.94 times more likely to have a mild rather than severe COVID-19 outcome and 19.61 times more likely to have a mild rather than critical outcome.

I would be highly skeptical of this since it came from a facebook post and I have not been able to verify the content. If someone else can do the leg work to independently verify what had been said?

comment by jimmy · 2020-09-11T03:46:41.824Z · score: 6 (7 votes) · LW(p) · GW(p)
There were 50 patients in the treatment group. None were admitted to the ICU. There were 26 patients in the control group. Half of them, 13 out of 26, were admitted to the ICU. So 13/26 vs. 0/50.

That's not what the paper says

Of 50 patients treated with calcifediol, one required admission to the ICU (2%),

The conclusions still hold, of course.

comment by Joe_Collman · 2020-09-12T20:19:55.331Z · score: 4 (4 votes) · LW(p) · GW(p)

Thanks for the post (and indeed the others).

This is interesting and, I think, important w.r.t the RCT: video looking at the molecular biology of vitamin D (the first half is interesting too)

With caveats that: I don't know what I'm talking about; this is a simplification of one research group's understanding; this isn't medical advice.... tl;dr:

There are two forms of vitamin D: 1,25-D as an agonist in VDR activity, 25-D as an antagonist. (VDR is involved in immune response)
The common medical test tests only for 25-D (this is not good - the 1,25 to 25 ratio is important).
A low 25-D level can be indirectly caused by the body's 'desire' to increase VDR activity - i.e. not as a consequence of vitamin D deficiency.
High doses of vitamin D may well be helpful in severe Covid-19 cases through dampening a dangerous immune response (cytokine storm...).
It's important not to be deficient in vitamin D, but high doses in healthy people may actually dampen initial immune response.
Much more research is needed.

I'd be interested if anyone has any thoughts/critiques on the more technical issues - it's really not my area. My personal takeaway is to take vitamin D supplements, but not to go crazy with them. In particular, on this view being deficient is not good, but it's not likely to be what's driving the seriously negative outcomes in the RCT.

comment by rockthecasbah · 2020-09-13T16:50:35.176Z · score: 1 (1 votes) · LW(p) · GW(p)

So you are suggesting that the mechanism between the vitamin D pills and the outcome might not be through resolving vitamin D deficiency but some other more specific pathway.

Does that update is as to the robustness of the study?

comment by Joe_Collman · 2020-09-13T17:22:33.182Z · score: 4 (3 votes) · LW(p) · GW(p)

What I take from the video is that the study is good, but isn't really saying anything about vitamin D deficiency: it's saying that high doses of D are useful in dampening immune response once the largest danger to the body is its own immune response - i.e. once it's in/near a cytokine storm situation.

So I think it's very likely that the high dose vitamin D is helpful in mid/late stage, but that this study says very little about the impact of your pre-existing levels of vitamin D. (again, if the video is correct: this is not my area)

If you buy the take in the video, you wouldn't want very high doses of D while you're still healthy. I suppose in an ideal world you'd want to experiment on yourself with different D supplement levels, and get your 25-D and 1,25-D levels checked.

Clinically, it seems to me that the main question is when you'd want to start giving vitamin D, and at what dosage. If the video's take is correct, it seems likely that high doses for a healthy person isn't likely to help (beyond the point where you're comfortably not deficient), but that high doses in the more serious Covid-19 cases are a good idea.

You'd need research to get a clearer idea of dosages and starting points.

If this is correct, then the study's implications for hospital treatment look good (but need more research).

If it's correct, the corollary that those healthy people with good vitamin D levels are around 92% safer than baseline already does not seem to be valid. To conclude that, you'd need to have tested the ICU patients' D levels while they were still healthy. On this model, low 25-D levels when you're already sick can be part of the body's response to the illness (often a counter-productive response).

Of course that'd be a great thing for someone to do some research on: for Covid-19 patients who happen to have had a recent vitamin D test when they were healthy, what is the correlation between the healthy D levels, and Covid outcomes? Have there already been studies like this? I don't know, but I haven't looked.

comment by df fd (df-fd) · 2020-09-13T22:09:25.471Z · score: 3 (3 votes) · LW(p) · GW(p)

but there is a great body of work proving that Vit D is beneficial for treating respiratory infection in general [I can cite on request] so unless most respiratory infections secretly involved cytokine storm [which I don't believe to be true] or Vit D is beneficial in other ways than what you suggest.

https://www.lesswrong.com/posts/LwcKYR8bykM6vDHyo/coronavirus-justified-practical-advice-thread?commentId=KEAc428gk4F6Ys3tv [LW(p) · GW(p)]

comment above suggested that the benefit is from Vit D prevent the pneumonia from covid from getting worse, and I found the argument persuasive

comment by Joe_Collman · 2020-09-13T23:02:10.106Z · score: 1 (1 votes) · LW(p) · GW(p)

Oh that's entirely plausible. I should have emphasised that this may well be something that's going on; it certainly doesn't make it the only thing. Again, I have no expertise in this area - so mainly I'd like people with more knowledge than I to watch the video and draw their own conclusions.

My main takeaway with respect to the RCT in the post is that measurements of the 25-D being low in patients can't be taken as evidence of deficiency if the 1,25-D levels are simultaneously high. So it's premature to draw conclusions about non-deficient healthy people being ~92% safer than baseline.

It still seems right to me that not being deficient is important, and that vitamin D treatment is important.

comment by Zvi · 2020-09-11T00:40:22.636Z · score: 4 (2 votes) · LW(p) · GW(p)

This has been fixed/updated, please reimport.

comment by Raemon · 2020-09-11T01:17:07.317Z · score: 6 (3 votes) · LW(p) · GW(p)

Done. (FYI, when you say "re-import" all we do is copy-paste the post from your blog into the LW editor. Happy to keep doing it but there's nothing magic going on)*

*okay we click one additional button on the edit-post page to convert it from HTML to LessWrong Docs.

comment by ErickBall · 2020-09-14T14:09:44.093Z · score: 2 (2 votes) · LW(p) · GW(p)

I would love to have a link to send my parents to convince them to take Vitamin D as a prophylactic. The one RCT, as noted above, has various issues that make it not ideal for that purpose. Does anyone know of an article (by some sort of expert) that makes a good case for supplementation?

comment by mingyuan · 2020-09-13T20:26:10.971Z · score: 2 (1 votes) · LW(p) · GW(p)

Thanks! I sent my parents this post and a bottle each of 5000 IU vitamin D capsules :)

comment by Theodore Ehrenborg · 2020-09-12T22:33:40.895Z · score: 2 (2 votes) · LW(p) · GW(p)

As Tyler Cowen points out, we could easily already have a vaccine ready at trivial additional cost.

It looks like that post was written by Alex Tabarrok.

comment by ChristianKl · 2020-09-11T19:07:12.011Z · score: 2 (1 votes) · LW(p) · GW(p)

In theory you can overdose on Vitamin D, but it’s something that can only be done on purpose with extreme supplementation. 

This is only true if you take K2 along with your Vitamin D. I know a fellow rationalist who thinks they overdosed on 5000 IU Vitamin D when they took it without K2.

See also: https://www.lesswrong.com/s/5CNs9wmHWFQTNjFKo/p/c5aycbSsSc38XWPEc

comment by adamzerner · 2020-09-11T05:18:12.490Z · score: 2 (1 votes) · LW(p) · GW(p)

Any thoughts on how Trump's admission to lying about covid to prevent panic changes things?

comment by Zvi · 2020-09-11T13:05:42.764Z · score: 6 (3 votes) · LW(p) · GW(p)

It won't.

comment by Vanilla_cabs · 2020-09-14T08:29:34.572Z · score: 1 (1 votes) · LW(p) · GW(p)

Thanks for voicing some of the things I thought better than I ever could.

I've noticed a trend on LW of cheap jabs at "anti-vaxxers". To me this seems like a partisan label which just makes it harder to voice legitimate concerns about vaccines. Like any medical treatment, we should ask:

  • how bad is the disease it's meant to ward off?
  • how much is it gonna cost? (and who's gonna pay)
  • how efficient is it?
  • do we know the side effects?

That said, AstraZeneca abandoning a vaccine for one patient with an adverse reaction seems absurd. I notice I am confused, so I wouldn't be surprised if it wasn't the whole story.

Prediction: even if vitamin D (or HCQ) is proven to greatly reduce mortality, I don't think there will be any consensus outside scientific circles. The matter has been far too politicised for any side to back out.

comment by ChristianKl · 2020-09-15T11:45:36.229Z · score: 7 (2 votes) · LW(p) · GW(p)

To me this seems like a partisan label which just makes it harder to voice legitimate concerns about vaccines. 

I don't think that's the case and I do happen to be a person who raised legitimate concerns about vaccines in the past. 

do we know the side effects?

That's not a good question. We never know all the side-effects. We make decisions in uncertainty and have to think about the expected value of our decisions given the uncertainty that we have. 

That said, AstraZeneca abandoning a vaccine for one patient with an adverse reaction seems absurd. 

Clinical trials are a heavily regulated domain which rules that are not made for fast development of new drugs. 

comment by Vanilla_cabs · 2020-09-15T14:47:47.190Z · score: -1 (2 votes) · LW(p) · GW(p)
I don't think that's the case and I do happen to be a person who raised legitimate concerns about vaccines in the past.

So you don't mind being called an anti-vaxxer? Maybe in the US it's not a big deal, but in France where I am, you might as well be called a flat-earther.

We never know all the side-effects. We make decisions in uncertainty and have to think about the expected value of our decisions given the uncertainty that we have. 

Of course. That shouldn't keep us from doing our best to find out the side effects in the time we have, and to keep searching afterward. And to use that knowlegde when wheighing the benefit/risk ratio of the treatment.

comment by ChristianKl · 2020-09-15T16:27:27.354Z · score: 7 (2 votes) · LW(p) · GW(p)

Nobody has called me an anti-vaxxer. Generally, I'm more concerned with giving anti-vaxxers who don't care for science more ammunition then I'm concerned with being called that term. 

The danger of people dying because I wasn't careful with qualifying criticism is much more serious then getting called a name.

That shouldn't keep us from doing our best to find out the side effects in the time we have

How serious are you? What sounds to me like babble.

Calling for doing our best basically means doing a lot of human challenge trials for which people like me and Zvi called for months ago. The current consensus is that most people don't want to do our best to search for side effects in the time that we have but rather want to follow traditional research processes.