How do the ivermectin meta-reviews come to so different conclusions?

I had a quick look and essentially it seems the latter found fewer studies (10 vs 19) and therefore fewer patients (1173 vs 2768)*

They have similar central estimates for RR of all cause mortality (0.37 vs 0.31) but due to having more patients the former has tighter CI (0.15 to 0.62) and concludes that there is an effect but the latter has wider CI (0.12 to 1.13) and concludes that there isn't an effect.

The latter could claim that there is as yet insufficient evidence of an effect based on the studies in their analysis but not that these isn't an effect. I especially take issue with the claim that "IVM is not a viable option for treating COVID-19 patients" when they themselves take such pains to talk about how low quality much of the evidence is!

The two meta-analyses also differ on their ratings of different papers - for instance the largest study (n=400, Lopez-Medina et al.) is rated as High quality (7 out of 7) in the former but at high risk of bias in the latter (due to deviations from intended interventions). 

Scanning the paper there are a few issues. For the most part the problems are mitigated but there could still be issues:

• There was a period of 17 days where the placebo group were receiving ivermectin(!)
  • The people from these 17 days were excluded from the primary analysis and additional subjects were enrolled
• The primary outcome was changed during the study
  • This occurred ~30% through the study.
  • The reasoning for changing from the old outcome seems reasonable
  • It's hard to comment on whether the selection of the new outcome could have seen bias
• Placebo was changed during the study from dextrose water to something tasting more like ivermectin
  • This was ~25% through the study
  • Results did not differ much between the 2 placebo groups
  • I don't feel like this would make a massive difference but I'm not sure

This paper is fairly typical of the quality of the studies (according to meta-analysis 2) or on the top end of study quality (according to meta-analysis 1) which causes me some concern.

In conclusion, if I was offered Ivermectin I would take it at this point (side effects seem to be small) and might even look to sign up to a trial if I had COVID - in the UK some people would be eligible for this one.

* 6 studies were common to both analyses.

Some general comments about medical research. Source: I have studied the statistics books in detail, and have read several cubic meters of medical papers and learned most of the lessons the hard way. 

When reading medical papers look for 

1. Funding sources for the study or for the authors of the study (e.g. "speaking fees" and "consulting fees"). He who pays the piper calls the tune. 

2.  Statistical incompetence, which is rife in medical research. For example, you routinely see "lack of statistical significance" interpreted as "proof of no effect". 

3. Pre publication of the study design, end points and intended statistical analysis. There is a lot of scope to move the goalposts and engage in p-hacking and other nefarious activities. 

4. Differences between the abstract and the text. Often you can read the abstract and wonder if it refers to the same paper .

5. In meta-analyses look for whether the selection criteria were adhered to  or not or whether subjective criteria were used to exclude inconvenient studies. 

6. Financial interests. For example it is notable that countries like India, that make generic drugs, appear to be more favourable to generic drugs. Meanwhile in the US, there seems to be a strong bias in favour of drugs in patent. 

7. Read the methods section very carefully. Once you have read enough papers this will become instinctive. 

8. Be ready for the vast majority of papers to be of low quality and worthless. 

9. I routinely see studies rigged to deliver a predetermined outcome. For example, if you want to find a non-statistically significant effect which can be misrepresented as "no effect", then run a small study, for a short period, and use suboptimal doses or take other measures to minimize differences between the groups compared.

The obvious difference is that the second does not include Elgazzar, while the first includes Elgazzar, which is bad for the first one because Elgazzar faked its data so incompetently it has been retracted: https://grftr.news/why-was-a-major-study-on-ivermectin-for-covid-19-just-retracted/ https://gidmk.medium.com/is-ivermectin-for-covid-19-based-on-fraudulent-research-5cc079278602 https://www.theguardian.com/science/2021/jul/16/huge-study-supporting-ivermectin-as-covid-treatment-withdrawn-over-ethical-concerns

Another meta-analysis (Bryant et al) has a very similar title but positive claims Ivermectin for Prevention and Treatment of COVID-19 Infection: A Systematic Review, Meta-analysis, and Trial Sequential Analysis to Inform Clinical Guidelines.

The authors have put out an official rebuttal of the negative meta-analysis which is an interesting read and point to many of their perceived flaws.

The comments on the preprint of the negative study (Roman et al) are also interesting.

For instance:

Hi, I'm Dr.Niaee and I was surprised that even basic data from our RCT is completely mispresented and is WRONG. We had 60 indivisuals in control groups and 120 in intervention groups and even this simple thing is mispresented.

And:

after your "mistake" inverting the control and IVM arm of the Niaee study, the RR goes from 1.11 to 0.37 yet you dare to not change a single word in your conclusion

My current impression is that the negative study is not very high quality at the moment, for any reason among rush to publish, incompetence or malice.
For sake of argument I still have to look at what studies Roman et al did include that was omitted by Bryant et al and Hariyanto et al as that would reveal any pro-ivm biases.

The negative meta-study is borderline malicious.

"This article has an embarrassing history whereby treatment arms in the study of Niaee were reversed, attracting protest from Dr Niaee himself. This egregious error has been corrected in the revised version, but with no change to the Conclusions in spite of dramatic change…" - from BIRDGroup twitter. 

Pubpeer is also useful in cases like this:

https://pubpeer.com/publications/955418F3D4D39742CFFA8C1B023AA3

I googled a bit to see whether ivermectin can be ordered online and while there are website that superficially look like normal online pharmacies selling it, those seem to be lacking an impressum and seem pretty shady.

The pharmacies that I found that sell it and aren't shady all online give it out for prescriptions.

Because it's political.  Some people are invested in Ivermectin being effective, other people are invested in it not being effective.  The extant studies are all inconclusive due to a small N, and mostly have problems with their methodology; if you pick and choose your studies in the right way you can get whatever result you want.

And the individual studies are often extremely bad.  I note Cadegiani et al, who claim that Ivermectin (and also Hydroxychloroquine, and also Nitazoxanide) are each so effective, either individually or combined (they didn't bother to track which patients got which drugs) that it is unethical to use a placebo group in studying those drugs.  I'm not sure how Elsevier can be affiliated with a journal that publishes material like that and retain any credibility.

By reading them?

It seems they hit different studies and one can check that. One also says that everything is low quality of evidence and other says everything is very applicable to be analysed.

It is also a bit funny how one of papers goes study by study "IVM reduced mortaliy but QoE was low" and then goes on to conclude that overall "IVM does not reduce mortality"

The two statement are not neccesarily so much in conflict, they are just weaseled in opposite directions. One of the them says "suggests" and other says "is not proven" which you get if you have a faint trace going one way.