Scientific research is not flawless; nor are the researchers

In these days of the COVID-19 pandemic, scientific research has been a source of much hope – the search for a vaccine, trials for preventive medication, surveys suggesting herd immunity in people, and so on. There is also contradictory information to crush our euphoria (no herd immunity, vaccines to take long, promising pills ineffective, et al). It is difficult to decide which side of the story is true, especially since scientific research is a complex discipline based on several volatile variables. I intend to dwell on some of these variables in my upcoming posts, as they are also an underlying theme in my maiden medical thriller ‘The AquilaTrials.’

 

I’ll begin by emphasizing that the significance of clinical research as a methodical science to discover new treatments is indisputable. Randomized controlled trials are considered the gold standard of evidence. Yet, as any other field, pharmaceutical research is fraught with errors, omissions – and frauds – especially as the noble discipline becomes more commercialized. Scientific decisions are based on evidence, but evidence can be deeply distorted. This note isn’t an account of villainous acts by pharma companies, or a story of conspiracy theories. Statins aren’t killing us, vaccines aren’t useless, and drugmakers aren’t hushing up cures of cancer. That said, trials can be fundamentally flawed, both by design and by error.

 

The current COVID-19 pandemic has exposed several irregularities in research. It has given rise to a global hunger for scientific work and a rush of medical journals to purvey the latest research. This compulsive urge amongst researchers to publish or perish, called the Darsee Syndrome, has also resulted in low-quality and erroneous studies. Two recent cases are worth a mention.

 

The first is a study on “Effectiveness of Surgical and Cotton Masks in Blocking SARS-CoV-2. A Controlled Comparison in 4 Patients,” which was published on Annals.org on 6^th April, 2020. The study that was ironically conducted on four patients was cited very widely (4

2 citations on Google Scholar and countless news articles) as evidence that cotton masks don’t work in containing COVID-19. It was retracted on June 2, 2020 because the authors conceded to not understanding and misinterpreting certain concepts such as the ‘limit of detection.’ (We know how even the WHO has changed its stance regarding the usage of masks.) The underpowered study of four in Annals.org was fairly meaningless even without these errors. The question whether citing it as clinical evidence was a matter of error or otherwise, is left to the readers’ judgement. Follow @zeynep for more on this.

The second study was published on 22^nd May, 2020 in The Lancet – a respectable British journal. It analyzed 96,000 patient records procured by a data analytics company called Surgisphere to conclude that hydroxychloroquine and chloroquine were ineffective against COVID-19 and even increased the risk of death. Irrefutably, the finding could not be ignored: antimalarial drugs once supported by the White House as possible COVID-19 treatments seemed to be not just futile, but downright deadly. Shortly after, some large randomized trials of the drugs were brought to a halt on the basis of this study.

 

But just as quickly, several research irregularities have come to the fore, and scientific questions have been raised about the quality of the data in the paper. One concern was that the study didn’t properly regulate for the likelihood that patients getting the experimental drugs were sicker than the controls. If a physician overseeing two patients with COVID-19 chooses to give hydroxychloroquine to one, it is highly likely that the selected patient is in a worse condition, such as relying on high levels of supplemental oxygen. What if the patient also has co-morbidities and passes away while on the drug? Wouldn’t that make hydroxychloroquine seem like the culprit? For a researcher, knowledge of such variances is essential, but the authors conducting the study were denied access to patient records by Surgisphere. In evidence-based medicine, raw data is everything, and the devil lies in the detail, as pointed out in ‘The Aquila Trials.’

 

On a side note, Surgisphere is under close scrutiny for several malpractices and research fraud. The Lancet has retracted its paper, albeit, not without facing flak for its editing standards. Ostensibly touted as one of the world’s most prestigious medical journals, the Lancet has failed its readers in the middle of a pandemic.

 

More on why to take scientific evidence with a pinch of salt in the next post.