<!–
E. Petersen/Science
This story was supported by the Science Fund for Investigative Reporting. Please help Science pursue ambitious journalism projects.
In June 2020, in the biggest research scandal of the pandemic so far, two of the most important medical journals each retracted a high-profile study of COVID-19 patients. Thousands of news articles, tweets, and scholarly commentaries highlighted the scandal, yet many researchers apparently failed to notice. In an examination of the most recent 200 academic articles published in 2020 that cite those papers, Science found that more than half—including many in leading journals—used the disgraced papers to support scientific findings and failed to note the retractions.
COVID-19 “is such a hot topic that publishers are willing to publish without proper vetting,” even in the face of retractions that made global headlines, says Elizabeth Suelzer, a reference librarian at the Medical College of Wisconsin who has written about problematic citations to a retracted 1998 study in The Lancet falsely linking vaccination to autism.
Both of the retracted COVID-19 papers, one in The New England Journal of Medicine (NEJM) and the other in The Lancet, were based on what appeared to be a huge database of patient records compiled from hospitals worldwide by Surgisphere, a small company operated by vascular surgeon Sapan Desai, who was a co-author on each article. The 22 May 2020 Lancet paper ostensibly showed that hydroxychloroquine, an antimalarial drug promoted by President Donald Trump and others, could harm rather than help COVID-19 patients. Its publication led to a temporary halt in a major clinical trial and inflamed an already-divisive debate over the drug, which has proved to be no help against COVID-19. The 1 May NEJM article corroborated other evidence that people already taking certain blood pressure medicines did not face a greater risk of death if they developed COVID-19.
Questions soon arose about the validity, and even existence, of the Surgisphere database, however, and the retractions followed on 4 June. But of the 200 papers examined by Science—all published after the retractions—105 inappropriately cited one of the disgraced studies. In several cases it was a primary source for a meta-analysis combining multiple studies to draw overarching conclusions. In most, the studies were cited as scientific support or context. Science also found a handful of articles that uncritically cited an influential April preprint based on the same Surgisphere data set, which described the antiparasitic drug ivermectin as beneficial in critical COVID-19 cases. (There is no standard way to retract preprints, however.)
Ivan Oransky, co-founder of the website Retraction Watch, says such blunders occur because “people are either willfully or negligently not checking references.” Many authors copy and paste lists of apparently relevant citations from similar papers without actually reading them, he says. “It’s frightening. It’s terrible, but common.”
Many of the tainted citations appeared in papers published by little-known journals, but at least a dozen found their way into major publications. For example, three articles in PLOS ONE, the prominent open-access journal, cited the retracted papers in discussions of pandemic conditions in Europe. A 28 December paper in the Proceedings of the National Academy of Sciences (PNAS)—one of the most influential journals—discussed the risks and benefits of drugs to treat COVID-19 and noted the Lancet retraction in its citations, yet the text merely noted the paper’s hydroxychloroquine findings as “controversial.”
Related
Editors at those two publications said they would correct the references and take steps to prevent such problems in the future. Renee Hoch, a PLOS ONE editor and publication ethics manager, wrote in an email that the publication relies on authors and its outside, volunteer editors to check citations, and she was caught by surprise when contacted by Science. “We are currently following up on this issue with high priority in light of the implications for public health and ongoing COVID-19 research,” she wrote.
Hoch added that reliance on retracted work, “either directly or in the form of supporting references,” can be harmful. “[W]here the retracted work has clinical implications, this can result in direct risks to patients.”
In a written response to questions about the citation of the Lancet paper, May Berenbaum, editor-in-chief of PNAS, said, “The authors really should have either removed the citation, added more text about why they included it, or cited the retraction notice itself.” Given that no editor or reviewer caught the problem, she said, “I plan to discuss with the staff incorporating such screening into manuscript processing.” A co-author of the paper, biostatistician Clelia Di Serio of Vita-Salute San Raffaele University, later said the reference to the Lancet paper would be expunged.
Some journal editors noted extenuating factors. In Stroke, a leading medical journal, a December article about ischemic stroke in COVID-19 patients referenced the NEJM paper without mentioning the retraction. Stroke Editor-in-Chief Ralph Sacco wrote in an email that the retraction occurred after the article was initially received. However, a revision of the article was resubmitted months after the event, according to the paper itself. Sacco said he would issue no correction because the retraction “is not material to the findings.”
A 16 December paper on SARS-CoV-2 genetics in Nature Communications, another high-profile journal, also cited the NEJM article without reference to the retraction. Elisa De Ranieri, the journal’s editor-in-chief, told Science her journal does not routinely check for “retractions or other postpublication updates.” A lead author of the paper, biomathematician Maik Pietzner at the University of Cambridge, said that although the paper was submitted after the retractions occurred, it was written beforehand, and “the current pandemic requires immediate response.” However, the paper was published 4 months after its submission.
Suelzer says inappropriate citations of retracted articles are hard to excuse. Retraction Watch publishes a free retraction database that has been integrated into a number of automated services to check citations, including scite.ai, Zotero, and RedacTek. The failure to use such tools “is a disservice to the readers and researchers,” Suelzer says. “They are pretty low bars.”
Yet Oransky estimates that in biomedicine, up to 90% of citations to retracted papers don’t mention their fall from grace. “Half the time [as seen with the Surgisphere papers] is an improvement. That’s what’s shocking about it.”