ABOVE: MODIFIED FROM
© ISTOCK.COM, TARIKVISION
Much of the public first heard of "preprints" on March 20, 2020, just nine days after the World Health Organization declared a pandemic, when a too-good-to-be-true treatment for SARS-CoV-2 infection appeared on medRxiv. The paper, which described a small clinical study and had been posted only four days after the last patient enrolled, touted the apparent benefit of hydroxychloroquine (long used for rheumatic diseases) and azithromycin (a familiar respiratory antibiotic) for treating COVID-19. It included medRxiv's standard disclaimer: “This preprint reports new research that has not been certified by peer review and should not be used to guide clinical practice.”
The International Journal of Antimicrobial Agents (IJAA) accepted the manuscript on the same day it appeared on medRxiv, just one day after it was submitted to the journal. Although the preprint process often allows crowdsourced discussion and debate that may ultimately improve the paper before formal publication, in this case the authors (one of them the editor-in-chief of IJAA) precluded that process by simultaneous submission.
If the preprint becomes the end product (or a “no-print,” or no formal publication) for some authors, we risk being awash in a large quantity of publications of low quality.
Also on March 20, the same day the preprint was posted online and accepted by IJAA, President Donald Trump endorsed hydroxychloroquine (HCQ), saying “I feel good about it” during a White House press conference. Subsequently, the treatment gained peer-reviewed support (though some of these studies were later retracted), and by June 15, the US government had stockpiled 63 million doses of the drug. US prescriptions for HCQ had spiked to record levels in April as COVID-19 case numbers climbed, with most prescriptions coming from primary care providers.
Within weeks, more-sober analysis yielded the conclusion that HCQ has no proven role in treating SARS-CoV-2 infections. By this time there had been several preprints about the antiparasitic drug ivermectin, and again we have gone down another rabbit hole of sometimes questionable studies lacking peer review.
See “A Surge in Pandemic Research Shines a Spotlight on Preprints”
Preprints aim to bridge the time gap between submission and publication in a peer-reviewed journal, something that becomes even more crucial as humans navigate a global public health crisis; but they also run the risk of spreading shoddy research. Peer-reviewed research is not infallible, of course. In the earliest months of the COVID-19 pandemic, leading medical journals began cranking out studies that pertained to the new pathogen, and more articles received peer review with shorter turnaround times. This allowed some questionable research to slip through and led to high-profile retractions in The Lancet and The New England Journal of Medicine, among others.
Preprints also stand to disrupt traditional science publishing if researchers see them as an end product rather than a stepping stone to peer review. According to a preprint posted in October 2020, fewer than one-half of COVID-19 preprints posted to arXiv, bioRxiv, and medRxiv in January of that year achieved peer-reviewed publication nine months later; fewer than one in five preprints posted in the whole study period reached full publication by October. This low rate could reflect the time required for peer review and revision. Usually this takes several weeks to a few months, depending upon manuscript quality, reviewer responsiveness, editorial decisions, and author revision speed. Sometimes it takes up to a year, particularly if the primary journal rejects the manuscript. It is likely that many more orphan preprints now exist. Some orphans will later find homes, but some will not. New orphans will arrive in the course of time.
The low speed and high rigor of peer review are not the only causes of orphan preprints. Some authors choose to stop with the preprint with no attempt at further publication. Preprints give authors a permanent digital object identifier (DOI), and preprints are immediately searchable and citable. These features create incentives for preprint authors to skip the scrutiny of peer review.
If the preprint becomes the end product (or a “no-print,” or no formal publication) for some authors, we risk a flood of publications of low quality. Peer review aims to assure the quality and veracity of published scientific work. Peer review and editorial guidance assist authors in refining and clarifying their messages. Omitting this step threatens to erode not just the enduring record of science but also the public’s trust in new research.
Nevertheless, preprints have a role to play. Despite long-standing efforts by journals to expedite publication of breaking research with time-sensitive public-health implications, we obviously cannot provide expedited review for every manuscript. We rely on the altruism of unpaid reviewers who voluntarily share their time and expertise. Preprints can bridge the time to later publication.
So how do we fix the problem of dubious preprints? For one thing, I propose that preprints no longer receive a permanent DOI. Instead, preprints should have a limited shelf life with a link that expires within 12 months. This is more than enough time to complete peer review even with a rejection or two along the way. Any extension of the DOI should require proof of journal submission.
In addition, preprints could display a digital watermark on all pages—perhaps something like “Preprint, not peer reviewed”—instead of a small disclaimer just on page 1 as medRxiv does now, to potentially deter the reporting of unverified data by the lay media. Preprints should also have a central registry—much like ClinicalTrials.gov—that links the preprint to the subsequent peer-reviewed article (and its DOI) or records its automatic retraction by expiration. This should motivate authors to complete the peer-review process and should improve the quality of the enduring record of science and medicine.
Recent years have brought accelerating changes to scientific publishing, and each innovation brings new promise and new problems. The rise of open access (OA) journals (one of which I edit) promised to make knowledge free to anyone with internet access anywhere in the world, but it also enabled anyone with a server to litter our email inboxes with invitations to publish in so-called open access predatory journals. As with OA, preprints (and the internet that made both possible) have a role in the dissemination of new science. Just as “news is the first rough draft of history,” preprints may become the first rough draft of science. But we need common sense measures to assure that they do not become the final version of record.
