Ghosts in the literature

Some things are meant to linger: a good conversation, old friends, romantic kisses. Retracted articles? Well, they don’t make the list.

Unfortunately, however, retracted science has a strange way of sticking around the literature with the tenacity of a guest that never leaves. The phenomenon is both puzzling and problematic. Puzzling, because it’s not entirely clear why researchers continue to cite retracted studies in support of new work, even years after the research has been tagged as unreliable. Problematic, because the persistence of discredited findings is a potentially serious threat to future science. 

Paper chase

Retraction Watch’s top 10 list of retracted articles with the most citations includes many papers whose counts continued to grow significantly after retraction. That may be just fine: citing a retracted study – such as Andrew Wakefield’s infamous paper linking autism to the MMR vaccine – in order to tell your readers it has been discredited is a good idea. But it’s alarming how often scientists cite retracted articles as if the papers had never been withdrawn. John Budd, of the University of Missouri, US, has studied the problem of citations of retracted articles. In a 1999 paper,¹ he and his colleagues identified more than 2000 post-retraction citations of 235 papers from 1966 to 1996. Fewer than 8% of those citations noted that the articles had been withdrawn.

Budd and colleagues repeated the study using 1112 articles retracted between 1997 and 2009.² The results were even worse, with only around 6% of citations indicating the retraction. When other researchers looked at citations of one particular paper retracted for fraud, their findings agreed almost exactly with Budd’s.³

Even the work of the most notorious fraudsters manages to persist. Consider the case of Scott Reuben, an anesthesiologist in Massachusetts, US, who served six months in federal prison for health care fraud after being caught in 2009 faking data. Despite more than two dozen high-profile retractions, a 2015 analysis of Reuben’s papers found that more than half continued to be cited long after the case became public. Only a quarter of those citations mentioned the retractions.

Another recent paper found that a retracted paper’s errors continued to propagate.⁴ Unsurprisingly, the researchers found that direct citation of the retracted article kept the tainted findings alive. The good news is that indirect citations – mentions of paper that cited the retracted article – did not appear to propagate the unreliable data.

That some retraction notices would escape the attention of researchers shouldn’t be surprising. Anyone who relies on paper journals exclusively, for example – does anyone like this still exist? – could easily miss a retraction. 

And databases such as PubMed are not always, or even usually, up-to-date with these notices; rather, they depend on publishers to send them information, a process that can lead to lags of several months between the issuance of a retraction statement and the appearance of that notice on PubMed. Sometimes, those notices don’t show up at all.⁵

Retraction engines

One solution to this citing madness is already available: the use of digital links among publishers and papers that notify readers (usually researchers) when a paper has been retracted. Similarly, such systems, including CrossMark, can hoist a red flag when an author cites a retracted article. But CrossMark requires publishers to implement it and incorporate the right sources of ongoing citations, which isn’t universal yet.

Another important step is to have a robust database of retractions. At The Center for Scientific Integrity, we are currently building this, in partnership with The Center for Open Science, thanks to generous grant funding from the MacArthur and Arnold Foundations. Once ready, the database will allow scholars to further study retractions as a phenomenon. Some may even use it to look back at their own work to see what papers they cited that were later retracted. 

What should happen to papers that cite a retracted source is less clear, and probably depends on the circumstance. They may require a correction, editor’s note or no further action. But simply ignoring what happened next in science shouldn’t be an option any more.

Adam Marcus is managing editor of Gastroenterology & Endoscopy News and Ivan Oransky is global editorial director of MedPage Todayand a distinguished writer in residence at New York University’s Arthur Carter Journalism Institute. They are the co-founders of Retraction Watch, a project of The Center For Scientific Integrity