Safeguarding science against falsehood demands debate

False scientific claims are, unfortunately, a reality. The motivations and reasons behind them are various: human error; psychopathic delusion; the lure of recognition and fame; career progression; peer pressure; and, most benignly, over enthusiasm can all play a role. There are some prominent examples. 

The late Martin Fleischmann, an eminent electrochemist of highest international standing, driven, no doubt, by his unbound enthusiasm for science, announced the discovery of cold nuclear fusion in 1989, together with his colleague Stanley Pons. Their claim had immense global importance and laboratories everywhere started to work on it, some claiming encouraging results, others unable to reproduce the effect. To date no functional cold nuclear fusion reactor has been built, which suggests that the original report was either flawed, or that the data were over interpreted. 

12 years later, Jan Henrik Schön, working as a postdoc at Bell Labs, appeared as the first author on a series of papers on nanoelectronics, published at an unrealistically high rate in prestigious journals. Schön’s ascension was abruptly ended when one referee noted irregularities in a graph that suggested the data were fabricated. It turned out Schön had hardly conducted experiments at all, simply producing fake data sets that would match his supervisor’s hypotheses.

Veterinary scientist Hwang Woo-suk lost his job in 2007 and was faced with possible imprisonment after he admitted having falsified claims to have successfully cloned human embryos for stem cell production. He has since moved on to the cloning of mammoths. 

Other errors have had significant societal consequences, such as the force-feeding of spinach to generations of children because of the erroneous claim the plant had an unusually high iron content. 

Correctional facility

The list of cases where either bad science or simply fraud has been uncovered and dealt with is long, and what, in hindsight, many of them have in common is not only the failure of the peer review system, but also a striking lack of post-publication debate. 

Ideally, science ought to be self-correcting. The cold fusion case is an example of how reproducibility protects science, but this was a high-profile exception. In general, once a new phenomenon has been described in print, it is almost never challenged unless contradicting direct experimental evidence is produced. Thus, it is almost certain that a substantial body of less topical but equally false material remains archived in the scientific literature, some of it perhaps forever.

Once a phenomenon has been described in print, it is almost never challenged

We need more debate in the scientific literature to weed out the errors and falsehood before they cost us time and money or, worse, become embedded in the underlying brickwork of the scientific edifice. But attempts to instigate such open debates are often met with significant editorial opposition. A recent example is Raphael Levy’s criticism of results published by Francesco Stellacci. Levy published his case in the journal Small, following several rejections by other journals and a three-year editorial process during which eight peer review reports were considered. These unusually exacting standards were applied because Levy’s paper makes the unconventional move of challenging the interpretation of experimental data presented in a number of earlier publications by Stellacci, a well-known Italian scientist, and his co-workers. Stellacci published experimental evidence that he interpreted as the existence of stripe-like domains of ligands on gold nanoparticles and, in several further high impact publications, went on to explore differences in properties between striped and non-striped particles. Levy disagrees with Stellacci’s interpretation, and constructs a scientific case against the stripes’ existence, and hence against the differences in properties. Stellacci published a rebuttal and the debate is now raging within the relatively small community of scientists interested in the ligand shell of gold nanoparticles. Perhaps a consensus will result, perhaps not. The truth, as usual, may be more complicated, and a deeper understanding of what these particles really look like may come from elsewhere, for example from the increasing number of careful single crystal x-ray studies. 

Better by debate

Compared to cold fusion, stripy nanoparticles are relatively inconsequential, so without this debate, their veracity might have remained unchallenged, like iron-rich spinach. At worst, if Levy is wrong, Stellacci’s assertions will be strengthened and given more authority. 

These debates are in everybody’s interest, including those whose research is debated, so why are journals so hesitant to enable them? The frequency of such secondary reports is unlikely to be high – most scientists are fully absorbed by their own projects and have neither sufficient time nor interest to scrutinise their colleagues’ work in such detail. But when it does happen, journals should act swiftly, welcome controversy and seize the opportunity for self-correction. And scientists should appreciate the need for critical debate. What are we afraid of?

Mathias Brust is professor of chemistry at the University of Liverpool, UK