Don’t let the burden of proof squeeze the life out of ideas

How important is it that papers published in scientific journals are correct? Put like that, it might sound an absurd, even dangerous question. Isn’t science already struggling with a reproducibility crisis? And isn’t it vital, in the midst of a global pandemic, to ensure that published studies of, say, pharmaceutical interventions in this disease are reliable?

Still, it’s an open secret that almost anything can get published somewhere, even if that’s in an obscure or dubious outlet. The fact is peer review is imperfect, and no guarantee of veracity. Yet even if the process were not such a flawed filter, asking ‘is it correct?’ isn’t necessarily the sine qua non of scientific publication we might imagine. There is another question that is arguably as important for the progress of science: is it stimulating?

These considerations are highlighted by a recent report that a classic cross-coupling reaction called the Suzuki reaction, which forges a carbon-carbon bond between unsaturated organic groups, can be conducted without the traditional palladium catalyst but instead with a small organic molecule.¹ The validity of this surprising metal-free process has now been challenged by new studies suggesting that trace amounts of palladium catalysed the reaction after all.^2-4

When I wore an editor’s hat, the key question seemed to be: is this paper now ready to be seen?

On the face of it, this is just business as usual: a demonstration of science’s self-correcting capacity. Claims that really matter (like this one) are sure to be examined by others, bringing potential flaws to light. Some might ask, though, whether the apparent problems should have been identified during peer review, saving other labs the bother of debunking the work (and the original authors the consequent embarrassment) and keeping the literature free of clutter. A better question, however, is how high the bar should be set before a paper is published. There’s some justification in the common complaint that referees keep asking for one more check after another, some of which require substantial new work, before acceding to publication. And there’s a danger that exciting or important new findings can be delayed or stifled by excessively stringent demands for claims to be made airtight.

The very nature of science as contingent knowledge means that the criteria for publication have to be broader and more intelligent than that. When I wore an editor’s hat, the key question seemed rather to be: is this paper now ready to be seen? This meant, among other things, that obvious potential sources of error had been excluded and that the conclusions did not over-reach the data or neglect caveats and uncertainties. Also, the demands on the evidence needed to some extent to be proportionate to the claims made: as Carl Sagan famously put it, extraordinary claims require extraordinary evidence.

But there are times when it can be productive to put a finding out there even when it remains unclear if it will hold up or what it might mean. Two notorious examples might help us judge where that contentious threshold lies. In retrospect the 1988 Nature paper on the ‘memory of water’⁵ was probably allowed to claim too much on the basis of poor technique. But the paper in the same journal a year later by physicists at Brigham Young University claiming evidence of ‘cold nuclear fusion’⁶ dutifully accommodated referees’ comments and framed its conclusions with due caution: it was ‘ready’, even if wrong. In a more positive vein, the first report of C₆₀ in 1985⁷ lacked any compelling evidence for the molecule’s football structure, but both referees and editors (and indeed the authors) were right to take the gamble, and not just because it paid off with a Nobel prize.

For a paper justifies its publication if the debate and further work it provokes is productive. The C₆₀ paper prompted follow-up experiments that verified the molecular structure by NMR in 1990, and it also sparked ideas about and studies of new carbon allotropes, paving the way for the discovery of carbon nanotubes in 1991. Similarly, a report of enhanced molecular diffusion (beyond the usual Stokes-Einstein law) in certain kinds of bimolecular reaction⁸ seemed a sufficient challenge to conventional physicochemical lore that it could easily have fallen foul of over-zealous referees. It’s still disputed whether the result is correct,^9,10 but the idea that reaction energy might transform ordinary molecules into something resembling active matter is stimulating enough to be let loose. This is precisely the kind of discussion that should happen in the open, not in behind-the-scenes wrangling of peer review. Similarly, the notion that a simple small molecule (an amine) might do the job of an organometallic compound in catalysing the Suzuki reaction, while perhaps seemingly unlikely, is worth airing.

In short, the point needs to be better articulated both to the public and to the scientific community itself that the scientific literature is not, and should not be viewed as, a collection of correct claims blemished by the occasional error. Rather, it hosts a discussion – bounded by sensible rules of engagement that include appropriate demands for evidence, acknowledgement of existing knowledge, and caveats – of what we can say with some confidence about the world. For that, facts are crucial – but so is leaving space for ideas.