The tyranny of peer review

Peer review, originally introduced in the 17th century as a means of vetting contributions to the Royal Society, is widely used to assess research. It serves science well and is seen as, while perhaps not perfect, far better than anything else we have been able to devise. So why do I refer to the tyranny of peer review? It may be unduly risk-averse, fostering incremental research at the expense of higher risk proposals that offer the prospect of a transformational change: a ’dead hand’ for high risk, high potential returns. Some examples from journal publishing may help illustrate the difficulty. 

I have heard it said that Tim Berners-Lee’s original paper on the World Wide Web was rejected as not sufficiently significant! Rationalise that if you can with the impact this advance has had on society. 

Maiman’s paper on the first experimental demonstration of a laser was ’office’ rejected by the editors of Physical Review Letters  so instead appeared in Nature.  

The idea of negative refraction was a very contentious topic just a few years ago. It was a radical idea, one which many found challenging to the point of simple refutation although all doubts have been convincingly laid to rest now. 

There have been suggestions that sonoluminescence from bubbles is the result of nuclear fusion, with counter indications too. It is interesting to speculate how a proposal for fusion research in this area might fare in peer review.  

It was a truly breakthrough paper by Bednorz and Muller that reported high temperature superconductors. But how would a research application proposing such a dramatic advance have fared in peer review? The authors received the Nobel prize for physics for this work less than two years after the paper was published. A phrase in the Nobel citation is pertinent here: ’. they had the audacity to concentrate on new paths in their research.’ 

So special, perhaps different, attention might be needed for certain types of research proposal that one might designate as either: high risk; high uncertainty; swimming against the tide; or flying in the face of conventional wisdom. 

There is room, surely, for research activities characterised by something along the lines: ’There is about ten per cent (say) chance of success, but if so the implications are very great indeed.’ 

It would be nice to have confidence that the established system would nurture the kind of proposals to which I am alluding, but anecdotal evidence is not encouraging. Proposals that don’t fit within established disciplines are said to give particular difficulty for the peer review establishment. After all, excellence in peer review requires excellent peer reviewers. Just who are the peers appropriate to judge anti-establishment, or rather extra-establishment, research? 

Considering this problem a few years ago the Engineering and physical sciences research council, UK, issued an ’adventurous multidisciplinary research’ call. A special panel undertook double blind review for outlines, followed by subsequent consideration of selected full proposals. Some very different projects were funded and it will be interesting to see in due course what results. 

But all this should not be interpreted as an attack on peer review per se. Rather it is a caution: all of us involved in the review process must avoid being unduly conservative or risk-averse if we are to secure the required balanced research investment portfolio, including an appropriate proportion of high-risk, high-return activities.  

Sir John O’Reilly is vice-chancellor of Cranfield University. This article is based in part on a presentation to the European Science Foundation on peer review in 2006, when the author was chief executive of the Engineering and physical sciences research council.