Peer review under the microscope

One of the largest ever international surveys of authors and reviewers has reiterated concerns over integrity, demand and quality control in peer review and highlighted the system’s inability to effectively detect plagiarism.

The Peer Review Survey 2009, conducted by UK organisation Sense About Science, collated responses from over 4000 reviewers and authors worldwide and explored the motivations behind reviewing and their opinions of the system. 

Some researchers worry that the integrity of peer review is under threat from high demand and low quality control. Although the peer review community is growing rapidly with the global research community, over half of the participants in the survey were based in North America or Western Europe, suggesting their persisting influence over developing countries. 

The survey found that respondents were divided over potential incentives for taking part in peer review. Over half felt payment in kind (e.g. journal subscription, waiver of their own publishing costs) would make them more likely to review, 41 per cent said cash payment would be their preferred reward (though this drops to 2.5 per cent if the author has to cover the cost), while 39 per cent favoured published acknowledgement in return for their services. 

A further finding was that although reviewers see detecting plagiarism as a noble aim, it is not practical within the current peer review framework. Alice Tuff, development manager at Sense About Science, said: ’The vast majority of authors and reviewers think peer review should detect plagiarism (81 per cent) but only a minority (38 per cent) think it is capable. The academic time involved in detecting plagiarism through peer review would cause the system to grind to a halt.’ Publishing house Elsevier, which helped fund the research, is now developing electronic plagiarism tools in partnership with journal editors to address this matter.

’The survey has raised many issues for publishers,’ says Tuff. ’In general, reviewers feel that peer review is a good system but many aspects could be improved, for example providing better guidance and training.’   The majority of researchers who completed the survey felt that formal training for peer reviewers should be introduced.

While there are undeniable gripes about the peer review system in its current form, no viable alternatives have yet presented themselves that could drastically improve the way research papers are reviewed and published - a fact echoed by the reviewers who took part in the survey.

Many of the findings from the survey can also be applied to grant peer review. Training, for example, is one of the many areas for improvement identified by the Engineering and physical sciences research council’s Peer Review Survey published in February 2009, and it is likely to be addressed in its ongoing review of peer review.

The full findings of the Sense about Science Peer Review Survey are due to be published in November 2009 and will identify trends in specific scientific communities.

Leila Sattary