Researchers spent an estimated 130 million hours peer-reviewing papers in 2020

In 2020, researchers globally spent an estimated 130 million hours – equivalent to 15,000 years – peer-reviewing manuscripts for scholarly journals, according to a study published last month. 

Although academics are usually not paid for conducting peer review, the study estimates the monetary value of peer review conducted last year in the US alone to be $1.5 billion (£1.1 billion) using the average salary of academics within the country. The cost of the time of referees based in China is more than $600 million and in the UK it is nearly $400 million.

The numbers are staggering

Gunther Eysenbach, University of Victoria in British Columbia

Study co-author Balazs Aczel, a meta-scientist at Eötvös Loránd University in Budapest, Hungary, cautions that his study makes many assumptions such as the number of hours referees spend evaluating each paper. ‘It’s pretty challenging because the publication system is not so transparent,’ he says.  

For instance, Aczel’s study assumes that rejected papers had been refereed by two reviewers while accepted ones had been reviewed three times even though the process usually varies by publication. Aczel justifies this by noting that previous studies have also used a similar system to make such predictions. His study also assumes that referees spend on average six hours reviewing each paper – an estimate that comes from a large-scale 2009 survey.

Aczel and his colleagues estimated the number of rejected papers from the total number of papers published and the typical rejection rates of the journals. It was difficult to work out how many papers were desk-rejected and therefore didn’t go through any rounds of peer review, he notes. 

Aczel says his numbers may be an underestimate as some papers go through multiple rounds of peer review before revised versions are accepted. What’s more, according to the study, an estimated 28 million hours of peer review time – equating to just under $300 million – can be saved annually in the US alone if journals adopt portable peer review, where referee reports from one journal are passed to another if a paper is ultimately rejected. 

For Aczel, it’s important that journals and publishers make efforts to recognise the efforts of reviewers and reach out to a diverse set of referees to help spread the workload. 

‘The numbers are staggering,’ concludes Gunther Eysenbach, a health policy researcher at the University of Victoria in British Columbia, Canada, and co-editor-in-chief of the Journal of Medical Internet Research (JMIR). At JMIR and its sister journals researchers can earn ‘karma points’ for reviewing papers, as well as authoring and editing manuscripts, which they can later deduct from the article processing charges for their own future papers. For each review, referees can typically earn up to 100 karma credits, which is equivalent to $100, Eysenbach says. 

A 2018 report from Publons – which enables researchers to claim credit for peer review – found that researchers working in western countries carry out a disproportionate amount of peer review despite the fact that academics in developing nations are more likely to accept requests and carry out peer review more quickly. 

Andrew Preston, who co-founded Publons but is no longer affiliated with the firm that was bought by Clarivate Analytics in 2017, agrees that the new study’s numbers are probably an underestimate. But he argues that this isn’t necessarily unpaid labour as reviewing manuscripts is considered to be part of an academic’s job and a way to give back to the research community. 

Despite peer review being a significant time sink, Aczel doesn’t think publishers should pay researchers for conducting peer review. Preston agrees: ‘There is a risk that direct payment for peer review could lead to undesirable incentives where people are just churning out reviews to get money.’