How to review a paper

Almost every chemist is asked to do peer review – but how do you make your feedback valid and valuable? We spoke to experts with a wealth of publishing and reviewing experience. Stuart Cantrill is chief editor at Nature Chemistry; Rob Eagling is editor-in-chief of Chem; Fiona Hutton is a publisher at Cambridge University Press; May Copsey is executive editor for Chemical Science; and Robert Baker is an assistant professor in inorganic and materials chemistry at Trinity College Dublin, Ireland and associate editor at RSC Advances.

You can say no

You’ve been asked to do a review, but how do you know whether to accept? ‘I think there are a number of things to consider when deciding whether to review a paper,’ says Eagling. ‘Obviously, is this in your area of expertise, or closely related? Can you provide an expert opinion on the article?’ Eagling also says checking for any conflicts of interest is important: whether you have worked for the principal author previously, they are a competitor or you know the authors personally. ‘An obvious reason not to accept is if you have been acknowledged or you have actually contributed to the article,’ Eagling adds.

Consider whether you’ll have the time to carry out the review. ‘As a referee, you should think of the time you’ve spent as an author,’ says Baker. ‘To have somebody read what you’ve written in five seconds isn’t very fair. It shouldn’t take more than two or three hours, but sometimes it’s really hard to put aside that time.’ Be honest with yourself: it’s much better to refuse than to accept and continually ask for an extension to the submission deadline. Copsey says the time taken to review a paper is highly variable; for some people it is a couple of hours, but many peer reviewers give up much longer. ‘Half a day isn’t unusual,’ she advises.

And all of our experts agree it’s absolutely fine to refuse a request to review. ‘But if you say no, say no straight away,’ adds Baker.

Know the formats

In general, your review will be anonymous. However, new versions of peer review have been introduced, such as double blind (where the names and identifying information of the authors are removed), and open peer review (where referees reviews are made public). If you’re unclear as to which type of review a journal uses, ask the editor in advance. ‘It’s a good idea to be familiar with the scope and structure of the journal,’ advises Copsey. ‘It could be objective peer review, so you are checking the science has been done in a rigorous way and you wouldn’t think about significance. At the Royal Society of Chemistry, all of our journal articles must have significance and novelty – not just good science but a broader significance to the community.’

Don’t buy the hype

Don’t accept a paper to review just because of the name attached to it. ‘If it’s a big name, sometimes you can tend to review it just to find out the new cutting-edge science,’ says Baker. ‘That sometimes changes your outlook on the paper.’ Don’t be blinded by names or institutions – just because the paper has been written by the leader in your field does not mean it will be good. Stay objective and approach every paper in the same way. However, Copsey says that sometimes it can be useful to know the author’s prior work. ‘In a practical sense, if you want to understand the significance of the paper it is useful to know the whole body of work – whether this is incremental, or a big step?’ The important thing is to be aware of the potential for bias.

Use your expertise

Once you’ve read the paper, how should you provide feedback? The best move is to read the referee guidelines, as each journal is different. ‘Some have more informal structure, some a more formal structure,’ says Hutton.

‘We ask them what’s great about this paper,’ explains Cantrill. ‘Is this paper technically sound? Why is it interesting? How could it be made better? Does it have any flaws? Does it cite the appropriate literature that you’d expect it to cite? Give us a critical summary of the actual work … have a look at the results, the methodology. Does the data support the conclusions that they make? Is the interpretation of the data reasonable?’

‘Go in with an open mind but be skeptical,’ adds Baker. ‘The idea is that the authors have to convince you that their data is correct.’

One of our editors’ pet peeves was receiving a recommendation to accept or reject a paper without any supporting explanation. As well as being a sign a referee has not read the paper in detail, it does little to provide helpful feedback. ‘A referee’s role is never about just accepting or rejecting, as this is the ultimate role of the editor. It’s about scientific validation and anonymous discussion to help move science forward,’ says Eagling.

Ignore the English

A common piece of advice from our editors is to leave English corrections for the editorial team. ‘I’m not contacting someone who has a PhD in chemistry to tell me how good the English is in the paper,’ says Cantrill. ‘What we really want is to use their technical expertise in chemistry to evaluate the technical aspects of the manuscript.’

Asking for things that are beyond the scope of the work is another common issue.

‘What we really want referees to do is evaluate what is there, don’t evaluate what isn’t there,’ adds Cantrill. If there’s a crucial experiment missing speak up – but don’t request multiple new experiments if the work can stand as it is.

‘Another thing that can be very frustrating is when a referee report comes back asking the author to insert lots of references authored by the referee themselves, or in a journal they are associated with,’ says Eagling. ‘These are not always so relevant. In some instances, this can be citation manipulation and an abuse of power.’