Trust in peer review

Exactly what the concept of trust in peer review means often depends on perspective. A researcher eager to publicise their groundbreaking paper must trust that their reviewer will be fair. And, upon passing the paper to a reviewer, an editor must trust that a review will be carried out with integrity.

Overall, the peer review process is a checkpoint through which researchers’ work passes to confirm that it answers meaningful questions and draws accurate conclusions from professionally conducted experiments. But how peer review is perceived and trusted by those outside of the academic and scientific community is also important. And it is possible that there are few outside scientific and academic disciplines that understand how the peer review process began building – and continues to support – the foundations of scholarly advancement nearly 300 years ago.

That foundation is a key factor that distinguishes a journal published paper from information available from non-scholarly and media platforms the world over, or in a paper shared by a predatory publisher. But does the public understand the difference between these types of information?

Wider trust issues

The director of community engagement at the United States’ National Information Standards Organization Alice Meadows considers wider trust in the process key. ‘Even if the scientists and scholars working in the particular field generally feel that the peer review system works, there’s a big gulf between that and the average person,’ Meadows says.

Meadows sees the retraction of papers dealing with Covid-19 as an example that is significant to the theme of trust. ‘The system kind of works and retractions themselves show that the system rights itself in the end but only because the community highlights something is wrong after a paper is published. The peer review system itself did not flag that up, in part because a dataset was not adequately reviewed. We need to help non-scientists understand how, even with a peer review process in place, there are still things that can go wrong.

‘How can you trust something if you don’t know how it works? People do not trust processes if what they see is not what they get.’

Research isn’t just done for researchers, the impact is going to be felt in all of our lives

Alice Meadows

‘Chemists need to very clear and transparent about every stage of the process – from research that is done in the lab through to publication,’ says Meadows, one of the organisers of Peer Review Week 2020 and the inspiration for this year’s theme of trust. ‘Both scientists and publishers have a part to play in showing how and what makes peer review reliable and trustworthy,’ she adds.

‘Personally I feel very passionate about this. I’ve had cancer twice. The first was breast cancer, for which there’s an overwhelming amount of information and much of it already well filtered.’ But the second cancer was a rare oral one, she says.

‘Much information about that was behind a paywall and there was a lot of misinformation. Had I not known how scholarly publishing worked, chat rooms might have been my sole source and I’d have feared the outcome. As it turned out, I was lucky and I’m fine.

‘But even when you look at peer reviews, there’s a huge difference between disciplines. We should not underestimate that research is not just done for researchers, it is being done for all of us. The impact is going to be felt in all of our lives,’ adds Meadows. 

Researcher’s reward

Analytical chemist Beatriz Jurado Sánchez won the highly coveted Ramon y Cajal post-doctoral work contract jointly sponsored by the Spanish government and European Union. Sánchez’s area of expertise is micromotors – defined as micro or nanoscale particles capable of autonomous motion in solution. She is currently conducting her research in the analytical chemistry, physical chemistry and chemical engineering department at the University of Alcalá in Spain.

Sánchez considers the peer review process vital in ensuring that the right scientists receive the financial backing they need to continue their research programmes. She has authored and co-authored scores of papers and was recognised by the RSC Advances journal as an Outstanding Reviewer in 2019 – and as such, she can see the peer review process from both an author’s and reviewer’s side.

She considers the failure to offer a detailed explanation for the recommendation made among the greatest sins a reviewer can commit when considering a paper. ‘Not giving a detailed explanation is bad. If you recommend to reject a paper, you need to put yourself in the position of the author because time and effort has been spent on it. You need to give advice on how to improve it.’

High impact journals tend to be more rigorous and assure fairness in the process

Beatriz Jurado Sánchez

As a government-sponsored researcher, Sánchez knows only too well how competitive the processes for winning grants can be. She considers fairness in the peer review process vital to ensure only those most deserving receive support for scientific endeavour. ‘Peer review bias can be unavoidable in some cases, but high impact journals tend to be more rigorous and assure fairness in the process.’

Sánchez stresses the importance of a robust peer review system for ensuring the best scientists fill society’s most important roles. ‘The articles are very important and it can depend on that whether you get promoted. Decision-makers look at the articles you have published.’

Thanks to her many years’ experience, Sánchez is able to complete a review in about two hours. Her field of expertise is niche. She has the knowledge and insight to spot a significant innovation or incremental advance as opposed to plagiarism, or plain error. If there is potential novelty in a new paper Sánchez knows how to assess its veracity.

Sánchez advises reviewers to stay within their field of expertise when reviewing, and be upfront if they are offered the chance to review a paper about a subject in which they are not suitably expert. ‘It also is not good to accept a review offer but be unable to actually do it.’ She says it is important that reviewers consider what it feels like to be awaiting a publishing decision, even being anonymous. 

Technology and peer review

Sandeep Verma is a professor in the department of chemistry at the Indian Institute Of Technology-Kanpur, India. He is also an associate editor for the Royal Society of Chemistry’s journal Chemical Communications, which, he says, holds a special place in his heart because it published his first paper as an independent researcher in 1999.

Verma has published nearly 200 papers including book chapters and reviews and, as an associate editor of the journal, he has handled as many as 130 papers in the past year. After submitting his first paper, he remembers the waiting process as if it was yesterday. ‘I was a nervous wreck, biting my fingernails but it was a pleasant experience.

Trust is related to the perception of quality that the investigator, or authors, have in a particular journal

Sandeep Verma

‘For me, the trust is related to the perception of quality that the investigator, or authors, have in a particular journal or a publishing house. It’s about journals they feel comfortable to share their unpublished data with, to get it reviewed with the purpose of publishing.

‘They hope that their data, their results, their deliverables will be handled with the highest integrity, transparency and fairness during the peer-review process, before it gets published. Such an honest review process is expected from the reviewers and manuscript handling editors. This is fairness for me.’

Verma sees a fitting circularity to his associate editorship of a journal he describes as ‘one of the RSC’s oldest and most respected’ and he’s happy to pay back into the system that helped fortify his own scientific career. However, he considers that the journal peer review system could be further incentivised for the greater good.

Quoting the high costs to researchers for colour image use in some journals (though not the RSC’s journals), or inclusion on the cover page as well as publication charges for their papers, Verma suggests a cryptocurrency solution. It could, he says, incentivise reviewers by being redeemable across all scientific publishers.

‘This cryptocurrency would build up as reviewers continue to contribute to the process, and they could use it for their scientific and academic needs.’

Verma also suggests artificial intelligence is used to streamline certain aspects of the system. ‘We should be bringing in custom platforms consisting of artificial intelligence and machine learning as a way to unclutter the process and make it even more fair and transparent.’

RSC publishing manager, quality and ethics Nicola Nugent keeps a close eye on factors that influence how peer review is seen both in and outside of the chemistry community.

‘While peer review is a human process and it is not perfect, it plays a really critical role in upholding the integrity and quality of the scientific literature. Because of this, peer review – in some form or another – has served as a real cornerstone of the scholarly publishing processes for centuries,’ Nugent says. 

‘It really naturally follows that we need a trustworthy peer review process to maintain trust in published research.’