Studies flag signs of gender bias in peer review

It is no secret that the scientific world is plagued by gender diversity issues. Female scientists are underrepresented at the higher levels of academia and among conference speakers. Research has found that not only are they paid less and saddled with more admin than their male counterparts, but are less likely to be hired in the first place and subject to unconscious biases that develop at an early age.

Now, two recent studies have suggested that gender bias might be engrained in another area of science: peer review. Initial results indicate that the editors and authors of scientific papers may be biased against female reviewers, although the researchers responsible say more work is needed.

We encourage other publishers, societies and organisations to do an internal audit of their inclusivity

Jory Lerback, University of Utah

It is difficult to carry out large-scale analyses on the diversity of peer reviewers, because most publishers keep reviewers’ identities anonymous. Often the only way to investigate is for individual publishers to audit their own data, which mostly produces small, niche studies that may not represent the scientific community as a whole.

Earlier this year, the American Geophysical Union (AGU), which publishes around 6000 papers every year across 20 journals in the fields of earth and space science, decided to investigate its own practices. As both a publisher and a professional body the AGU has access to self-reported age and gender information for most of its nearly 60,000 members from 2013–15, as well as tens of thousands of individuals who had registered for AGU accounts or engaged with the organisation in other ways since 2011.

They were able to merge this information with publishing data to get a dataset of ages and genders for nearly 25,000 distinct authors and 15,000 reviewers, as well as information on more than 97,000 reviewer suggestions by authors and 118,000 reviewer requests by editors across AGU journals. Their analysis suggested that female peer reviewers are underrepresented. While women made up 27% of published first authors and 28% of AGU members they made up just 20% of reviewers in 2012–15. The reasons for this could be related to gender bias during the process of recruiting reviewers. Just 15% of reviewers suggested by male authors were female. This figure was higher, at 21%, for female authors. The same trend could be seen for AGU editors’ recommendations: male editors recommended female reviewers 17% of the time, in contrast to 22% for female editors.

Gender gap

Jory Lerback, a former analyst at AGU who is now a research assistant at the University of Utah, US, says this gap between the proportion of female reviewers and female authors was surprising. ‘This gap is robust throughout age cohorts, and thus not a symptom of recommending older people to review, who are more dominated by men,’ she tells Chemistry World.

In another recent study, researchers from the Max Planck Institute for Dynamics and Self-Organization in Germany looked at data from the publisher Frontiers, which publishes the identities of each paper’s reviewers. They were able to access a huge dataset, analysing information from 142 Frontiers journals covering a range of science and humanities subjects that were published from 2007–2015. Their data included the names of more than 9000 editors, 43,000 peer reviewers and 126,000 authors for about 41,000 papers. Like the AGU, their findings suggest the peer review process is affected by gender bias.

They found women were underrepresented among both peer reviewers and authors, even taking into account their lower numbers in the pool of potential reviewers. They also showed that associate editors of both genders seemed to display a preference for their own gender when it came to appointing reviewers – male editors choose more male reviewers than would be expected based on simulations that assume the reviewer’s gender is random, and the same goes for female editors.

‘Homophily – a preference for something similar – seems to be a robust characteristic of human social networks,’ says Markus Helmer, who led the study. ‘Whether it also existed in peer review was, however, not clear from previous studies.’

‘As our study was based on a [large] number of reviews and based on reviews from a large variety of scientific disciplines we could clarify that ambiguous picture.’

Raising awareness

The teams behind both studies agree that more work needs to be carried out to build up a detailed picture of how bias affects peer review. And they agree it is difficult to know how to tackle the problem.

‘One could enforce quotas,’ suggests Helmer, ‘but simply increasing the number of women in science or peer review might not be enough to remove all gender bias, as same-gender preferences might persist.’

‘It is therefore important to make people aware of this kind of bias.’

He adds that while it is important for editors to be able to choose reviewers freely based on how appropriate they are, they could perhaps be given statistics of their reviewer selections to allow them to be mindful of potential biases.

Lerback suggests that publishers should follow the AGU’s lead and self-monitor. ‘We encourage other publishers, societies and organisations to do an internal audit of their inclusivity,’ she says. ‘Knowing a baseline of what their constituency looks like is important to measure progress towards an inclusive and productive environment, and being transparent about the results can help create a culture of accountability.’