Initiative to curb bias hasn’t eliminated racial disparities in NIH funding

Researchers have analysed National Institutes for Health (NIH) grant proposals to try to understand why African American researchers are less likely to succeed in their funding applications than their white counterparts. The investigation found that racial disparities are present at the earliest stages of the peer review process, but not in how reviewers combine preliminary impact scores.

Research has previously revealed racial disparities in the success rates of applications for R01 grants – the NIH’s core funding mechanism for health-related research. In particular, black researchers were the least likely to secure such grants for their research.

In 2009, the NIH overhauled its evaluation protocols to try and improve transparency and fairness, introducing a new process known as ‘enhanced peer review’. During the initial stage of review, applications are scored against five different criteria – significance, investigator(s), innovation, approach and environment. Reviewers then weigh the criteria scores as they see fit to derive a ‘preliminary overall impact score’ – a metric that determines whether applications will be put forward for panel discussion in the next phase of the review process.

‘One of the things that psychological research suggests is that in evaluating applicants, what we ought to do is focus our attention on specific criteria to avoid bias and focus attention on merit-related factors,’ says the University of Washington’s Carole Lee. ‘And so, one thing you might expect … is that when NIH shifted its peer review practices so that reviewers were asked to evaluate applications along specific criteria, you might see a lower funding gap between black and white researchers.’

With support from the NIH, a group led by Lee and her Washington colleague Elena Erosheva set out to see whether data gathered from the first stage of the new review process would shed light on where racial disparities arise. In particular, the group wanted to see whether reviewers combined criteria scores in ways that would favour white researchers when arriving at their preliminary overall impact scores – a phenomenon known as commensuration bias.

‘Previous research has found that the single largest source for funding disparities between black and white researchers lies in this decision point of which applications move on to the panel discussion,’ explains Lee. ‘So, if we can take a look at those scores that informed that decision that can give us some insight into a key part of the process.’

The group found no meaningful differences in the ways that criteria scores were aggregated into the preliminary overall impact scores. However, their analysis did reveal a stark difference between the criteria scores awarded to black and white researchers.

‘When reviewers score these applications on the various criteria, we see differences on those criteria scoring,’ says Erosheva. ‘And they are quite uniform in the sense that black applicants score worse on all of those criteria.’ This translates into a statistically significant difference in the average preliminary overall impact scores awarded to black and white researchers.

While there are many stages of review between the preliminary overall impact scores and final award decisions, the study also showed that racial disparities persist in overall award rates. ‘As the researchers before us, we find that funding disparities still exist in this data from 2014 to 2016,’ says Erosheva. ‘We see that the overall award rate for black applicants is 55%, that of the white applicants – that’s a pretty large gap.’ Even when factors like career stage and scientific field are taken into account, black researchers still appear to be only 75% as likely to succeed in their funding applications.

‘I have always believed that the next research step in understanding the black/white funding gap would be to examine whether information from the enhanced peer review process could be used to understand what is happening, and I’m pleased that the Erosheva et al study has done so,’ comments University of Kansas economist Donna Ginther, whose research first exposed racial disparities in the success rates of NIH R01 grant proposals.

However, Ginther points out that the results are not directly comparable to her earlier studies – something the authors themselves acknowledge – noting that the study doesn’t separate type 1 proposals for new projects and type 2 proposals, which depend on the track record from previously funded research. ‘These two types of grants are very different, as are their success rates – success rates in type 2 awards are higher,’ says Ginther. ‘If researchers are going to make progress on understanding the causes of the black/white funding gap, we need to strive to make apples-to-apples comparisons.’

Priority boarding

Within the criterion scoring, the largest disparity between black and white applicants was found in the research approach category. Ginther suggests that this is a particularly important line of future investigation.

Kristin Kramer from the NIH’s centre for scientific review also says that more data is needed to fully understand the origins of the funding disparities. She points out that other factors need to be taken into account to gain a clearer picture of the review process – for example, the award rates at the particular funding institute that applications are assigned to, and whether applications are being submitted for the first time. ‘NIH is committed to understanding the source of these differences so that we can then make data-driven changes to policy or procedure to address disparities,’ she adds.

While the latest analysis doesn’t explain the racial discrepancies in preliminary criteria scoring, the authors suggest there are probably many factors involved. These include reviewers’ implicit racial preferences and the ‘cumulative effect of disparities experienced over a PI’s academic career including differences in mentorship and social networks’. Lee also notes previous research that showed that black scientists were statistically more likely to pursue research on topics considered of lower priority by reviewers.

Vanderbilt University’s Ebony McGee, who investigates racial marginalisation in science, technology, engineering and mathematics, says that scientific funding bodies need to do more to eliminate these sorts of disparities, and to broaden participation from underrepresented communities.

‘Black people are going to pursue scholarship that helps their communities, their societies in the world at large … this is rarely appreciated by the scientific community,’ says McGee. She adds that organisations like the NIH should ensure researchers of colour are included in leadership positions on diversity and inclusion initiatives, and that more effort should be made to increase minority representation on review panels. ‘Scholars of colour, and particularly black Stem-ers need the benefit of the doubt like their white colleagues seem to automatically get.’