US to measure research impact

A new federal project promises to monitor the impact of the $8.9 billion (?6.1 billion) contribution to US scientific research included in the American Recovery and Reinvestment Act of 2009 stimulus package. 

The initiative aims to provide solid evidence on the returns to the US from its recent investment in science research, enabling the federal government to monitor the investment’s impact on employment, knowledge generation and health. Following a successful pilot involving seven institutions, the voluntary programme is being rolled out across the US and 60 universities have already expressed interest in taking part.

The venture - Science and technology for America’s reinvestment: Measuring the effect of research on innovation, competitiveness and science (Star Metrics) -   is a multi-agency venture led by the National Science Foundation (NSF), the National Institutes of Health and the White House Office of Science and Technology Policy.

Julia Lane, director of the Science of Science and Innovation Policy programme at the NSF, led the project and believes the key to successful reporting in science is automation. ’Manual reporting is inefficient, produces bad quality data, and puts a high level of burden on academics. We have automated reporting by linking existing databases on publications, citations and patents.’  

The Star system takes a similar approach to impact appraisal to the original Research Excellence Framework (REF) proposal in the UK which was driven by bibliometrics but later rejected in favour of a peer review approach. Unlike the REF the US initiative is voluntary, and will focus on both economic and social impacts. The results of Star assessment will not be used to measure the quality of research or drive funding for individual institutions. 

It is clear to Clare Grey, a chemist at the University of Cambridge, UK, who worked in the US for many years, that researchers have a responsibility to explain to the public the benefits to the economy and society of the research they perform. ’However, measuring research impact is difficult to do in many cases without metrics. We need to justify the continued funding of research and cast it in the language used by decision makers on Capitol Hill (US) or Whitehall (UK).’

However, the question of how to measure social impact remains unresolved by Star Metrics in the US and the REF in the UK. ’We cannot fully describe impact yet but the measures we are developing are in collaboration with the academic community,’ says Lane. 

Together, NSF and NIH have committed $1 million to carry out and develop the Star Metrics initiative in the first year.

Leila Sattary