Consultation exercise highlights controversy over impact assessment element of the UK's proposed Research Excellence Framework
Universities, learned societies and educational establishments have raised concerns about the UK’s proposed new Research Excellence Framework (REF), with particular controversy surrounding the impact assessment element of the evaluation process.
Currently the REF, a new system for assessing research quality due to be implemented from 2013, proposes that excellence be rated based on research output (60 per cent), impact (25 per cent) and research environment (15 per cent).
In December 2009, the Higher Education Funding Council for England’s (Hefce) consultation on the REF closed after receiving hundreds of responses. Common themes concerned the high weighting of impact, the effect of the REF on academic behaviour and the timescales Hefce has set for introduction of the programme.
Impact is a new feature of the research assessment which Hefce admits is ’comparatively untested’, and the council therefore launched a pilot exercise to test and develop the proposals, due to conclude in summer 2010.
Most organisations responding to the REF consultation called for the weighting of impact to be reduced, citing a variety of reasons. The Russell Group of large research-intensive universities in the UK said that ’the introduction of impact at a level of 25 per cent carries substantial risks which are unlikely to be fully addressed by the impact pilot projects’ and suggested that 15 per cent should be the absolute maximum considered.
However Million+, a think tank representing new universities, said many of the concerns regarding impact are ’largely exaggerated’ or will be addressed through the pilot exercise. The organisation believes impact should be given at least a 15 per cent weighting, and research output boosted to 70 per cent.
As for business, the Association for University Research and Industry Links is comfortable with Hefce’s 25 per cent while the Confederation of British Industry suggests impact should be raised to 30 per cent.
Stealing someone else’s thunder?
Learned societies have raised further concerns about how impact will be attributed. Fundamental breakthroughs in science are often exploited by researchers in other disciplines where the eventual impact arises. The current REF proposal explicitly disallows the originating department to claim an impact of this nature.
The Institute of Physics disagrees ’in the strongest possible way’ with REF proposals that that it is not sufficient for research to have impact, but that researchers must have also been involved in the exploitation of that research themselves. The Russell Group agrees that ’it is not acceptable for a third-party institution to be able to take someone else’s research, carry out non-research activity to exploit this and then be able to claim the impact as part of the REF submission.’
The Royal Society was also anxious as to how the new programme could affect academic behaviour. ’Metrics can drive behaviour. If they are narrow, they can weaken those current strengths of the system which happen not to be measured,’ the society said. It called on Hefce to ’ensure that the impact indicators used in the pilot study are carefully considered and do not provide perverse incentives.’
The University and College Union (UCU), currently spearheading a campaign against the REF which has collected 18,000 signatures, took the consultation as an opportunity to make the reasons for its opposition clear. The UCU blames the REF’s predecessor, the Research Assessment Exercise, for the closure of higher education departments with strong research profiles. ’Unless there are major changes to the current proposals, we fear that the REF will be even more disastrous for UK higher education,’ the group said, labelling the impact agenda as direct interference from government in university research.
Hefce is expected to unveil the final details of the REF shortly after the pilot exercise is completed in the summer, and has asked higher education institutions to submit their portfolios of research in 2012 for assessment in 2013. Many groups and institutions used their consultation responses to call on Hefce to delay the REF by a year to allow time to fully digest the results of the impact pilot exercise.
Leila Sattary
No comments yet