Comment

No comments

Following the recent release of guidelines detailing how universities will be assessed in the upcoming REF, Richard Catlow talks about this important exercise

The research assessment season is with us again, and no doubt all university departments are busy with their preliminary preparations for this pivotally important exercise. The documents summarising the Guidance on Submissions and Assessment Procedures, and draft panel criteria and working methods have now been issued by the UK Funding Councils for consultation; and sub-panel 8 - chemistry’s own expert review panel - has already met twice and has made a strong input into these documents.

In this comment piece, I will try to cover some general aspects of the REF 2014 exercise: the ways in which it both resembles and differs from the research assessment exercise (RAE) 2008 and its importance for the higher education sector in general and for chemistry in particular.

RAE versus REF

Similar to previous RAEs, REF 2014 is based on expert peer review and, as with RAE 2008, it will produce an overall quality profile for each submitted unit. The exercise will also resemble RAE 2008 in that the quality of the outputs will be the largest component (65 per cent) of the resulting profile. This crucially important and onerous part of the assessment will be based on panel members reading the outputs. Unlike RAE 2008, citation data will be available for sub-panel 8; but these data will not be a substitute for the panel members’ expert judgement. No other metric (such as journal impact factor) will be used; nor will the panel attempt to assess the contribution of individual authors in multi-author papers. The assessment will be based solely on the quality of the research. And, in this context, a note of warning: if reviews are submitted, the panel will only consider the new work in the review.

The biggest change in REF 2014, compared with RAE 2008, is the inclusion of impact - a change that I believe should be welcomed by the chemistry community as there is no doubt that the discipline has a major impact on the UK economy and, indeed, broader societal impact. As is clear from the Guidance on Submissions, impact is broadly defined and not narrowly confined to economic aspects, which are of course important; but, for example, impact may be in the development of public awareness of scientific issues. The impact component of the submission will have two components: a section that will describe the submitting unit’s strategy for developing impact and impact case studies, the number of which will depend on the number of staff submitted.

Case studies must demonstrate a link to high quality outputs from the submitting unit over the 20 year period 1993-2013; and it will be necessary to provide clear evidence of the reach and significance of the impact. But there will be no prescription as to the types of impact submitted. The panel will recognise that different departments will produce different types of impact; and each case study will be assessed on its own merits. Impact will contribute to 20 per cent of the overall quality profile and I am confident that the chemistry community will rise to the challenge of demonstrating the impact of our discipline.

Research environment will be the third component, contributing 15 per cent of the profile. The keywords here are ’vitality’ and ’sustainability’. As in RAE 2008, submitting units will need to show that they provide an exciting, successful and sustainable environment for contemporary chemistry. Metrics including research grant income and completed PhDs and PhD studentships will be significant, as will staffing and recruitment strategies. It will also be important to show how the submitting unit engages more broadly with other disciplines and departments.

Additional experts

The panel undertaking the assessment has a broad base of expertise covering the discipline, which will be further enhanced by the recruitment of additional expert assessors who will contribute to the assessment both of outputs and impact. Moreover, the work of sub-panel 8 will be firmly embedded in that of main panel B, which covers physical, mathematical and engineering sciences. The main panel will ensure that there is consistency between the working methods of the different sub-panels and will achieve a fair and consistent set of quality profiles across the different disciplines.

Let me conclude by emphasising the importance of the exercise. REF 2014 will determine the distribution of QR funding - currently running at approximately ?2 billion a year for several years after 2014. It will have a major impact on chemistry departments and chemistry as a discipline. The panel will ensure that the exercise fairly reflects the strengths of UK chemistry. Meanwhile, I wish all in UK chemistry departments the best in the next two years in producing excellent outputs and in demonstrating the impact of their science and the vitality of their research environment.

Richard Catlow is dean of the mathematical and physical sciences faculty at University College London, UK, and chair of the REF sub-panel 8