Helen Allbutt and Stewart Irvine
Does research add value? How can we tell? With no mechanism to quality rate research outside of the university sector, research can be overlooked, or worse discontinued, particularly when organisations face ever-increasing pressures. In this blog, we discuss how we sought to protect our research investment by providing an evidence trail of how project findings contributed to strategic priorities. This blog covers the key points of what we did and what we found: for a fuller version, see our Evidence & Policy article, ‘Research assessment in a National Health Service organisation: a process for learning and accountability’.
Finding a suitable model for research assessment
We looked for ways to demonstrate that research was of benefit to our organisation. One solution was to undertake a research impact assessment. We found an impact framework that was suited to our context and, although not an exact fit, we adapted it and used the most relevant impact categories for our purpose. We then chose 20 projects to study, selecting those that were developed enough to have published outcomes but were not too old that information about them was not available. We asked researchers or accountable officers to provide evidence for their project outputs and impacts in a simple template under different category headings. These self-assessed claims were quality assured.
What we found
The impact categories helped researchers think through different forms of project outcomes. Over four months, we grouped the responses under the impact categories and mapped them to our strategic objectives. Our 20 projects yielded a far wider range of outputs, outcomes and impacts than we had anticipated.
Projects with the most outcomes and impacts were those where:
- engagement with end users had been built into the study design
- dissemination involved multiple audiences and different forms of messaging
- leaders endorsed results and had publicly indicated their support.
This research impact assessment enabled us to think about project work in a different way. Instead of only logging the number of publications we produced each year, we were able to demonstrate how our research was influencing and changing our policies and practice. Further analysis revealed that research teams needed to better plan for the impact they desired. Projects might generate many outcomes but fail to achieve the key purpose of the study. More targeted benefits might arise if researchers committed to pursue specific outcomes from the outset.
This exercise provided evidence to our board of where there was alignment between the work of researchers and corporate goals. It also allowed us to specifically link project findings to strategic objectives. In this way, we were able to provide an evidence trail of project contributions to core business. Whilst this does not guarantee continued funding of research, it helps to account for and justify its resource. In a culture of increasing organisational performance and governance, research assessment can be used as a strategy for both learning and accountability.
Helen Allbutt is a Principal Lead at NHS Education for Scotland. A nurse by background, she completed her MSc in Public Health and PhD in health research at the University of Edinburgh.
Stewart Irvine is acting Chief Executive Officer of NHS Education for Scotland. He is an Honorary Professor at the University of Edinburgh, and was awarded Honorary Fellowships by the Royal Colleges of Physicians and of Surgeons in Edinburgh. Stewart is a medical graduate from the University of Edinburgh where he also completed his MD.
You can read the original research in Evidence & Policy:
Allbutt, H. and Irvine, S. (2019). Research assessment in a National Health Service organisation: a process for learning and accountability. Evidence & Policy, DOI: 10.1332/174426419X15580814604785.
Image credit: Sergiu Bacioiu, Wikimedia Commons.
If you enjoyed this blog post, you may also be interested to read: