David Gough, Chris Maidment and Jonathan Sharples
This blog post is based on the Evidence & Policy article, ‘Enabling knowledge brokerage intermediaries to be evidence-informed.’
Research evidence can be useful (alongside lots of other information) in informing policy, practice and personal decision making. But does this always happen? It tends to be assumed that if research is available and relevant then it will be used in an effective self-correcting ‘evidence ecosystem’, but in many cases the ‘evidence ecosystem’ may be dysfunctional or not functioning at all. Potential users may not demand relevant evidence, not be aware of the existence of relevant research, or may misunderstand it use and relevance.
Knowledge brokerage intermediary (KBIs) agencies (such as knowledge clearinghouses and What Works Centre) aim to improve this by enabling the engagement between research use and research production. We believe that KBIs are essential innovations for improving research use. In this blog, we suggest four ways that they might be further developed by having a more overt focus on the extent that they themselves are evidence informed in their work, as we explore in our Evidence & Policy article.
Evidence for planning a KBI
How does a KBI decide what would be most useful to do to enable user engagement with research? They could, for example:
- work with decision makers and identify their priorities and seek or commission research to fulfill these needs
- enable co-creation of research between users and researchers
- enable the implementation of evidence informed decisions
- focus on providing access to research findings or recommendations for action based on such research.
They could try to do all of these things. And how should their work fit in with other agencies working on the topic? In education, for example, there are many concerned with research, policy and/or professional practice, including schools’ inspectorates, such as OFSTED in England. What is a KBI’s relation to all those other actors? In sum, how does a KBI position itself in its role in relation to evidence and to other players in the existing evidence ecosystem (as per the figure below)? Do many KBIs default to making research evidence available (the ‘push’ of evidence) rather than also enabling the demand ‘pull’ for relevant research?
KBIs usually tell you what they do, the content of their work, but to what extent do they undertake a needs analysis of the functioning of the pre-existing evidence ecosystem to inform how they could contribute to make it more effective?
Evidence to inform a KBI’s methods
In whatever way a KBI decides to intervene in their evidence ecosystem, there are many methods that they could use to achieve their detailed aims. What are these specific aims? Who will be the users and ultimate beneficiaries of their work? What are their methods to achieve these aims? Is there evidence from ‘research on research use’ that such methods will achieve the outcomes they desire? KBIs may assume, for example, that making evidence available to users will increase its uptake. But studies of research use have shown that access to research evidence alone does not increase its use. Access is a necessary, not sufficient, condition. If the KBI is not dealing with the capacity, opportunity and motivation to make use of evidence, then who is?
Evidence on the wider systems in which KBIs operate
A key consideration for KBIs as intermediary organisations is how they sit and work within wider systems and contexts. This includes not just the systems of evidence production, mobilisation and use they are part of, but also the wider political and societal systems in which the benefits of evidence use will be realised (e.g. policy, accountability). These wider systems are not always structured in a way that is receptive to research use, therefore KBIs need to be adept at identifying levers of influence and nimble in capitalising on opportunities as they arise. Doing so relies on being able to understand and influence the wider systems and contexts in which they operate. Is there a receptive infrastructure for the work of KBIs? What is the relationship between a KBI and that infrastructure? What strategic choices are KBIs making to engage with the wider systems outside of research?
Evidence standards for making evidence claims
If research evidence is to be used to inform decision making it needs to be relevant and trustworthy. KBIs have evidence standards to ensure that the evidence claims that they make are justifiable. But are these evidence standards sufficiently stringent? Are they based on a systematic review of all of the relevant evidence base? Some KBIs make evidence claims on the basis of one or two good quality studies, yet there may be many other good quality studies coming to other conclusions. Even if KBIs use stringent evidence standards are they consistent in their use across different products and areas of their work?
Evidence that a KBI is effective
Many KBIs are so busy fulfilling their obligations to enable engagement with research that they simply do not have the resources to monitor and evaluate their own work. The methods of self-appraisal may be very limited such as the number of ‘hits’ on their websites. But if KBIs are advocating the use of research to inform decision making, then should they not apply the same logic to the evaluation and development of their own work? And could not such evaluations then contribute to the evidence base on ‘research on research use’ – on how KBIs might best develop their work?
KBIs as part of joined up policies on research evidence
Governments invest in policies and infrastructures for both policy making and for research production. KBIs are part of moves to do the same for the ‘missing link’ between decision makers such as policy makers and research. But this is all still a bit ‘ad hoc’. Could we develop more overarching and consistent policies, structures and investments for linking policy and practice decision making with those for producing research? This is not an argument for a monolith structure that does not tolerate plurality and flexibility. It is an argument for more thought in how we link up the use and production of research. Read more in our Evidence & Policy article.
Image credit: Authors’ own.
David Gough, EPPI Centre, University College London. Twitter: @ProfDavidGough
Chris Maidment, Energy Institute, University College London
Jonathan Sharples, EEF and EPPI Centre, University College London
Read the original research in Evidence & Policy:
Gough, G. Maidment, C. and Sharples, J. (2021). Enabling knowledge brokerage intermediaries to be evidence-informed. Evidence & Policy, DOI: 10.1332/174426421X16353477842207. OPEN ACCESS
If you enjoyed this blog post, you may also be interested in reading:
Cutting through the noise during crisis by enhancing the relevance of research to policymakers
The policy impact of entrepreneurship research: challenging received wisdom
Disclaimer: The views and opinions expressed on this blog site are solely those of the original blog post authors and other contributors. These views and opinions do not necessarily represent those of the Policy Press and/or any/all contributors to this site.
Evidence & Policy Blog 