Jonas Videbæk Jørgensen
This blog post is based on the Evidence & Policy article, ‘Knowledge Utilisation Analysis: measuring the utilisation of knowledge sources in policy decisions.’
Using research-based knowledge to inform policy decisions constitutes a key ambition in most modern democracies. As such, enhancing the utilisation and impact of research has gained widespread attention among scholars and policymakers, with a range of initiatives to promote it. But how often is research-based knowledge used in policy decisions? And what kinds of knowledge have the strongest impact? Despite years of scholarship on the topic, measuring knowledge utilisation remains a significant challenge. In a new Evidence & Policy article, I discuss existing measures of knowledge utilisation and present a new approach called ‘Knowledge Utilisation Analysis’ (KUA).
Studying knowledge utilisation is tricky. Research-based knowledge can be used in many different ways and its impact can be difficult to disentangle from other sources of influence. Therefore, it is hard to tell whether, for example, research reports, policy analyses or evaluations actually influence policy decisions. Most existing studies rely on policymaker statements to measure knowledge utilisation. However, policymakers might struggle to recall their use of particular knowledge sources or overstate their use of research to appear competent. A growing number of scholars, therefore, use documents to measure knowledge utilisation. Some scholars apply computational methods to study how often policy documents cite or copy text from knowledge sources, while others apply content analysis to identify more indirect occurrences of utilisation. Contributing to this field, the article introduces a matching tool to link knowledge sources with policy documents.
Knowledge Utilisation Analysis is an analytical tool for researchers and policy workers to investigate the extent to which different kinds of knowledge sources are reflected in policy decisions. KUA involves a four-step procedure (collecting documents, developing a coding scheme, coding the documents, and measuring knowledge utilisation) and relies on two types of documents (policy documents and knowledge sources). The documents are coded thematically and matched based on three dimensions, which allows the analyst to measure knowledge utilisation.
- Whether the policy decision addresses the same topic and target group as the knowledge source
- Whether the policy decision follows or contradicts the conclusion(s) from the knowledge source
- The strength of the agreement or contradiction.
These dimensions are scored as displayed in the figure below. Together they provide a good indicator of knowledge utilisation based on the assumption that if a relevant and timely knowledge source has been available during a policy process and the final decision follows its conclusions, it is likely it has been used. Following the four analytical steps, KUA can be applied to study knowledge utilisation systematically across a large number of documents and cases.
The article further illustrates how KUA can be applied to examine patterns of knowledge utilisation in Danish primary education and active labour market policies from 2016 to 2021. By analysing 1,159 documents, the study finds higher levels of knowledge utilisation in active labour market policy than in primary education policy and a higher likeliness of using systematic reviews and randomised controlled trials (RCTs) to inform policy decisions compared to other types of research.
With this Evidence & Policy article, I hope to further empirical research on knowledge utilisation and discussions on how to measure it. Learning about the extent to which different kinds of knowledge sources are used in policy decisions is important for developing strategies to increase research impact and advance debates on evidence and policy.
* Figure 1 is not covered by a Creative Commons licence and is not to be reproduced without prior permission from the publisher, Bristol University Press.
Image credits: Photo by Towfiqu barbhuiya on Unsplash
Jonas Videbæk Jørgensen is PhD fellow at the Department of Social Sciences and Business, Roskilde University, Denmark. His research focuses on government ministries and the use of evidence in education and employment policy. Visit his institutional profile here. Twitter: @jonasvidebaek.
Read the original research in Evidence & Policy:
Jørgensen, J.V. (2023). Knowledge Utilisation Analysis: measuring the utilisation of knowledge sources in policy decisions. Evidence & Policy, DOI: 10.1332/174426421X16917585658729.
If you enjoyed this blog post, you may also be interested in reading:
Improving evidence use: a systematic scoping review of local models of knowledge mobilisation OPEN ACCESS
Academic knowledge brokering in local policy spaces: negotiating and implementing dynamic idea types
Disclaimer: The views and opinions expressed on this blog site are solely those of the original blog post authors and other contributors. These views and opinions do not necessarily represent those of the Policy Press and/or any/all contributors to this site.
Evidence & Policy Blog 