Evaluations provide important information to improve public services, but only if they yield valid and reliable findings – so we believed for a long time. Evaluation communities have therefore established certain criteria that should define evaluation quality. Yet against prior studies on evaluation utilisation, we show that evaluation quality measured by the criteria is not necessarily associated with evaluation use, but rather linked to the perception of quality and impact of the evaluation. Evaluators should adjust their communications strategies accordingly.
Efforts to advance evidence-based policy quickly recognise the need for ‘research brokers’ to take on the critical role of linking the separate worlds of research and policy. Research brokers work in a range of organisations to transform, translate and package complex research findings into ideas and formats that can be used by policymakers, as well as facilitate meetings and establish relationships between researchers and policymakers. These research brokers are typically engaged in shaping the policy agenda, identifying promising solutions and influencing policy decisions. Yet, we know little about who plays that role, nor how they think about the evidence-policy connection.
Of particular interest are advocates working in foundations, think tanks, associations, lobbying firms and non-profit organisations pursuing specific policy objectives. These advocates are well-positioned to serve as research brokers since they are actively involved in most policy formulation and implementation processes and policymakers often use them as an important source of information. Yet advocates are also participants in the policy process who work to advance their own positions and preferences (and those of employers/clients). Research dissemination is just one of many tools advocates use to advance their policy goals.
After a period in which the onward march of evidence-informed decision-making appeared to be faltering in countries such as the US and UK, the acute uncertainties of the COVID-19 pandemic have triggered a fresh explosion of engagement with evidence and policy interactions – from diverse disciplinary, sectoral and institutional perspectives.
Today’s decision-makers need the evidence and insights of transdisciplinary research. Transdisciplinarity enriches our capacity to respond to complex problems by broadening perspectives on issues that are too complicated to be understood fully from one disciplinary angle.
COVID-19 presents an obvious example. The pandemic requires the insights and advice not only of medical and public health experts, but of policy scholars to inform government action; urban planners to model population movements and transport usage; epidemiologists to run big data models on potential virus spread; mental health experts on the implications of lockdowns and isolation; educationalists on the opportunities and pitfalls of home-schooling; behavioural psychologists on how to ensure restrictions will be accepted; the list goes on.
But how do we create diverse and effective research collaborations?
More controversially, my article argues that this would be a better way to reform research funding than lotteries, which others’ research indicates would be better than current norms. Norms are changing though – one of the things I’ve learnt more about since publishing this article is how the Health Research Council of New Zealand has been using a lottery to allocate some grants. They have been doing that for long enough to publish a peer-reviewed paper about it.
Behaviour change policies, known as nudges, have been used by governments across the world to get people to behave in pro-social ways, such as making healthier lifestyle choices or reducing their environmental footprints. Nudges use behavioural insights to steer people into doing the right thing, while also giving them the choice. Critics argue that traditional nudge policies are top-down, manipulative and un-transparent. Nudge policies seem to expect the worse in people, and are easy to caricature as a technocratic approaches to policy design.
However, a new kind of nudge – ‘nudge plus’ – has started to spring up. Nudge plus tackles the risks of paternalism in traditional approaches through the participation of those being nudged. If nudges are going to be even more ‘bottom-up’, how can such behavioural public policies be developed?
The COVID-19 pandemic has sparked a major debate about the role of experts in policymaking and the capacity of politicians to ‘follow the science’. The trend we have seen, where expert advisers have increasingly become the public face of the pandemic, raises questions about the evolving role of experts in other public policy challenges – including challenges where the scientific base is arguably far clearer about effective policy responses. If politicians are willing to ‘follow the science’ with such diligence in relation to COVID-19, why does the same principle not apply to other public health challenges?
Many of us place our hopes on innovative breakthroughs and groundbreaking discoveries, believing them to be our best bet to achieve a better world. And indeed, science has produced extraordinary breakthroughs. Vaccines radically reduced the risk of death from communicable diseases. Nitrogen-based fertilisers vastly increased the production of food. Computers completely transformed how modern humans learn, work and communicate. Surely, it would seem that investing in scientific breakthroughs is the key to progress. In this spirit, social scientists develop ‘evidence-based’ practices and policies and create hierarchies of evidence to determine ‘what works’. Many believe that if only science can produce enough evidence, discoveries will follow that can change the world – if only we can effectively compel others to accept them.
In a recent article published in Evidence & Policy, we explored the use of Aristotle’s three knowledge types: empirical knowledge, technical knowledge and practical wisdom, in the everyday work and decision-making of frontline public service professionals.
Our qualitative case study of a Scottish local authority revealed the importance of integrating and recognising the different types of knowledge that are needed to respond to complex policy problems, often referred to as ‘wicked’ problems. Understanding the craft of integrating different types of knowledge, and valuing what can be learnt from frontline workers, is key in achieving impactful evidence-informed policy.
In the current context of a rapidly changing policy landscape resulting from COVID-19, making policy decisions informed by the most appropriate types of evidence is crucial. In this blog, we discuss how Aristotle’s knowledge types can help us understand the types of evidence that should be considered in this ever changing landscape.
Matthew Johnson, Elliott Johnson, Laura Webber and Kate Pickett
The COVID-19 pandemic has increased interest in Universal Basic Income (UBI) as a means of addressing a range of socio-economic insecurities. While previous trials of cash transfer schemes have often focused on low-level transfers inadequate to satisfy the needs for which the policy was originally developed, emerging pilots are moving toward a position of increasing generosity. Our multidisciplinary project, Examining the Health Case for UBI, has brought together colleagues in behavioural science, public health, epidemiology and economics to establish pathways to health impact outlined in Figure 1 below. Our work suggests the potential for significant health impact and attendant economic benefit via reduced healthcare costs and increased economic activity. The model suggests that elements of impact may only be felt if payment is set at a more generous level. This could create greater return on investment and, ironically, a more cost-effective system.