Louise Warwick-Booth, Ruth Cross and James Woodall
This blog post is based on the Evidence & Policy article, ‘Obstacles to co-producing evaluation knowledge: power, control and voluntary sector dynamics’, part of the Special Issue: ‘Learning from Failures in Knowledge Exchange’.
Co-production has been increasingly discussed as a positive and useful approach in health and social care research, based on principles such as partnership working, reciprocation, power sharing and the appreciation of all expertise. We have used co-production values to inform our evaluation work for many years, but in our Evidence & Policy article we reflect upon the challenges that such approaches bring, specifically in relation to sharing findings, known as knowledge exchange. Our article discusses evaluation work across three interventions that constitute perhaps the most challenging of our experiences in over a decade of such work. Conflict in evaluation work remains largely underreported, but we feel our experiences provide a useful contribution for readers.
How did we co-produce?
In each of the three evaluations we drew upon co-production principles, attempting to share power and responsibility with stakeholders by tailoring the methodological design and data collection tools towards their requirements, working transparently to discuss our research processes, analysis, findings and approach to reporting. In each evaluation our co-production approach differed as we adapted principles, worked with varying levels of stakeholder involvement and managed power dynamics associated with commissioners, delivery partners, service users and community members. For example, some stakeholders had more involvement in developing and contributing to final evaluation reports. Across all three evaluations we used co-production as a way to facilitate knowledge exchange.
What are the obstacles to co-producing evaluation knowledge?
Each evaluation resulted in the production of a detailed final report shared with funders, workers, service users and the general public. Academic publications were developed and shared, and community dissemination events were also held, but not uniformly across all three evaluations. Despite this knowledge exchange, several challenges arose:
- Vested interests: a commissioning model of funding, and local politics set the context for these evaluations. Vested interests influence knowledge exchange in such political contexts, resulting in tensions, aggressions, challenges, unfair treatment, damaged relationships, and knowledge exchange failures, with commissioners unwilling to share power over resources for research, access to data and publishing of results
- Emotions: voluntary sector work is led by highly invested, passionate stakeholders who care about what they do, as well as the services, and the people, that they support. Stakeholders can understandably be often emotionally charged rather than being neutral participants. Emotional labour is involved for all; stakeholders and researchers alike.
- Power: varying and shifting power dynamics also limited knowledge exchange, for example, via the non-disclosure of evidence. In our experiences in the three evaluations, a commitment to sharing power and decision making was not consistently seen, rather power was a tug of war, arising from challenges in communication linked to vested interests and consequent attempts to resolve perceived problems.
Recommendations for future work
Future research should gather detail on how changing research practices can enhance power-sharing and facilitate more transparent and honest knowledge exchange. Our experiences led us to conclude that:
- universities can work to implement stronger contracting processes, in which the independence of evaluators is made clear, to enable academics to report openly and fairly. Terms of reference specifically relating to expectations could also be developed and shared as part of the co-production process;
- knowledge exchange processes can be better managed through the creation of opportunities for the evaluated and evaluators to share concerns openly and honestly in safe spaces, with mediation available if required. In person communication should be prioritised because email messages can exacerbate frustrations when sent quickly, or if their content is misunderstood;
- emotional risk management also needs attention; for example, considerate reporting and paying attention to semantics may reduce upset, frustration and stress for those being evaluated;
- finally, researchers should have access to independent support enabling them to discuss and manage any emotional labour associated with their co-production and knowledge exchange practices.Debrief can be facilitated through the use of clinical supervision and/or peer support.
Image credit: Photo by Elisabeth Arnold on Unsplash
Louise Warwick-Booth, PhD, is a Reader in Health promotion in the Centre for Health Promotion Research, Leeds Beckett University. Louise’s research work focuses on evaluating voluntary and community sector interventions supporting vulnerable community members, especially women. Contact at l.warwick-booth@leedsbeckett.ac.uk.
Ruth Cross, PhD, is a Course Director in Health Promotion at Leeds Beckett University. Ruth’s current research in the Centre for Health Promotion Research focuses on evaluating interventions designed to support vulnerable young women and other marginalised populations. Contact at r.m.cross@leedsbeckett.ac.uk.
James Woodall, PhD, is a Professor in Health Promotion in the Centre for Health Promotion Research, Leeds Beckett University. James’ research interest is the health promoting prison and how values central to health promotion are applied to the context of imprisonment. James has published more broadly on health promotion matters. Contact at J.woodall@leedsbeckett.ac.uk.
Read the original research in Evidence & Policy:
Warwick-Booth, L. Cross, R. & Woodall, J. (2024). Obstacles to co-producing evaluation knowledge: power, control and voluntary sector dynamics. Evidence & Policy, DOI: 10.1332/17442648Y2023D000000008.
If you enjoyed this blog post, you may also be interested in reading:
Practical points of failure in police-university collaboration: reconceiving knowledge exchange
Obstacles to co-producing evaluation knowledge: power, control and voluntary sector dynamics
Learning from failures in knowledge exchange and turning them into successes
Disclaimer: The views and opinions expressed on this blog site are solely those of the original blog post authors and other contributors. These views and opinions do not necessarily represent those of the Policy Press and/or any/all contributors to this site.
Evidence & Policy Blog 