Grace Hindmarch, Alex R. Dopp, Karen Chan Osilla, Lisa S. Meredith, Jennifer K. Manuel, Kirsten Becker, Lina Tarhuni, Michael Schoenbaum, Miriam Komaromy, Andrea Cassells and Katherine E. Watkins
This blog post is based on the Evidence & Policy article, ‘Mis-implementation of evidence-based behavioural health practices in primary care: lessons from randomised trials in Federally Qualified Health Centers’, part of the Special Issue: ‘Learning from Failures in Knowledge Exchange’.
“This is disappointing, but I agree we’ve done the best we can.”
– CEO of a rural health care system
In October 2021, a rural healthcare system in the US discontinued implementation of a new program to improve access to quality care for patients with co-occurring opioid use disorder and mental health disorders. The program’s mission, fueled by passion for patients, was to help complex patients not fall through the cracks. After two years of immense effort, the system experienced ‘mis-implementation.’ Mis-implementation refers to unsuccessful efforts to implement treatments in real-world settings. Although it is a disappointing outcome, studying mis-implementation can provide insights to improve processes and make changes more successfully in the future.
Motivated by experiences like these, we recently published a study in Evidence & Policy examining mis-implementation of treatments that work for mental health and/or substance use disorders (together called ‘behavioural health disorders’) in Federally Qualified Health Centers (FQHCs). FQHCs are community health centers in the US that provide services regardless of patients’ ability to pay, and behavioural health is a key part of their mission. The United States is in a behavioural health crisis – with 1 in 5 adults experiencing diagnosable problems – which has driven researchers, clinicians and policymakers to look for and implement effective behavioural health treatments. However, evidence that treatments can work is not the same as if, how, where and when treatments are successfully (or unsuccessfully) implemented for different populations and health systems, especially low-resource settings like FQHCs. Exploring why implementation does not succeed is a relatively untapped and useful source of information.
Our study examined mis-implementation in three community-based research trials. These trials tested treatment delivery models for substance use disorders, PTSD, or co-occurring substance use and mental health problems. We compared and contrasted factors that influenced implementation failure across the three trials.
Barriers that increased the likelihood of mis-implementation fell into the following categories:
- Characteristics of the treatment (e.g. high intervention complexity)
- Characteristics of the people at the FQHC involved with implementing the treatment (e.g. low buy in from staff and/or providers)
- Characteristics of the FQHC (e.g., high staff turnover)
- The external context in which the FQHC operates (e.g. high levels of stigma in community toward behavioral health disorders)
- The approach to implementing the treatment (e.g. implementation only driven by leadership mandates)
Prescence of these barriers led to the following mis-implementation outcomes:
- The treatment not being a good fit for patients, providers, or organisations
- The treatment organisations not using the treatment to begin with
- The treatment not being widely or fully used
- The treatment not being sustained long-term
Across research trials, there was limited adoption and full, high-quality use of the treatments, which led to eventual discontinuation. Treatment complexity, low buy-in from overburdened providers, lack of alignment between providers and leadership within organisations, and COVID 19-related stressors contributed to mis-implementation. This happened earlier in research trials that experienced both patient- and provider-level barriers, and that were conducted during the COVID-19 pandemic.
To overcome mis-implementation, strategies supporting implementation need to be tailored to providers, patients and organisations. For example, providers may benefit from resources that build on their prior training, or from simplification of the treatment. Patients may benefit from ensuring patient-centered interventions and focused engagement efforts. Finally, organisation leadership may benefit from support for problem-solving with providers and adapting the treatment to their local context. These implementation support strategies should be flexible enough to address site-specific contextual factors and broader contextual factors that can be unpredictable, like COVID-19.
These findings reveal that studying mis-implementation provides insights into why treatments that work aren’t successfully implemented and/or sustained in routine practice. This research begins to uncover potential areas for large-scale improvements in health services research and practice. Knowledge exchange of both successes and failures between researchers, providers and policymakers opens the door for more sustainable interventions to improve behavioural health.
Image credit: Pexels.com
Grace Hindmarch is an MS Health Policy student at University of California Los Angeles Fielding School of Public Health, USA, and Adjunct Researcher at RAND Corporation, USA. email: ghindmarch@g.ucla.edu.
Alex R. Dopp is a Behavioral/Social Scientist at RAND Corporation, USA. Email: adopp@rand.org; X: @alex_dopp_phd
Karen Chan Osilla is Associate Professor at Stanford University School of Medicine, USA
Lisa S. Meredith is Senior Behavioral/Social Scientist at RAND Corporation, USA
Jennifer K. Manuel is Associate Professor at University of California San Francisco, USA, and Chief of Psychology, San Francisco VA Health Care System
Kirsten Becker is Survey Research Group Director at RAND Corporation, USA
Lina Tarhuni is a PharmD student at University of Washington, USA
Michael Schoenbaum is Senior Advisor for Mental Health Services, Epidemiology, and Economics at the National Institute of Mental Health, USA
Miriam Komaromy is Medical Director of Grayken Center for Addiction at Boston Medical Center, USA
Andrea Cassells is Vice President for Clinical Affairs at Clinical Directors Network, Inc., USA Katherine E. Watkins, Senior Physician Policy Researcher at RAND Corporation, USA
Read the original research in Evidence & Policy:
Dopp, A.R. Hindmarch, G. Chan Osilla, K. Meredith, L.S. Manuel, J.K. Becker, K. Tarhuni, L. Schoenbaum, M. Komaromy, M. Cassells, A. and Watkins, K.E. (2024). Mis-implementation of evidence-based behavioural health practices in primary care: lessons from randomised trials in Federally Qualified Health Centers. Evidence & Policy, DOI: 10.1332/17442648Y2023D000000016.
If you enjoyed this blog post, you may also be interested in reading:
Practical points of failure in police-university collaboration: reconceiving knowledge exchange
Obstacles to co-producing evaluation knowledge: power, control and voluntary sector dynamics
Learning from failures in knowledge exchange and turning them into successes
Disclaimer: The views and opinions expressed on this blog site are solely those of the original blog post authors and other contributors. These views and opinions do not necessarily represent those of the Policy Press and/or any/all contributors to this site.
Evidence & Policy Blog 