Sarah Morton and Ailsa Cook
We gain fascinating insights working alongside organisations across sectors in public services, that want to use data and evidence well to understand and track their impact. We specialise in working with organisations where it is hard to simply measure the difference they make, and where the main focus for change is relationships: work that educates, empowers, inspires, supports, encourages or influences people. What we have observed over the last 5 years, is that every organisation is influenced by their data culture, but it is rarely talked about. It is something we highlight in our new book: How do you know you are making a difference, from Policy Press.
Through our company Matter of Focus, we support organisations to understand the context for their work, set out their theories of change, and use this as a lens to collect and analyse data that can help them understand their change processes and evidence the difference they make. This means we host workshops and meetings where people really get to grips with different elements of their work, and we see what is inside the pandora’s box when organisations start to review the data they hold about their own work.
We describe data cultures as ‘invisible systems of beliefs and practices that underpin what is valued and what is important in any organisation’ (p36, Morton and Cook, 2022). We have observed that these play out in different ways for different kinds of organisations. We conceptualise data culture along spectrum from formal to informal.
At the formal end of this data culture spectrum are often organisations linked to health. These cultures usually have an assumption that a traditional hierarchy of evidence is important, with randomised control trials seen as the best form of evidence. Of course, this is entirely appropriate for interventions focussed on the physical world such as drugs or devices. This culture has its place but permeates programmes and approaches much less well suited to reductionism. People working in services that are not orientated around empirically measurable treatment outcomes, like public or community health, often find themselves struggling to make the case for the use of more appropriate means of assessing effectiveness.
These more formal data cultures often have empirically measured outcomes, with well-developed data systems. There is little or no control locally over what data is important or will be collected, and a punitive culture when data is ‘wrong’. Data is seen as neutral and telling the ‘truth’.
At the informal end of the spectrum are data cultures that value lived experience of people above all else. Participation is deeply embedded into organisation processes and structures. They often have creative and permissive approaches to collecting data – allowing for variation to suit specific people and settings. There are no formal systems for data processing, and more formal approaches to collecting and storing data may be seen as cutting across the person-centric ethos of the organisation. There is a lot of local control over what kinds of data to collect and use. These kinds of organisations can be empowering and innovative in their data practices. They can also undervalue more formal ways of understanding the difference they make, and don’t have ways of balancing different views about what is important.
Most organisations exist somewhere between these two extremes, but ideally, to understand complex social change, everyone needs to be somewhere in the middle: collecting a mix of formal and informal data, feedback and evidence, and bringing this together to reflect on whether change is happening as anticipated, what is going well and less well, where to focus attention and where more evidence is needed.
By calling out the core assumptions and values about these data cultures, organisations can start to develop a more open approach to the data they need to understand if they are making a difference and whether they are having the impact they hoped they would.
Identifying data culture can help highlight people’s personal, as well as an organisation’s preferences and assumptions, giving opportunities for reflection, adaptation and improvement.
Questions for diagnosing data culture:
- Is there a dominant paradigm in your organisation that values certain kinds of data over others?
- Are there data collection systems in place? What kinds of data do they focus on?
- Are there any data requirements from the core, funders or government? What kind of data do these require?
- What resources are allocated to collecting and using data?
- Whose job is it to collect, store and analyse data? What does this tell you about data for accountability or learning?
Dr Sarah Morton, Director, Matter of Focus and Honorary Fellow, University of Edinburgh
Sarah is internationally recognised for her work developing innovative approaches to ensure that decision-makers have access to the best evidence for taking organisations, policies and practices forward. She led knowledge exchange work at the University of Edinburgh from 2001-2017, before setting up with Ailsa Cook their company Matter of Focus, that offers tech and consultancy to help organisations use data and evidence better in their everyday work. She has worked in policy and practice influencing roles, in the third and public sector, and supported impact work across many organisations including NHS, large and small charities, UNICEF, Scottish Government, universities and public bodies.
She has supported researchers in the UK, Canada and Australia with research impact work, and developed a research impact assessment method that has been widely used. She has carried out 7 independent impact studies, including 14 case studies, of research centres and initiatives across 8 countries around the world.
Twitter: @sasmort / @matter-of-focus
Dr Ailsa Cook, Director, Matter of Focus
Ailsa is a leader in the field of outcomes. Through her research and work with policy and practice, Ailsa has made a significant contribution to shifting the focus of public services in Scotland towards the outcomes that are important to citizens and communities.
She has conducted research into the outcomes important to people who use services; the practice of outcome measurement and recording and the use of outcomes focused approaches across public services. She led pioneering improvement programmes bringing organisations together to develop and improve practice in focusing on outcomes at programme, personal and population levels.
Ailsa is a co-originator of the Talking Points Personal Outcomes Approach, now used across thousands of organisations in Scotland and beyond. She co-founded Matter of Focus to take this work to more organisations.
Twitter: @AilsaCook
Image credit: Photo by Matthew Henry
If you enjoyed this blog post, you may also be interested to read:
Disclaimer: The views and opinions expressed on this blog site are solely those of the original blog post authors and other contributors. These views and opinions do not necessarily represent those of the Policy Press and/or any/all contributors to this site.
Evidence & Policy Blog 