Are good evaluations used more than bad ones?

This blog post is based on the Evidence & Policy article, ‘Does evaluation quality enhance evaluation use?

Pirmin Bundi, Kathrin Frey and Thomas Widmer

Evaluations provide important information to improve public services, but only if they yield valid and reliable findings – so we believed for a long time. Evaluation communities have therefore established certain criteria that should define evaluation quality. Yet against prior studies on evaluation utilisation, we show that evaluation quality measured by the criteria is not necessarily associated with evaluation use, but rather linked to the perception of quality and impact of the evaluation. Evaluators should adjust their communications strategies accordingly.

Continue reading

Stateless nomads, with little to trade (or how knowledge brokers are set up to fail and how to avoid this)

This blog post is based on the Evidence & Policy article, ‘Understanding knowledge brokerage and its transformative potential: a Bourdieusian perspective‘.

Graham Martin, Sarah Chew and Natalie Armstrong

Some problems in society result from institutions’ traditional tendency to work in isolation from one another. An example is the slothful pace at which evidence from healthcare research reaches practice:  some estimates suggest it can typically take as long as seventeen years. Increasing collaboration between institutions is the obvious remedy, but ‘If you think competition is hard, you should try collaboration’.

The institutional fields of research and practice have very different structures and value systems. This means that getting them to collaborate requires some external impetus. Recently, knowledge (brokering a range of activities designed to link the producers and users of knowledge by, for example, encouraging new relationships, devising new ways of working together, and helping to move knowledge across boundaries) has been promoted as a way of enabling collaboration and even bringing about changes in the working relationships of institutions. Knowledge brokerage has become a role in its own right, but its popularity as a remedy outstrips evidence for its efficacy.

Continue reading