Search

One of the most dominant trends in the field of evaluation in recent years is the institutionalization of evaluation under headlines such as “evaluation culture,” “evaluation policy,” and “evaluation systems.” There has been less interest in how evaluation systems can be changed, modified, or improved, not to mention deinstitutionalized, if necessary. Can a variety of stakeholders deliberate about the consequences of an evaluation system, and can it lead to policy change? A case study of a ministerial advisory group on national tests in Denmark shows how specific challenges were dealt with, such as the design of the deliberative process, the potential dominance of experts, and the distinction between technical-evaluative and practical-political arguments, and how these maneuvers paved the way for policy change. Based on the case study, the article discusses the prospects for democratic deliberation about evaluation systems.

What can be said about effect of an intervention without a control group? The lack of evaluative evidence is a long-standing problem for regulatory policies against work-related health and safety risks. The European Union Occupational Safety and Health Framework has been in operation for three decades and covers more than 200 million workers, but the most recent evaluation was inconclusive about the benefits generated by this framework. A theory-based evaluation focusing on mechanisms in combination with a design capturing within-intervention variations offers a way forward. The idea is to measure the prevalence of most likely mechanisms and their correlation with outcomes. This approach is illustrated in a large- N evaluation of the use of workplace assessments in the public sector in Denmark. The strengths and weakness of the workplace assessment legislation are assessed. It is shown how findings based on the presented approach contribute to the public debate about workplace assessments.

An important motivation for the institutionalization of evaluation culture in countries around the world is the belief that accountability and transparency will thereby be enhanced. We subject this narrative about evaluation’s contribution to good governance to empirical analysis. We also argue that the meaning and relevance of this general narrative differs across national contexts. We build on data from a systematic assessment of evaluation culture in 19 countries (Jacob et al., 2015), add one country using the same systematic approach, and combine these findings with an indicator of transparency in government provided by Transparency International. We find a positive correlation between evaluation culture and transparency, and discuss threats to a causal interpretation hereof. We go into depth with two particular countries at either end of the transparency scale. We argue that the meaning of the link between evaluation culture and transparency differs whether the chosen perspective is generally comparative or situated in particular national, historical, and political contexts. In countries where transparency is high already, there might be diminishing marginal returns on evaluation, at least regarding its contribution to accountability and transparency.

Based on experiences from two evaluation projects, the two partners in the following dialogue discuss the meaning of ‘resistance’ to evaluation, or more specifically, resistance to evaluation touching upon values that are central in some communities. Step by step, the dialogue identifies a tension between evaluation as a modernist undertaking and the traditional values of these communities. Through the dialogue about this tension, the ‘rough ground’ in evaluation is revealed, then the implications for evaluation practice are discussed.

Theory-based evaluation utilizes terms such as ‘moderators’ and ‘CMO configurations’ to describe how the validity of a programme theory depends on the context in which it operates. From a constructivist perspective, however, contexts are influenced by ongoing social (re)construction. The truth value of a programme theory is contingent upon this construction. This helps explain why tests of programme theories provided by theory-based evaluators are often unconvincing to programme advocates. A typology of ‘magic’, ‘tragic’ and ‘competing’ programmes, each of which influence their contexts in different ways, is described and the implications for evaluation discussed.