A Literature Review on Health Communication Campaign Evaluation with Regard to the Prevention and Control of Communicable Diseases in Europe

Report Executive summary Introduction This review collates and summarises the literature on communication campaign evaluation with relevance to the prevention and control of communicable diseases. The purpose of this review is to contribute to the evidence base on health communication evaluation research in order to aid public health professionals and researchers in the development of future evaluation strategies. The review is divided into two sections. In the first section the focus is on reviewing evaluations of campaigns undertaken in European Union (EU) and European Economic Area (EEA) countries, however, examples from the wider European region are also included. The second section, addressing challenges posed by campaign evaluation, draws on broader international literature pertaining to the identification of health communication campaign evaluation tools, frameworks and models. It would appear that there has been very few high quality European evaluation studies carried out in the last decade in relation to communicable disease prevention campaigns. Nevertheless, it is evident in reviewing the literature that there are valuable existing frameworks and guidelines that can help guide and inform evaluation research development. Methodology The review examined the international English-language literature published between 2000 and 2011. The search strategy devised included a review of a number of databases of published academic literature using identified key words which included but were not restricted to: health communication public health campaign and evaluation/effectiveness and communicable infectious disease . In addition, specific journals were extensively reviewed for relevant articles. Results The reviewers retrieved 160 references of which 35 were examples of evaluations of health communication campaigns carried out in EU/EEA Member States. A review of examples of campaign evaluations Interventions labelled health communication campaigns vary greatly in their topic focus, activities, design and exposure and this is demonstrated in the breadth of examples identified. The range of approaches to study design include: systematic and exploratory reviews, experimental and randomised, non-randomised, time-series, multiple method, longitudinal, before-after, cross-sectional, content analysis and cost-effectiveness. The critique of these examples reveals evidence of: weak study designs, small sample sizes, lack of control or comparison groups, lack of theoretical foundation, underuse of formative and process evaluation, lack of reference to capturing unintended effects and evaluation aims and outcome measures that do not correspond with campaign objectives. These limitations are not restricted to the European context and have also been recognised as challenges in the international literature on the evaluation of health communication campaigns. Addressing the challenges of campaign evaluation A number of principles, frameworks and guidelines are outlined in the literature that can assist researchers and public health professionals in planning campaign evaluation. Those identified include: Bauman s guidelines for campaign developers [1], Centers for Disease Control and Prevention (US) Framework for Programme Evaluation [2], Audience, Channel, Message, Evaluation (ACME) Framework developed by Noar [3], the US. National Cancer Institute s, Making Health Communications Programs Work [4], and the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) Framework [5]. A number of indicators of success have been consistently cited in the literature as contributing to efficient and effective practice in relation to evaluation of health communication campaigns. Identified indicators of success include: be clear about what is being evaluated, be realistic and explicit about the expected direct and indirect effects, consider potential unintended effects, and use appropriate theory to inform campaign development and evaluation. An important factor identified is that evaluation, including cost effectiveness analysis, should be integrated with formative, process and summative evaluation through campaign planning, development and implementation. Preferred evaluation research designs promoting rigour include: pre-post test, cohort designs, time-series designs and natural experiments. The use of post-test only designs is severely compromised but may be the only option in situations of scarce resources. Acknowledging and documenting the context in which campaigns are implemented will facilitate the transfer of knowledge between the diverse EU/EEA states, and sharing experiences will optimise the use of resources and the development of skills. 1Literature review on health communication campaign evaluation TECHNICAL REPORT Conclusion The development of a strong evidence base is imperative to drive effective and efficient policy and practice in the use of health communication campaigns for the prevention and control of communicable diseases. The challenge within the context of EU/EEA countries is to develop capacity for evaluation research within countries, while acknowledging and addressing the barriers to the application of this knowledge base to health communication evaluation practice. European Centre for Disease Prevention and Control (ECDC) - Framework Partnership Agreement Grant/2009/007 non-peer-reviewed