The effectiveness, cost-effectiveness and policy processes of regulatory, voluntary and partnership policies to improve food environments: an evidence synthesis

Background Dietary factors are among the largest and costliest drivers of chronic diseases in England. As a response, the government implements a range of population interventions to promote healthy diets by targeting food environments. Objectives This study aimed to conduct a systematic review of the effectiveness, cost-effectiveness and policy process of real-world evaluations of national and state policies on improving food environments, with a focus on whether they were regulatory, voluntary or partnership approaches. Data sources Fourteen relevant English-language databases were searched in November 2020 for studies published between 2010 and 2020. Methods Six separate evidence reviews were conducted to assess the evidence of effectiveness, cost-effectiveness and policy processes of policies to improve food environments. Results A total of 483 primary research evaluations and 14 evidence syntheses were included. The study reveals considerable geographic, methodological and other imbalances across the literature, with, for example, 81% of publications focusing only on 12 countries. The systematic reviews also reveal the effectiveness and cost-effectiveness of reviewed regulatory approaches designed to improve health, consumer behaviour and food environment outcomes while public–private partnerships and voluntary approaches to improve diets via reformulation, advertising and promotion restrictions or other changes to the environment were limited in their effectiveness and cost-effectiveness. The study also revealed key enabling and impeding factors across regulatory, voluntary and public–private partnership approaches. Conclusion From the available evidence reviewed, this study finds that regulatory approaches appear most effective at improving the food environment, and voluntary agreements and partnerships have limited effectiveness. These findings should be carefully considered in future public health policy development, as should the findings of geographic imbalance in the evidence and inadequate representation of equity dimensions across the policy evaluations. We find that food policies are at times driven by factors other than the evidence and shaped by compromise and pragmatism. Food policy should be first and foremost designed and driven by the evidence of greatest effectiveness to improve food environments for healthier diets. Limitations This was a complex evidence synthesis due to its scope and some policy evaluations may have been missed as the literature searches did not include specific policy names. The literature was limited to studies published in English from 2010 to 2020, potentially missing studies of interest. Future work Priorities include the need for guidance for appraising risk of bias and quality of non-clinical studies, for reporting policy characteristics in evaluations, for supporting evaluations of real-world policies equitably across geographic regions, for capturing equity dimensions in policy evaluations, and for guideline development for quality and risk of bias of policy evaluations. Study registration This study is registered as PROSPERO CRD42020170963. Funding This award project was funded by the National Institute for Health and Care Research (NIHR) Public Health Research programme (NIHR award ref: NIHR128607) and is published in full in Public Health Research; Vol. 12, No. 8. See the NIHR Funding and Awards website for further award information. Plain language summary Poor diet is a leading cause of death, globally, including in the United Kingdom. It also causes many types of illness and is one of the biggest drains on the United Kingdom National Health Service budget. Governments act in various ways to promote healthy diets by improving food environments: these are the physical and social surroundings that influence what and how much people eat. Some actions are regulated by government, for example, to control food production, marketing and promotions. Other actions are led by, or with, food businesses, making voluntary changes to the foods they produce, for example, by reducing salt content; this can be done by businesses alone or in partnership with government (referred to as ‘public–private partnerships’). The six reviews of published research look at whether, and how, these actions to improve diets work, and whether they can provide value for money. Most regulations appear to be effective at supporting better diets. However, voluntary changes led by businesses had limited success. There were not many evaluations that assessed the effectiveness of public–private partnerships. Of those that did, partnerships with the food industry had limited effectiveness, resulting in largely unchanged outcomes. When looking at how these actions improve diets, we found that clear leadership, public support for the policy, the use of the best evidence and of local expertise helped with getting actions implemented. Factors that appear to make it harder to implement policy actions include a lack of evidence specific to the context, conflicting beliefs about what works, limited human or financial resources, lack of engagement by key people. Although the findings may help us to think about the ways forward to improve diets, more research is needed to understand whether actions to reduce diet-related ill health work, and provide value for money. Scientific summary Background Unhealthy diets are now estimated to be responsible for more deaths than any other risk globally, including in England. Dietary factors underpin many of the major chronic diseases currently faced by the population and are estimated both to be the largest contributor to overall disease and to have the highest impact on the NHS budget. It is in this context that the English government has implemented a range of population interventions to promote healthy diets by improving food environments. These range from public–private partnerships (PPPs) and voluntary programmes to regulatory policies, aiming to reduce ingredients like sodium and sugars in foods through interventions, such as reformulation, advertising reduction/bans and labelling. Governance arrangements in policy are broadly defined here as whether or not a policy intervention is voluntary, PPP or government led. Governance is a key overlooked mechanism in these interventions, and also in reviews of policy responses, and it is a fundamental part of the context, which is insufficiently discussed. It is essential to understand how governance arrangements in policy have an impact on effectiveness, by understanding what factors relating to interventions, providers, populations and settings affect the implementation of such population interventions to improve diet. Aims This study, cost-effectiveness and policy processes of population interventions to improve diet with a focus on governance, and with a view to informing the most effective responses to unhealthy diets, formulating implications for diets in England. Research questions How are regulatory interventions, voluntary approaches and PPPs to improve diet assumed to work in theory? What regulatory interventions, voluntary approaches and PPPs to improve diet, and reduce inequalities in diet improvement, have been evaluated? What factors relating to interventions, providers, populations and settings affect implementation of such population interventions to improve diet? Have such population interventions improved process, impact (intermediate and distal) and cost outcomes? Are there any reported unanticipated effects of such population interventions? What is the cost-effectiveness of such population interventions? How can the findings of the evidence review be translated into recommendations for improved interventions? Methods This evidence synthesis consists of six individual evidence syntheses based on a common systematic literature search [see Figure A (Figure 2 in main text)]. The evidence syntheses consisted of a systematic evidence map of primary research, an overview of reviews on the effectiveness of regulatory, voluntary and PPP approaches, two systematic reviews on the effectiveness of PPPs, and voluntary approaches by private actors to fill gaps in the overview of review, as well as another systematic review on the cost-effectiveness of regulatory, voluntary and PPP approaches, and a qualitative evidence synthesis of the policy process. FIGURE ARelationships between review outputs. We ran searches of real-world evaluations of policies (defined as evaluations conducted while the policy was adopted or implemented, or as part of a state or national public consultation) aiming to improve diets by targeting the food environments, and published between 2010 and 2020, across 14 databases in November 2020. Records were uploaded to the Evidence for Policy and Practice Information-Reviewer Web for the removal of duplicates, screening, as well as part of the data extractions for the different reviews of the series. About 12% of titles and abstracts and 33% of full texts eligible for the overarching project were screened by at least two reviewers independently. The remainder were screened by one reviewer after reaching a 90% agreement rate, except for those excluded because they were not about real-world policy (all were checked by a second reviewer). Disagreements were discussed with a third reviewer. Depending on the review, data were either extracted by one reviewer and checked by another or extracted by two independent reviewers. The specific methods and results for each review are presented in their respective chapters. All papers included in one of the four effectiveness or cost-effectiveness reviews were independently critically appraised by two reviewers. The quality of evidence syntheses in the overview of reviews was appraised using the checklist by the Scottish Intercollegiate Guidelines Network. For the systematic reviews of PPP evaluations and of voluntary approaches, study quality was assessed using a modified version of the Newcastle–Ottawa Scale for cross-sectional studies. The review of voluntary approaches included 10 single cross-sectional studies and 5 repeat cross-sectional studies, all about food items, advertising material and other non-human ‘participants’. The studies in the PPP review consisted of two single cross-sectional studies, nine repeat cross-sectional studies, one interrupted time series and five policy document analyses; five of which involved human participants. The quality of the cost-effectiveness analyses was assessed with Drummond’s 10-criteria checklist version 2015. For the policy process review, their contributions were moderated on the basis of three criteria: relevance, rigour and richness. Data for the five reviews that used quantitative data were synthesised using narrative approaches since they were not suitable for statistical pooling techniques, such as meta-analysis. In the systematic reviews and overview of reviews about effectiveness, we also used an effect direction plot to represent the summary findings graphically. We had planned to use the Grading of Recommendations, Assessment, Development and Evaluation approach in the three reviews about effectiveness but did not for different reasons explained in the respective chapters. Data in the qualitative review about policy process were analysed using a thematic analysis. Results Literature search and screening for the overarching project: A total of 38,209 records were retrieved from the databases; 27,887 remained after removing duplicates and had their title and abstract screened against the eligibility criteria. Of these, 1859 met the criteria and had their full text screened, resulting in 500 records included. In parallel, 72 additional full texts were retrieved by screening websites and reference lists. Of these, 33 met the eligibility criteria, contributing to a total of 533 publications: 483 reporting on primary research evaluations and 50 on evidence syntheses. All 483 primary studies were included in the evidence map. All evidence syntheses were screened for inclusion in the overview of reviews. Of the 483 publications on primary research, all those assessing the effectiveness of PPPs (n = 18) were considered for the systematic review on the latter, and all those assessing the cost-effectiveness of policies (n = 4) were included in the systematic review on the latter. For the systematic review of the effectiveness of voluntary approaches by private actors, given the high number of potential primary studies (n = 186), only those comparing outcomes between participants and non-participants (NP) in the policies and that were published in journals in the most recent 4 years (2017–20) were considered (n = 15). Lastly, the review of policy process included both primary studies and evidence syntheses of both qualitative and mixed-methods design that assessed factors influencing policy development or implementation. Again, due to their high number (n = 87), only those published in 2019 and 2020 were considered (n = 33). Systematic evidence map: We found imbalances across the 483 included studies, suggesting that policy evaluations are conducted and published inequitably across the world both in terms of quantity and quality. Though 70 countries were represented overall, 81% of publications focused on only 12 countries (USA, UK, Australia, Canada, Mexico, Brazil, Chile, France, Spain, Denmark, New Zealand and South Africa), and 30% included the USA. Few evaluations were found about Africa, Central and South Asia, and the Middle East. Inequities were also detected in the study designs, with the most quantitative robust methods mainly documenting the abovementioned 12 dominant countries. Few publications reported on PPPs (n = 31), and only one assessed the development of voluntary policies led by the public and private sectors each. Using a generous interpretation of the PROGRESS-Plus equity dimensions, we found that not only 50% of publications assessing policy effectiveness did not compare outcomes by any equity domain, but that the proportion of those doing so has decreased over time. Age was the most frequently assessed dimension, while occupation, religion and culture, social capital and disability were barely considered. Overview of reviews: An overview of reviews of 11 systematic reviews was conducted to review the effectiveness of policies by governance approach. Three additional systematic reviews were considered for addressing questions about equity. The 11 systematic reviews primarily assessed regulatory policies (especially taxation) and a few voluntary approaches by the public sector, food retailers and restaurant chains. No PPP was included, and few regulatory and voluntary initiatives could be directly compared. Overall, the results suggest that, except for salt-related taxes, most regulatory approaches designed to improve health, consumer behaviour (e.g. food intake, purchases), and food environment outcomes were effective. These mainly consisted of trans-fat bans, taxes on sugar-sweetened beverages (SSBs) and non-essential foods (except for salt intake), and front-of-pack labelling (FOPL). Effects for voluntary approaches by public and private actors were also generally positive for salt and trans-fat reformulation (but regulatory trans-fat bans were more promising), labelling on products and supermarket shelves, and changing defaults in children’s menus in restaurants, although some of them relied on single cross-sectional studies in the USA. Results for voluntary menu labelling and multicomponent commitments by large retail chains were mixed. The findings by PROGRESS-Plus categories indicate a lack of reporting of outcomes in systematic reviews for population groups that are prone to health inequalities: overall, evidence on equity is patchy, incomplete, mainly inconsistent and largely relies on single studies (although some evaluations had large samples) rather than aggregated bodies of evidence. Review of PPPs: This systematic review aimed to complement the overview of reviews by assessing the effectiveness of PPPs targeting the food environment since the overview of reviews did not include data on the latter. We reviewed 17 studies evaluating seven PPPs to improve population diets and food environments. Five involved humans or data about human behaviours (sales and purchases) using cross-sectional design (single or repeat) or intermittent time series, seven assessed food products or labelling practices (all single and repeat cross-sectional), and five examined policy content and progress using documents. Overall, studies found that partnerships with the food industry to improve diets via reformulation or other changes to the environment have limited effect at achieving this aim. Review of voluntary approaches by private actors: This systematic review aimed to complement the overview of reviews by assessing the effectiveness of voluntary approaches by private actors between participants and NPs. Sixteen studies evaluating nine voluntary approaches were included. Policies were mainly about advertising and marketing control, reformulation, and the retail and catering sectors. All were cross-sectional: nine single and seven post–post. All measured effects or associations on the food environment, such as product or advertising characteristics using cross-sectional designs. None evaluated humans. When comparing the commitment’s participants to NPs, the direction of effect or of association for most outcomes was either inconclusive or worse for participants. There is no evidence to suggest that policies designed as voluntary approaches led by commercial actors are effective at reducing advertising and other promotion of unhealthy products to children, improving the nutritional composition of food and beverages, encouraging calorie menu labelling in chain restaurants, or reducing marketing targeting children on food packages. Review of cost-effectiveness: Four studies of the cost-effectiveness of real-world policies to promote healthy diets were identified. Two studies suggested positive impacts for the fiscal measure of the Mexican SSB tax, and one for the voluntary government-led Health Star Rating FOPL intervention in Australia. The fourth one showed a lack of effectiveness and cost–benefit for the PPP intervention to reduce salt consumption in England as part of the Public Health Responsibility Deal. There is a pressing need to build on the extensive literature on the effectiveness of interventions with high-quality evidence on cost-effectiveness, to support meaningful action to tackle the scourge of diet-related ill health. Review of the policy process: We included 33 papers studying the policy process (factors shaping from design to implementation) of regulatory, voluntary and PPP approaches. The studies on regulatory policy processes highlighted key enabling factors, including clear leadership; policy entrepreneurs, champions and strategists; policy supporters’ coalitions; the active use of best evidence and local expertise; institutional and financial capacity; and harnessing focusing events. They also pointed to impeding factors, including practical considerations often around policy implementation and the generation of context-specific evidence; differing ideological positions; capacity constraints; the role of evidence; lack of key stakeholder engagement, and the deployment by the food and beverage industry of frames and framing strategies to ultimately oppose a policy process. This review also advanced the understanding of phenomena and mechanisms underpinning the policy process for improving diet, in particular how large food and beverage industries have become legitimate actors in policy interventions to improve diets. Conclusion To our knowledge, this is the first review of real-world evaluations of policies to improve food environments. From the available evidence reviewed, regulatory appears to be the most effective approach, and voluntary industry approaches and PPPs have limited effectiveness. The geographic imbalance in the evidence and inadequate representation of equity dimensions across the policy evaluations should be redressed. Food policy should be designed and driven by the evidence of greatest effectiveness to improve food environments for healthier diets. The role and interests of policy stakeholders, and the accepted legitimacy of corporate actors in food policy, should be critically interrogated with healthy diets for the population as the first priority. Implications for future work Implications for public health policy include prioritising structural changes through regulation as they are found to be the most effective approach to improve the food environment; expanding geographical representation of the evidence, capturing dimensions of health equity as policy outcomes; taking a systems approach across policies to improve food environments. Implications for future policy evaluations include developing guidance for appraising risk of bias and quality of non-clinical studies and for reporting policy characteristics in evaluations. Implications for future research and research funding include conducting studies on the role of corporate actors in food policy, conducting evaluations of real-world policies equitably across geographic regions, capturing equity dimensions in policy evaluations, and developing guidelines for quality and risk of bias of policy evaluations. Study registration This study is registered as PROSPERO CRD42020170963. Funding This award was funded by the National Institute for Health and Care Research (NIHR) Public Health Research programme (NIHR award ref: NIHR128607) and is published in full in Public Health Research; Vol. 12, No. 8. See the NIHR Funding and Awards website for further award information.