Search

Background Whole-school interventions modify the school environment to promote health. A subset of these interventions promotes student commitment to school to prevent substance (tobacco, alcohol, other drugs) use and/or violence. A previous review identified the theory of human functioning and school organisation as a comprehensive theory of such interventions, and found evidence that these interventions reduce substance use and/or violence. Objectives The objectives were to search for, appraise and synthesise evidence to address the following questions: (1) What whole-school interventions promoting student commitment to school to prevent substance use and/or violence have been evaluated, what intervention subtypes are apparent and how closely do these align with the theory of human functioning and school organisation? (2) What factors relating to setting, population and intervention affect implementation? (3) What are the effects on student substance use, violence and educational attainment? (4) What is the cost-effectiveness of such interventions? (5) Are intervention effects mediated by student commitment to school or moderated by setting or population? Data sources A total of 56 information sources were searched (in January 2020), then an updated search of 48 of these was carried out (in May 2021). Reference lists were also searched and experts were contacted. Review methods Eligible studies were process/outcome evaluations of whole-school interventions to reduce student violence or substance use among students aged 5–18 years attending schools, via actions aligning with the theory of human functioning and school organisation: modifying teaching to increase engagement, enhancing student–staff relationships, revising school policies, encouraging volunteering or increasing parental involvement. Data extraction and quality assessments used existing tools. Theory and process reports were synthesised qualitatively. Outcome and economic data were synthesised narratively; outcome data were meta-analysed. Results Searches retrieved 63 eligible reports on 27 studies of 22 interventions. We identified four intervention subtypes focused on student participation in school-wide decisions, improving staff–student relationships, increasing engagement in learning and involving parents. The theories of change of most intervention subtypes aligned closely with the theory of human functioning and school organisation, and informed refinement of an intervention theory of change. Theories of change for interventions increasing learning engagement did not align with this theory, aiming instead to increase school commitment primarily via social skills curricula. Factors influencing the implementation included whether or not interventions were tailorable, workable and well explained. Interventions with action groups comprising staff/students, etc. and providing local data were well implemented. Implementation was also affected by whether or not schools accepted the need for change and staff had the resources for delivery. Meta-analyses suggest small, but significant, intervention effects in preventing violence victimisation and perpetration, and substance use. There was sparse and inconsistent evidence of moderation and some evidence of mediation by student commitment to school. Two economic evaluations suggested that there is the potential for the interventions to be cost-effective. Limitations The quality of the studies was variable and the economic synthesis was limited to two studies. Conclusions Whole-school interventions aiming to promote student commitment to school share similar theories of change and factors affecting implementation. They have the potential to contribute to preventing violence and substance use among young people. Future trials should aim to optimise intervention effectiveness by better theorisation, and assess implementation and effect moderators and mediators. Study registration This study is registered as PROSPERO CRD42019154334. Funding This award was funded by the National Institute for Health and Care Research (NIHR) Public Health Research programme (NIHR award ref: 17/151/05) and is published in full in Public Health Research; Vol. 12, No. 2. See the NIHR Funding and Awards website for further award information. Plain language summary Whole-school health interventions aim to modify how schools are run, to promote students’ health. Some aim to promote student commitment to school to prevent the important interlinked outcomes of substance (tobacco, alcohol, other drugs) use and violence. We searched for all evaluations of such interventions. We summarised what this research said about the sorts of interventions used, how they are meant to work, what factors affect delivery, whether or not they reduce violence and substance use and whether or not they are worth the money. We found 63 reports on 27 studies of 22 interventions. We identified four subtypes of interventions. These aimed to involve students in school decisions, improve staff–student relationships, increase engagement in learning or involve parents. Most of these interventions were intended to work by making sure schools focused on student needs, or by improving relationships between staff and students, between different areas of learning or between schools and communities. This aimed to make students feel committed to school and therefore avoid violence or substance use. A few aimed to work mostly by teaching students how to avoid violence and substance use. We found that interventions were well implemented if they were tailored for each school and had good materials and support. Interventions were well delivered if they were led by action groups (comprising staff, students, etc.) or provided schools with information on students’ needs. Implementation was affected by whether or not schools accepted the need for change and whether or not staff had the necessary time and money to do the work. These interventions appear to have small, but significant, intervention impacts in preventing violence and substance use among young people. There was not consistent evidence of different effects for different students. A small number of studies suggest that such interventions might show economic benefit, but this would need further research. Future research should focus on interventions that are refined to make sure that they can be well delivered. Scientific summary Background and rationale Substance use and violence are important interconnected outcomes often associated with disengagement from school. Whole-school interventions aiming to modify the school environment to promote student commitment to school might be an effective way to prevent these outcomes. This review synthesised evidence on such interventions. According to a previous review, the theory of human functioning and school organisation provides the most comprehensive theory of how schools can influence student commitment to school and health behaviours. This proposes that promoting student commitment to school can decrease involvement in risk behaviours by promoting student practical reasoning and positive peer affiliations. Schools can promote commitment by reframing provision on student needs and eroding boundaries between staff and students, between different areas of learning and between schools and local communities. We used this theory to define our initial theory of change and inclusion criteria, and as a starting point for categorising interventions and synthesising theories of change. Aim and review questions The aim was to search systematically for, appraise and synthesise evidence to address the following questions: What whole-school interventions promoting student commitment to school to prevent student substance use and/or violence have been evaluated, what subtypes are apparent and how closely do these align with the theory of human functioning and school organisation? What factors influence implementation of such interventions? Overall and by intervention subtype, what are the effects of such interventions on student substance use, violence and educational attainment? What is the cost-effectiveness of such interventions? Are the effects of such interventions on student substance use and/or violence mediated by student commitment to school, or moderated by setting or population? Methods Inclusion criteria Eligible studies were process or experimental/quasi-experimental outcome evaluations of whole-school interventions aiming to reduce violence or substance (tobacco, alcohol, other drug) use among students aged 5–18 years in schools, by modifying teaching to increase engagement in learning, enhancing student–staff relationships, revising school policies, encouraging student volunteering or involving parents in school. Searching information sources The search strategy included terms covering population, intervention and evaluation design. We initially searched 21 databases, 3 trial registries and 32 websites (16–27 January 2020), and updated the search across 14 databases, 2 trial registries and 32 websites (11–25 May 2021). We searched reference lists and contacted subject experts. Information management and study selection Citations identified by our searches were de-duplicated before being uploaded to EPPI-Reviewer 4.0 software (Evidence for Policy and Practice Information and Co-ordinating Centre, University of London, London, UK). Two reviewers double-screened batches of 50 references. Disagreements were resolved by discussion. After reaching an agreement rate of ≥ 90% on these batches, each subsequent reference was single-screened on title/abstract. If references were considered likely to be eligible based on the title/abstract, the full report was reviewed using a similar process. Data extraction Two reviewers independently extracted data using existing tools. Disagreements were resolved by discussion. For intervention descriptions, date were extracted on domains included in a standard framework. For theory reports, we extracted data on constructs, mechanisms and contextual contingencies affecting these. For empirical studies, we extracted data on basic study details, methods, interventions and findings. Assessments of quality and risk of bias The quality of each report was assessed independently by two reviewers using existing tools. The reviewers met to compare assessments and resolve any differences through discussion. Theory reports were assessed on the basis of whether or not this described the path from intervention to outcomes, clarity of constructs, clarity of inter-relationships between constructs, whether or not the underlying mechanisms were explained, and whether or not the theory considered how mechanisms and outcomes might vary by context. Process evaluations were assessed on the basis of rigour of sampling, data collection and data analysis; the extent to which findings were grounded in data; whether or not the study privileged the perspectives of students; and breadth/depth of findings. Outcome studies were assessed for risk of bias on the basis of sequence generation, allocation concealment, blinding of participants or personnel, blinding of outcome assessors, incomplete outcome data, selective outcome reporting and other sources of bias. Economic evaluations were assessed using an adapted version of an existing tool. Data analysis To create a categorisation of intervention subtypes, we drew on descriptions of interventions and theories of change, using these to refine an initial categorisation informed by the theory of human functioning and school organisation. Reviewers grouped interventions into discrete categories and subcategories. To synthesise theories of change, we used best-fit framework synthesis. We defined a priori themes based on the theory of human functioning and school organisation and then coded data from included studies against these. When concepts from the included studies could not be coded with these, we coded inductively. This coding was then used to refine the existing theory of change by intervention subtype. We synthesised qualitative and quantitative elements of process evaluation reports using thematic synthesis methods. We conducted a narrative synthesis of outcome evaluations ordered by outcome then, within this, by intervention subtype, follow-up time and study design. Outcomes were categorised into violence, use of tobacco, alcohol and other drugs, and academic attainment. We produced forest plots for different outcomes and follow-up times. Plots included point estimates and standard errors for each study, expressed as standardised mean differences (Cohen’s d) to ensure comparability across reports. When data allowed, we calculated pooled effect sizes within each pairwise comparison, accounting for the extent of heterogeneity among the studies, estimating separate models for each outcome. We regarded follow-up times of up to 1 year and > 1 year post baseline as different outcomes, pooling first by follow-up times and, when appropriate, overall across follow-up times. We used the Grading of Recommendations Assessment, Development and Evaluation to present the quality of evidence. For synthesising economic evaluations, measures of costs, indirect resource use and cost-effectiveness were summarised in a table and adjusted for currency and inflation to the then-current UK context. These data were used to inform a narrative synthesis of economic evidence. Stakeholder consultation We consulted policy and practice stakeholders once during the review. In October 2021, stakeholders reviewed slides summarising the main findings. We asked stakeholders to advise on interpretation, implications and knowledge exchange. Ethics approval The research involved no human participants and drew solely on evidence in the public realm, so ethics approval was not required. Results Included studies The original searches retrieved 62,742 unique references and 56 eligible reports. The updated search retrieved 9709 unique references and 9 eligible reports. In total, 63 reports on 27 studies of 22 interventions were included: 63 on theories of change, 16 on process evaluations, 48 on outcome evaluations and 3 on economic evaluations. Of the included interventions, 10 addressed violence (with 2 of these also addressing educational attainment), 3 addressed substance use (with 1 of these also addressing educational attainment) and 9 addressed violence and substance use (with 1 of these also addressing educational attainment). What interventions have been evaluated and what intervention subtypes are apparent? We identified four intervention subtypes, focused on the following: developing student participation in school-wide decisions, improving staff–student relationships, increasing student engagement in learning and increasing parental involvement in school. How closely do intervention theories of change align with the theory of human functioning and school organisation? We synthesised interventions developing student participation, enhancing staff–student relationships and involving parents together because of commonalities in theories of change. Despite only one intervention being explicitly informed by the theory of human functioning and school organisation, theories of change aligned closely with this theory, with recurrent themes that reciprocally translated with this theory and each other. Inductive coding suggested a number of refinements to the starting theory, for example relating to how school organisation, learning, discipline and school–community links were reframed. Interventions aimed not only to reduce risk behaviours but to promote positive overall development via increasing commitment to school. The theories of change for interventions developing student engagement in learning aligned much less with the theory of human functioning and school organisation. These interventions were theorised to increase students’ school engagement primarily through social skills curricula, with whole-school elements secondary. What factors relating to setting, population and intervention influence the implementation of these interventions? Evaluations suggested that school staff were more likely to understand what was required to implement an intervention when provided with good materials and support. Whether or not staff would commit to delivery was influenced by whether interventions could be integrated with existing practices, locally tailored or built on existing work. Interventions providing local data helped build buy-in. Staff were more likely to commit when external providers were credible and experienced. Students were more likely to commit if an intervention offered opportunities for active roles or expressing their views. School leaders were more likely to commit to an intervention that addressed an issue they were already interested in tackling and when there was already a recognition of the need for change. To ensure those in schools worked together to deliver interventions, interventions needed to be locally workable, fitting with timetables and providing clear guidance. Interventions that included ‘action groups’ (consisting of staff, students, etc.) helped build collective action. Interventions with synergistic components were implemented more successfully. Whether or not staff could come together to deliver interventions also depended on having the time, budgets, authority and connections to support this. Action groups enabled members to reflexively monitor implementation and assess what else was needed. This could give participants the permission and resources to try different things, persisting with what worked. What are the effects on student substance use, violence and educational attainment of whole-school interventions aiming to promote student commitment? Overall, meta-analyses suggested that interventions led to a small, but statistically significant, reduction in violence perpetration at up to 1 year post baseline [odds ratio (OR) 0.85, 95% confidence interval (CI) 0.76 to 0.96] and > 1 year post baseline (OR 0.79, 95% CI 0.65 to 0.98). Considering intervention subtypes, meta-analyses suggested possible, but inconsistent, impacts on violence perpetration of interventions promoting student participation in school policy decisions and student relationships with teachers at up to 1 year post baseline and at > 1 year post baseline. Meta-analyses suggested that interventions promoting student engagement in learning and parental involvement were unlikely to affect violence perpetration up to 1 year post baseline and at > 1 year post baseline. Overall, meta-analyses suggested that interventions led to small, but statistically significant, reductions in violence victimisation at up to 1 year post baseline (OR 0.84, 95% CI 0.72 to 0.98) and > 1 year post baseline (OR 0.85, 95% CI 0.73 to 0.99). Considering subtypes, evidence suggested possible but inconsistent impacts of interventions promoting student participation in decisions and student relationships with teachers on violence victimisation at up to 1 year post baseline and > 1 year post baseline. Studies suggested that interventions promoting parental involvement were unlikely to affect violence victimisation up to 1 year post baseline and > 1 year post baseline. Studies of intervention effects on observed violence were not meta-analysed owing to heterogeneity of study designs, but evidence suggested that interventions promoting student participation in decisions significantly reduced observed violence at > 1 year post baseline. Interventions promoting parental involvement significantly reduced observed violence at up to 1 year post baseline and at > 1 year post baseline. Overall, meta-analyses suggested that interventions led to small, but statistically significant, reductions in substance use at up to 1 year post baseline (OR 0.83, 95% CI 0.70 to 0.97) and at > 1 year post baseline (OR 0.79, 95% CI 0.62 to 0.998). Meta-analyses of such interventions suggested small and statistically non-significant reductions in measures of specific outcomes (alcohol, smoking, illicit drugs, general substance use) at up to 1 year post baseline and at > 1 year post baseline. Considering subtypes, the evidence suggested possible, but inconsistent, impacts of interventions promoting student participation in decisions in reducing substance use at up to 1 year post baseline and at > 1 year post baseline, with similar findings for alcohol outcomes and illicit drug use outcomes separately. There was less evidence of impact on tobacco outcomes. An analysis of omnibus substance use outcomes suggested a larger, but still substantially heterogeneous and non-significant, effect. One trial reported on the effects of an intervention promoting student–teacher relationships: up to 1 year post baseline, there were no significant effects on alcohol or cannabis outcomes, and no significant effects on any reported lifetime smoking, but there was a significant effect on regular smoking; at > 1 year post baseline, there were no significant effects on alcohol use, tobacco smoking or cannabis use outcomes. Findings from two randomised trials suggested possible, but inconsistent, impacts of interventions promoting student engagement in learning in reducing substance use, but a non-significant impact on reducing alcohol use at up to 1 year post baseline. Evidence suggested possible, but inconsistent, impacts of interventions promoting student engagement in learning in reducing substance use at > 1 year post baseline, and a non-significant impact of such interventions on reducing alcohol use at > 1 year post baseline. Our syntheses of effects on academic attainment are exploratory. All such studies were of interventions promoting student participation in decisions and reporting attainment at > 1 year post baseline. Findings did not suggest clear evidence of impact on academic attainment. Are the effects of whole-school interventions on student substance use and violence mediated by student commitment to school, or moderated by setting or population? Evidence was sparse and inconsistent across trials for moderation of effects on all outcomes by population and school characteristics. Three studies examined mediation of intervention effects by student commitment to school. All focused on interventions promoting student participation in decisions. There was some evidence that student commitment to school mediated intervention effects on violence and substance use outcomes. What is the cost-effectiveness of such interventions, overall and by intervention subtype? Economic evidence was sparse but suggested that two interventions aiming to involve students in decision-making were cost-effective. Conclusions Whole-school interventions aiming to promote student commitment to school to prevent violence and substance use can be categorised into those promoting (1) student participation in decision-making, (2) staff–student relationships, (3) engagement in learning and (4) parental involvement in school. Despite not generally being explicitly informed by the theory of human functioning and school organisation, most interventions are underpinned by theories of change closely aligning with this. Factors influencing implementation include whether or not interventions were tailorable, workable, well explained and had synergistic components. Implementation was also affected by whether or not schools accepted the need for school transformation and whether or not staff had the necessary time, budgets, authority and connections for delivery. Interventions were effective in preventing violence victimisation and perpetration and substance use, although effects are small. Such interventions may contribute to broader efforts to reduce violence and substance use among young people but are unlikely to achieve large population impacts alone. Future trials should aim to optimise intervention effectiveness by better theorisation, and assess implementation and effect moderators and mediators. Study registration This study is registered as PROSPERO CRD42019154334. Funding This award was funded by the National Institute for Health and Care Research (NIHR) Public Health Research programme (NIHR award ref: 17/151/05) and is published in full in Public Health Research; Vol. 12, No. 2. See the NIHR Funding and Awards website for further award information.

Background Most research on community empowerment provides evidence on engaging communities for health promotion purposes rather than attempts to create empowering conditions. This study addresses this gap. Intervention Big Local started in 2010 with £271M from the National Lottery. Ending in 2026, it gives 150 relatively disadvantaged1 communities in England control over £1M to improve their neighbourhoods. Objective To investigate health and social outcomes, at the population level and among engaged residents, of the community engagement approach adopted in a place-based empowerment initiative. Study design, data sources and outcome variables This study reports on the third wave of a longitudinal mixed-methods evaluation. Work package 1 used a difference-in-differences design to investigate the impact of Big Local on population outcomes in all 150 Big Local areas compared to matched comparator areas using secondary data. The primary outcome was anxiety; secondary outcomes included a population mental health measure and crime in the neighbourhood. Work package 2 assessed active engagement in Big Local using cross-sectional data and nested cohort data from a biannual survey of Big Local partnership members. The primary outcome was mental well-being and the secondary outcome was self-rated health. Work package 3 conducted qualitative research in 14 Big Local neighbourhoods and nationally to understand pathways to impact. Work package 4 undertook a cost-benefit analysis using the life satisfaction approach to value the benefits of Big Local, which used the work package 1 estimate of Big Local impact on life satisfaction. Results At a population level, the impacts on ‘reporting high anxiety’ (–0.8 percentage points, 95% confidence interval –2.4 to 0.7) and secondary outcomes were not statistically significant, except burglary (–0.054 change in z-score, 95% confidence interval –0.100 to –0.009). There was some effect on reduced anxiety after 2017. Areas progressing fastest had a statistically significant reduction in population mental health measure (–0.053 change in z-score, 95% confidence interval –0.103 to –0.002). Mixed results were found among engaged residents, including a significant increase in mental well-being in Big Local residents in the nested cohort in 2018, but not by 2020; this is likely to be COVID-19. More highly educated residents, and males, were more likely to report a significant improvement in mental well-being. Qualitative accounts of positive impacts on mental well-being are often related to improved social connectivity and physical/material environments. Qualitative data revealed increasing capabilities for residents’ collective control. Some negative impacts were reported, with local factors sometimes undermining residents’ ability to exercise collective control. Finally, on the most conservative estimate, the cost–benefit calculations generate a net benefit estimate of £64M. Main limitations COVID-19 impacted fieldwork and interpretation of survey data. There was a short 4-year follow-up (2016/20), no comparators in work package 2 and a lack of power to look at variations across areas. Conclusions Our findings suggest the need for investment to support community organisations to emerge from and work with communities. Residents should lead the prioritisation of issues and design of solutions but not necessarily lead action; rather, agencies should work as equal partners with communities to deliver change. Funding This project was funded by the National Institute for Health and Care Research (NIHR) Public Health Research Programme (16/09/13) and will be published in full in Public Health Research; Vol. 11, No. 9. See the NIHR Journals Library website for further project information. Plain language summary The Communities in Control study is looking at the health impacts of the Big Local community empowerment programme, funded by the National Lottery Community Fund and managed by Local Trust (a national charitable organisation). Residents of 150 English areas have at least £1M and other support to improve the neighbourhoods. There have been three phases of the research. This report shares findings from their third phase, which began in 2018. First, we used data from a national survey and data from national health and welfare services to compare changes in mental health between people living in Big Local areas and those in similar areas that did not have a Big Local partnership. Furthermore, we also used publicly available data on crime in the neighbourhoods. We found weak evidence that Big Local was linked with improved mental health and a reduction in burglaries. Second, we used data from a survey conducted by Local Trust to look at health and social impacts on the most active residents. We found an increase in mental well-being in 2018 but this was not maintained in 2020, probably due to the COVID-19 pandemic. Third, we did interviews and observations in 14 Big Local areas to understand what helps and what does not help residents to improve their neighbourhoods. We found that partnerships need to have legitimacy, the right balance of support, and learning opportunities. Residents suggested that creating social connections and welcoming social spaces, improving how people view the area and tackling poverty contributed to health improvements. Direct involvement in Big Local was both stressful and rewarding. Finally, we did a cost–benefit analysis by putting a monetary value on residents’ increase in life satisfaction due to Big Local and comparing it with the costs of Big Local. We found that the benefits exceed the costs by at least £60M, suggesting that Big Local provides good value for money. Scientific summary Background This research was commissioned in response to a call in 2016 by the National Institute for Health and Care Research (NIHR) Public Health Research Programme (PHRP) for research ‘to address urgent gaps in the evidence on which interventions, using a community engagement approach, are effective in improving health and well-being and reducing health inequalities.’ We identified that the most glaring gaps were in relation to community empowerment initiatives. In theory these have great potential for reducing health inequalities, but research has focused instead on initiatives adopting a much narrower model of engagement of communities in professional-led interventions. We were able to take advantage of the Big Local (BL) programme across England, a rare example of a nationwide experiment in community empowerment. Big Local is the largest community empowerment initiative implemented in England. Launched in 2010 with £271M from the National Lottery Community Fund, and ending in 2026, it involves giving 150 relatively disadvantaged communities in England control over £1M each to enable residents to improve their neighbourhoods. It is overseen by a national organisation, Local Trust, and managed by a BL partnership board in each area, with at least 51% resident members. Our Communities in Control (CiC) study is a long-term evaluation spanning 7 years, with the latest 3 years funded by PHRP. It offers an unparalleled opportunity to learn from the BL experience for future community empowerment initiatives, focusing on the effects on health and well-being and on reducing health inequalities. Research objectives Building on two earlier phases in 2014–15 and 2015–17, the objectives of this third phase of the CiC study were to: investigate longer-term population-level health and social outcomes of BL investigate impacts of BL on health and well-being of engaged residents assess changes in collective control over decisions and actions among BL residents and pathways to changes identified illuminate residents’ perspectives on health and well-being impacts and pathways to these conduct an economic evaluation of BL draw out implications for the future design and evaluation of place-based initiatives that aim to increase collective control, particularly in disadvantaged communities. Methods The study adopted a systems theoretical framework. It is a longitudinal mixed-methods evaluation comprising four work packages. Work package (WP) 1 focused on Objective 1. Secondary data were analysed to assess whether BL had any positive impacts on social and health outcomes for populations in BL neighbourhoods. It employed difference-in-differences (DiD) methods comparing changes in outcomes in BL areas to those in comparator areas between 2011–15 and 2016–19. Comparator areas were matched to BL areas based on observed characteristics such as the Index of Multiple Deprivation, population ethnic and age profiles from the 2011 census and the Office for National Statistics (ONS) area classification. The primary outcome, from the Annual Population Survey, was a binary variable taking the value 1 if individuals reported a score of more than 6 in response to the question ‘Overall, how anxious did you feel yesterday?’, where 0 is ‘not at all anxious’ and 10 is ‘completely anxious’. Secondary outcomes included a composite measure of population mental health, recorded crimes and antisocial behaviour. Work package 2 addressed Objective 2, assessing whether BL had any positive health impacts for actively engaged residents using the 2016, 2018 and 2020 waves of a biannual survey of BL partnership members conducted by Local Trust. The analysis used both the repeated cross-sectional sample provided by the survey and a nested cohort of partnership members who completed all three surveys. The primary outcome was the Short Warwick–Edinburgh Mental Well-being Scale and the secondary outcome self-rated general health status. Unfortunately, we were unable to construct comparators as planned using the 2016, 2018 and 2020 Health Survey for England, because the relevant questions were not in the 2018 survey and the survey was not undertaken in 2020. Work package 3 addressed Objectives 3 and 4. First, qualitative in-depth interviews with national BL stakeholders and informants with expertise in place-based policies aimed to identify changes in the BL programme over time and explore the relationship with national policy. Second, continuing our longitudinal qualitative research, we conducted 171 interviews in 14 areas with people actively involved with BL from November 2018 to January 2021 (overlapping with the pandemic). Interviews covered perceptions of processes enabling or constraining residents’ attempts to improve their area and the impact of BL on health and well-being. In the analysis, we drew on the data from earlier phases of the CiC study. We also conducted observations of BL partnership meetings. Work package 4 addressed Objective 5 by conducting a cost–benefit analysis using the life satisfaction approach to value the benefits of Big Local. This approach compares the impact of an intervention on life satisfaction to the impact of an increase in income on life satisfaction. It used the DiD estimate of the impact of BL on life satisfaction from WP1, estimates of annual household income in BL areas from the ONS, and average household sizes in the BL areas from the 2011 census to put a monetary value on the benefits generated by BL. It then compared the benefits to the costs, that is, the funding BL areas received and the value of the unpaid hours provided by BL partnership members (estimated from responses to the biannual survey of BL partnership members). Results For Objective 1, we found limited evidence for population-level social or health impacts in BL areas versus comparators. The estimated impact on the primary outcome – reporting high anxiety yesterday – was small and not statistically significant (–0.8 percentage points, 95% CI –2.4 to 0.7). Similarly, the estimated impacts on the secondary outcomes were small and not statistically significant, with the exception of burglary (–0.054 change in z-score, 95% CI –0.100 to –0.009). However, there is evidence that BL had some effect on reducing levels of anxiety after 2017. Furthermore, in BL areas that had spent more than 80% of their grant by 2019/20 and thus had made the most progress implementing their plan, we found a statistically significant reduction in the composite measure of population mental health (–0.053 change in z-score, 95% CI –0.103 to –0.002), indicating an improvement in mental health. This improvement in mental health was greater in BL areas that had spent more than 80% of their grant and prioritised social activities, while the reduction in burglaries was also larger when the analysis was limited to these BL areas and greater still in those in this group that had focused on environmental activities. For Objective 2, the impact of BL on engaged residents varied between the cohort and cross-sectional data and by subgroups. In the nested cohort there was a significant increase in mental well-being in engaged BL residents in 2018, but this was no longer statistically significant by 2020. However, this may be explained by the fact that the 2020 survey was delivered in the summer during the COVID-19 pandemic when the mental well-being of the whole country had declined. The bias created by the COVID-19 pandemic potentially affects both the outcomes and all of the explanatory variables, making interpretation of the 2020 survey data problematic. However, across all waves – even in the 2020 COVID-19 data point – residents who perceived that people in the area are willing to help each other and those who agreed that collectively they can influence decisions in the area had significantly higher mental well-being scores. Hours volunteered also had a small positive association with increased mental well-being in both the nested cohort and the repeated cross-sectional analyses, for both men and women and for groups with both high and lower levels of education. There is some evidence of unequal benefits across subgroups. For the cohort, there was a significant improvement in the mental health score in 2018 and 2020 among those with one or more degrees but not for those with ‘no degree’. In both study designs, collective control and willingness to help in the area had positive associations with mental well-being. In the repeat cross-section, residents in the lower education group who agreed that they had collective control had a higher mental well-being score in 2018 and 2020 than those who did not. In the cohort, the positive association with collective control was present for both educational groups. Together, these results suggest that the health benefits of BL participation – especially for residents with a lower level of education – may depend on whether they felt a sense of collective control. The results for gender were similarly mixed across the study designs. In the cohort, there was no significant difference in mental well-being over time for women, but men had a significant increase in the short term only (2018). In the repeat cross-section, more highly educated women had a small significant increase in mental well-being in 2020. Collective control was positively associated with higher mental well-being scores for women and – especially – men. Feeling that people in the area were willing to help was positively associated with mental well-being for women but not for men. For Objective 3, we utilised qualitative data to explore changes in collective control. Qualitative, empirical markers derived in earlier phases of CiC revealed increasing capabilities for collective control – defined as different types of power – over time. Residents in all fieldwork sites reported growing confidence in their individual and collective ability to improve the area (power within); greater understanding of the need for alliances to deliver improvements, and enhanced skills in forging and sustaining external relationships (power with); and in all these areas, meaningful improvements were delivered (power to act). Using longitudinal data, we also started to identify new qualitative markers which signalled an evolution of collective control capabilities and milestones of this evolution through three programme stages: initial implementation, developmental and bedding in. However, there was variation across BL areas. Local dynamics – internal and external to BL partnerships – could undermine residents’ ability to develop the forms of power needed to exercise collective control and also limit the potential for collective control capabilities to develop in the wider community. Without input from support workers, other professionals and local organisations, achievements in some BL areas would have been compromised. On the other hand, these actors sometimes also behaved in ways that undermined the development and exercise of collective control capabilities by residents. For Objective 4, qualitative data illuminated residents’ perceptions of BL’s impact on health and well-being. Frequent references were made to positive impacts on mental well-being at individual and collective levels. Varied pathways were identified, including improvements in social connectivity and cohesion, the physical environment and material living standards. However, there were also accounts of negative impacts on individuals, most often arising from problematic relationship dynamics and/or the burden of responsibility associated with BL partnerships, particularly with leadership roles. Residents experiencing difficulties frequently ‘stepped back’ from the partnership – sometimes temporarily, sometimes permanently. This response reflects the complex relationships many had with BL. Despite sometimes significant negative impacts, residents often also stressed positive benefits from involvement. As a national participant observed, part of the reason why community initiatives may encounter challenges or at times fail is not because people involved are ‘not thoughtful’, but because ‘being resident led is hard’. For Objective 5, the economic analysis suggests that the benefits of BL exceed its costs. The estimated benefits vary across the 150 BL areas due to variation in household income, household size and the number of residents in each area. There are also variations in the costs across BL areas. In total, the estimated annual benefits for all BL areas are £270M, so over the 4-year period from 2016 to 2019 the total benefits are £1080M. The total costs are £121M, so the net benefits are £959M. Sensitivity analyses suggest that BL’s impact on life satisfaction needs to be much lower than our current estimate before the net benefit becomes negative. Using the current impact estimate combined with the most conservative assumptions for the other elements of the cost–benefit calculations generates a net benefit estimate of £64M. Conclusions Big Local has spanned a decade of unprecedented cuts in public sector finances, restrictions on welfare benefits and latterly the COVID-19 pandemic. Despite this difficult context, our findings suggest that capabilities for collective control among engaged BL residents had increased considerably, and they had exercised this control to deliver demonstrable improvements in their neighbourhoods. However, the story is complex. Increases in capabilities for collective control were not always linear, and progress varied across BL partnerships. There is tentative quantitative evidence that BL improved mental health and reduced burglaries, especially in areas making the most progress in implementing plans. There is also evidence of improvement in mental well-being in resident partnership members between 2016 and 2018, though with some indication that these benefits were not equally distributed. The qualitative findings reinforce the possibility of quantifiable benefits, with residents highlighting positive social and mental health impacts through various pathways. However, there is also evidence of negative impacts on health and well-being. Using the most conservative assumptions, BL provides a 30% rate of return on the original National Lottery grant of £196,873,499. Our findings are particularly timely, with growing calls in England for a new ‘community power paradigm’ drawing on the BL model and the anticipated publication of government proposals for ‘levelling up’ policies that may extend current policies, devolving more decision-making and resources down to local people via place-based empowerment initiatives. Our findings have implications for future community empowerment initiatives. (1) Investment is needed to develop and sustain smaller-scale community associations and organisations to emerge from and work with communities. Careful targeting of this investment is crucial if the benefits of BL-type empowerment initiatives are to be distributed equally within and across communities. (2) Residents should be in the lead in defining and prioritising issues and designing solutions; they should be partners in, but not necessarily leaders of, action. (3) Local agencies should prioritise working as equal partners with communities over enabling communities to act for themselves. (4) Care needs to be taken that community-based initiatives are not held accountable to externally determined governance standards that undermine the development of capabilities for collective control. Our findings also have implications for future research evaluating community empowerment initiatives. (1) Evaluations need to be conducted over a long time frame and include a cohort of community members and comparator areas. (2) Secondary data need to be supplemented with primary data providing bespoke quantitative measures tailored to the programme. (3) Evaluations need to be sufficiently powered to detect small but important effects and to allow subgroup analyses that take account of variations in local programmes. (4) Evaluations must integrate an equity lens. (5) Evaluations should include longitudinal qualitative components. Funding This project was funded by the National Institute for Health and Care Research (NIHR) Public Health Research programme (16/09/13) and will be published in full in Public Health Research; Vol. 11, No. 9. See the NIHR Journals Library website for further project information.

Background: The UK still has the highest rate of teenage births in western Europe. Teenagers are also the age group most likely to experience unplanned pregnancy, with around half of conceptions in those aged

Background: ‘Dating and relationship violence’ is intimate partner violence during adolescence. Among dating adolescents in England, 66–75% of girls and 32–50% of boys report victimisation. Multicomponent school-based interventions might reduce dating and relationship violence. We optimised and piloted Project Respect, a new intervention in secondary schools in England, and study methods, to assess the value of a Phase III randomised controlled trial. Objectives: To optimise Project Respect and to then conduct a pilot randomised controlled trial in southern England, addressing whether or not progression to a Phase III trial is justified in terms of prespecified criteria. To assess which of two dating and relationship violence scales is optimal, to assess response rates and to consider any necessary refinements. Design: Optimisation activities aimed at intervention development and a pilot randomised controlled trial. Setting: Optimisation in four secondary schools across southern England, varying by region and local deprivation. A pilot cluster randomised controlled trial in six other such schools (four intervention schools and two control schools), varying by region, attainment and local deprivation. Participants: School students in years 8–10 at baseline and staff. Interventions: Schools were randomised to the intervention or control arm in a 2 : 1 ratio; intervention comprised staff training, mapping ‘hotspots’ in school for dating and relationship violence, modifying staff patrols, school policy review, informing parents and carers, an application supporting student help-seeking, and a classroom curriculum for students in years 9 and 10 (including student-led campaigns). Main outcome measures: Prespecified criteria for progression to Phase III of the trial, concerning acceptability, feasibility, fidelity and response rates. Primary health outcomes were assessed using the Safe Dates and short Conflicts in Adolescent Dating Relationships Inventory measures collected and analysed by individuals who were masked to allocation. Feasibility of economic analysis was assessed. Data sources: Baseline and follow-up student and staff surveys, interviews, observations and logbooks. Results: The intervention was optimised and approved by the Study Steering Committee. The student response rates in intervention and control groups were 1057 (84.8%) and 369 (76.6%) at baseline, and 1177 (76.8%) and 352 (83.4%) at follow-up, respectively. Safe Dates and the short Conflicts in Adolescent Dating Relationships Inventory had high levels of completion and reliability. At follow-up, prevalence of past-year dating and relationship violence victimisation was around 35% (Safe Dates scale and short Conflicts in Adolescent Dating Relationships Inventory). Staff response rates were very low. Training occurred in all four schools, with suboptimal fidelity. The curriculum was delivered with optimal fidelity in three schools. Other components were delivered inconsistently. Dating and relationship violence was addressed in control schools via violence prevention and responses, but not systematically. Intervention acceptability among students and staff was mixed. An economic evaluation would be feasible. Limitations: One school did not undertake baseline surveys. Staff survey response rates were low and completion of the logbook was patchy. Conclusions: Our findings suggest that progression to a Phase III trial of this intervention is not indicated because of limited fidelity and acceptability. Future work: High prevalence of dating and relationship violence highlights the ongoing need for effective intervention. Potential intervention refinements would include more external support for schools and enhanced curriculum materials. Any future randomised controlled trials could consider having a longer lead-in from randomisation to intervention commencement, using the short Conflicts in Adolescent Dating Relationships Inventory as the primary outcome and not relying on staff surveys. Trial registration: Current Controlled Trials ISRCTN65324176. Funding: This project was funded by the National Institute for Health Research (NIHR) Public Health Research programme and will be published in full in Public Health Research; Vol. 8, No. 5. See the NIHR Journals Library website for further project information.