Search

Objectives: To present results from an outcome evaluation of the Henry J. Kaiser Family Foundation's Community Health Promotion Grants Program (CHPGP) in the West, which represented a major community-based initiative designed to promote improved health by changing community norms, environmental conditions, and individual behavior in 11 western communities., Methods: The evaluation design: 14 randomly assigned intervention and control communities, 4 intervention communities selected on special merit, and 4 matched controls. Data for the outcome evaluation were obtained from surveys, administered every two years at three points in time, of community leaders and representative adults and adolescents, and from specially designed surveys of grocery stores. Outcomes for each of the 11 intervention communities were compared with outcomes in control communities., Results: With the exception of two intervention communities-a largely Hispanic community and a Native American reservation-we found little evidence of positive changes in the outcomes targeted by the 11 intervention communities. The programs that demonstrated positive outcomes targeted dietary behavior and adolescent substance abuse., Conclusions: Improvement of health through community-based interventions remains a critical public health challenge. The CHPGP, like other prominent community-based initiatives, generally failed to produce measurable changes in the targeted health outcomes. Efforts should focus on developing theories and methods that can improve the design and evaluation of community-based interventions.

The Community Health Promotion Grants Program, sponsored by the Henry J. Kaiser Family Foundation, represents a major health initiative that established 11 community health promotion projects. Successful implementation was characterized by several critical factors: (1) intervention activities; (2) community activation; (3) success in obtaining external funding; and (4) institutionalization. Analysis of the program was based on data from several sources: program reports, key informant surveys, and a community coalition survey. Results indicate that school-based programs focusing on adolescent health problems were the most successful in reaching the populations they were targeting. The majority of the programs were able to attract external funding, thereby adding to their initial resource base. The programs were less successful in generating health promotion activities and in achieving meaningful institutionalization in their communities.

Currently, there is considerable interest in studies that use the community as the experimental unit. Health promotion programmes are one example. Because such activities are expensive, the number of experimental units (communities) is usually very small. Investigators often match communities on demographic variables in order to improve the power of their studies. Matching is known to improve power in certain circumstances. However, we show here that if the number of communities is small, the matched design will probably have less power than the unmatched design. This is due primarily to the loss of degrees of freedom in the matched design, which outweighs the benefits of matching on any but the strongest correlates of changes in behaviour. In the community intervention situation, even small differences in sample size between the matched and unmatched analyses can have expensive consequences.

The growing interest in community-based approaches to health promotion and disease prevention (HP/DP) has been accompanied by a growing need to evaluate the effectiveness of such programs. Special issues that arise in these evaluation studies include (1) entire communities are assigned to intervention and control groups, (2) only a small number of communities can usually be studied, (3) the time course of changes in behavior and other outcomes is often of interest, and (4) surveys to measure such changes over time can be conducted with either repeated cross-sectional samples or with longitudinal samples. This paper shows how these issues can be addressed under a mixed-model analysis of variance approach. This approach serves to unify several ideas in the literature on evaluation of community studies, including use of time-series regression and the question of whether the individual or the community should be the unit of analysis. We also describe how the method can be used to estimate sample size requirements, statistical power, or minimum detectable program effect.