Intact School Matching in Education: Exploring the Relative Importance of Focal and Local Matching

The nested data structure inherent in education (i.e. students nested in schools nested in districts) makes intact school matching an appealing approach in observational studies of educational interventions and policies for both theoretical and practical purposes. This paper provides guidance to applied education researchers who are employing intact school matching. First, using a within study comparison (WSC) methodology the authors examine whether intact school matching is able to replicate experimental results. In addition, they examine what approaches to intact school matching lead to the greatest level of bias reduction. Specifically, the authors estimate the ability to reproduce Randomized Control Trial (RCT) results when comparison units are matched. The study draws on data from two RCTs conducted in education and data on other schools and students in the states in which the RCTs were conducted. The RCTs were conducted in Tennessee and Indiana and include schools and students from across the state in both cases. Data from schools (and students attending those schools) that participated in the two RCTs were used to calculate the experimental benchmark. In the case of the Indiana RCT, the quasi-experimental comparison group was drawn from all other schools in the state that did not participate in the RCT. In the case of the Tennessee RCT, the comparison group was constructed from within the RCT using a synthetic WSC design. This paper provides preliminary evidence that intact school matching can replicate experimental estimates and provides some guidance for which approaches are more likely to effectively reduce bias. Tables are appended.