The Futility of Propensity Score Methods in a Statewide Study of International Baccalaureate (IB)

With the goal of increasing students' academic readiness for college, high schools in the United States are increasingly offering "credit-based transition programs," including International Baccalaureate (IB), Advanced Placement (AP), and dual enrollment. Existing research points to the promise of IB, AP, and other credit-based transition programs for improving students' academic readiness for college; however, conclusions about program effects are often limited by potential issues of selection bias. Despite strong statistical controls and assumptions to address selection, observational research may not be able to determine whether differences in outcomes are caused by program participation or are simply an artifact of the unmeasured characteristics of schools, students, and families that correlate with the decision to participate in these optional programs. To address this knowledge gap and inform future studies of the impacts of IB and other credit-based transition programs, this paper makes three contributions: (1) A review of existing literature is used to produce an empirically-based conceptual model of selection into IB; (2) The conceptual model is used to identify the characteristics of students and schools that participate in the IB Diploma Programme using data from the National Center for Education Statistics and the Florida Education Data Warehouse. The conceptual model also allows researchers to identify key predictors for which there are no data available; and (3) A test of the ability of the available data to adjust for observed selection bias using propensity score methods, with the degree of bias reduction reported for each predictor is described. The data used in this study come from the Florida K-20 Education Data Warehouse (FL-EDW) and the U.S. Department of Education's Common Core of Data (CCD). The subset of data from FL-EDW has student-level records for 20,373 students who participated in an IB Diploma Programme and graduated between 2002 and 2007, and student-level records for 86,008 randomly sampled students who did not participate in an IB Diploma Programme and graduated over the same time period. This study reveals that, when looking at the statewide population in Florida, the selection bias associated with voluntary participation in IB is very large, and that mechanisms for dealing with selection bias using propensity scores may not be sufficient. Tables and figures are appended.