Search

In November 2012, the Bill & Melinda Gates Foundation invested in seven innovative district-charter partnerships with "the potential capacity and commitment to accelerate student college ready rates through deep collaboration and sharing of best practices" (District-Charter Collaboration Grant Request for Proposal [RFP]). These partnerships brought together traditional public school districts with individual charter management organizations (CMOs) and local charter schools (and, in some cases, Catholic schools) in Boston, Massachusetts; Denver, Colorado; Hartford, Connecticut; New Orleans, Louisiana; New York, New York; Philadelphia, Pennsylvania; and Spring Branch, Texas. All seven sites received three-year grants intended to (1) facilitate collaboration on evidence-based solutions aligned with the Foundation's College Ready strategy; and (2) improve equity of access, resources, and accountability across district and charter schools. The grants ranged in size from approximately $2 million to $5 million. As part of the District-Charter Collaboration Grant program, the Foundation has contracted with Mathematica Policy Research to conduct an evaluation of its implementation and effects. This final report summarizes the full three-year implementation of the grants from December 2012 to December 2015. One appendix is included: Detail on Leadership Development Participant Selection and Training.

KIPP (Knowledge is Power Program) is a national network of public charter schools whose stated mission is to help underserved students enroll in and graduate from college. Prior studies (see Tuttle et al. 2013) have consistently found that attending a KIPP middle school positively affects student achievement, but few have addressed longer-term outcomes and no rigorous research exists on impacts of KIPP schools at levels other than middle school. In this first high-quality study to rigorously examine the impacts of the network of KIPP public charter schools at all elementary and secondary grade levels, Mathematica found that KIPP schools have positive impacts on student achievement, particularly at the elementary and middle school levels. In addition, the study found positive impacts on student achievement for new entrants to the KIPP network in high school. For students continuing from a KIPP middle school, KIPP high schools' impacts on student achievement are not statistically significant, on average (in comparison to students who did not have the option to attend a KIPP high school and instead attended a mix of other non-KIPP charter, private, and traditional public high schools). Among these continuing students, KIPP high schools have positive impacts on several aspects of college preparation, including more discussions about college, increased likelihood of applying to college, and more advanced coursetaking. This report provides detailed findings and also includes the following appendices: (1) List of KIPP Schools In Network; (2) Detail on Survey Outcomes; (3) Cumulative Middle and High School Results; (4) Detailed Analytic Methods: Elementary School (Lottery-Based Analyses); (5) Detailed Analytic Methods: Middle School (Lottery-Based Analyses); (6) Understanding the Effects of KIPP As It Scales Mathematica Policy Research; (7) Detailed Analytic Methods: Middle School (Matched-Student Analyses); (8) Detailed Analytic Methods: High School (Matched-Student Analyses); (9) Detailed Analytic Methods: High School (Matched-School Analyses); and (10) Detailed Tables For What Works Clearinghouse Review. [For the executive summary, see ED560080; for the focus brief, see ED560043.]

States and school districts are exploring alternatives to state tests for measuring teachers' contributions to student learning. One approach applies statistical value-added methods to alternative student assessments such as commercially available tests and end-of course tests. The evidence suggests that these methods can reliably distinguish among teachers. A second approach requires teachers to develop student learning objectives at the beginning of the school year; these can be used in instructional planning as well as evaluation. Ensuring consistency across teachers and schools is challenging, and implementation is demanding, but student learning objectives have the advantage that they can be implemented in any grade or subject. [This report was prepared for the Institute of Education Sciences (IES) by Regional Educational Laboratory Mid-Atlantic administered by ICF International. For full report, see ED544205.]

States are increasingly interested in including measures of student achievement growth, or "value- added," in evaluating teachers. Annual state assessments, however, which are the typical measure of student growth, usually cover only reading and math teachers and only in grades 4-8. These state assessments thus cannot generally be used to measure contributions to student achievement growth for early elementary school teachers, most high school teachers, and teachers of other subjects. As a consequence, a growing number of states and school districts are exploring alternatives for measuring teachers' contributions to student learning. These alternatives have the potential to be used for evaluating not only teachers who work in grades and subjects outside the annual state testing regime but also as complementary growth measures for teachers of tested grades and subjects. This report reviews the literature on two categories of alternative measures for evaluating teachers: (1) Alternative student outcome measures used in statistical growth (or value- added) models; and (2) Teacher-developed student learning objectives used for measuring growth. It also summarizes the evidence on the use of three types of alternative growth measures: (1) Student assessments other than annual state assessments used in statistical growth models; (2) Nontest student outcomes used in growth models; and (3) Student learning objectives (SLOs). Three appendixes are included: (1) Literature search methodology; (2) Results of the literature search for alternative student outcomes in statistical growth models; and (3) Results of the literature search for student learning objectives. (Contains 3 notes and 16 tables.) [This report was prepared for the Institute of Education Sciences (IES) by Regional Educational Laboratory Mid-Atlantic administered by ICF International. For the report summary, see ED544208.]

At the request of Pittsburgh Public Schools (PPS) and the Pittsburgh Federation of Teachers (PFT), Mathematica is developing value-added models (VAMs) that aim to estimate the contributions of individual teachers, teams of teachers, and schools to the achievement growth of their students. The analyses described in this report are intended as an early step in a multi-year project that aims to produce fair, valid, reliable, and robust estimates of the contributions of individual teachers and schools in Pittsburgh to the achievement growth of their students, ultimately including as many grades, subjects, schools, and teachers as possible. In addition to scores on the Pennsylvania System of School Assessment (PSSA), this report also applies VAMs to Pittsburgh's own curriculum-based assessments (CBAs) in various subjects and courses in grades 6-10; to PSAT assessments conducted in grades 10 and 11; and to Scholastic Reading Inventory (SRI) assessments in grades 6 to 10. Composite value-added measures that incorporate results for each of the assessment outcomes relevant to each school were also created. In addition, school-level VAMs are estimated for the non-test outcomes of student attendance and credit accumulation. This report describes the methods for measuring teacher and school effectiveness using those student outcomes and the findings from the authors' analyses to date. It describes in general terms the distribution of value-added estimates for teachers and schools across the district. The findings in this report reinforce the findings of the August report (Lipscomb, Gill, and Booker, 2010) that "VAM estimates can provide meaningful information about teacher and school performance in Pittsburgh--and they confirm that VAMs can be usefully applied not only to state accountability tests and nationally normed assessments, but also to locally developed assessments and to non-test measures of student outcomes such as attendance and credit accumulation." An appendix contains supplementary tables and figures.

New Leaders for New Schools, a nonprofit organization committed to training school principals, heads the Effective Practices Incentive Community (EPIC), an initiative that offers financial awards to effective educators. Through this initiative, New Leaders offers financial awards to educators in two urban school districts and a consortium of charter schools. New Leaders contracted with Mathematica Policy Research to help design the methods for identifying effective schools and teachers. The approach used for each partner differs, depending on the priorities of the partner and the type of information available to measure school and teacher performance. This report presents the methods used to identify effective schools and teachers for a consortium of over 140 charter schools in 17 states and the District of Columbia during the second year of this project. This year's model differs from that of last year in four respects. First, the authors used a shrinkage method to help ensure that schools (and teachers) with small numbers of students in their model were not overrepresented at the top (and bottom) of the resulting performance measures. Second, they added teacher-level estimates for charter schools. Third, they estimated models using two years of performance data rather than just one. Finally, they changed the treatment of schools that cover multiple grade spans. Last year, those schools had multiple chances for winning awards (one for each grade span). This year, they altered their model so each school had only one chance of getting an award. More details on these changes are presented in the discussion of this year's methods. A section on Technical Details of the Value-Added Model is appended. (Contains 2 tables, and 7 figures, and 26 footnotes.)

New Leaders for New Schools, a nonprofit organization committed to training school principals, heads the Effective Practices Incentive Community (EPIC), an initiative that offers financial awards to effective educators. New Leaders and its partner organizations have received from the U.S. Department of Education tens of millions of dollars in financial support for EPIC. Through this initiative, New Leaders offers financial awards to educators in two urban school districts and a consortium of charter schools. New Leaders contracted with Mathematica Policy Research to help design the methods for identifying effective schools and teachers. This report presents the method used to identify effective schools in the Memphis City Schools (MCS), one of the partner school districts, during the second year of this project. This year's model differs from last year's model in that the authors used a shrinkage estimator to help ensure that schools with small numbers of students in their model were not overrepresented at the top and bottom of the resulting performance measures. A shrinkage estimator is a statistical technique that "shrinks" the school effects toward the average, with greater shrinkage for schools whose results were less precisely estimated--typically smaller schools. Key aspects of the Mathematica model are outlined in this paper, along with a more detailed technical description, found in the appendix. (Contains 1 table, 3 figures and 12 footnotes.)

Title I of the federal Elementary and Secondary Education Act, as reauthorized by the No Child Left Behind Act (NCLB), requires states to establish standards, assessments, and accountability systems to ensure that every child achieves proficiency in reading and mathematics by the year 2014. This report describes exploratory analyses of the effects of components of the No Child Left Behind accountability system on the achievement of students in affected Title I schools. The analyses used school-level and student-level assessment data from two states and three school districts, employing a quasi-experimental regression discontinuity method to examine whether schools that fell short of "adequate yearly progress" (AYP) or were identified for improvement under NCLB showed subsequent improvements in student achievement. The purpose of the analysis was to explore the usefulness of the regression discontinuity method for examining the effects of the NCLB accountability system. This analysis was conducted under the National Longitudinal Study of No Child Left Behind (NLS-NCLB), which is examining the implementation of key NCLB provisions at the district and school levels. This report divides into five parts. Following the Introduction in Part 1, Part 2 contains: Using RD to Examine the Effects of Not Making AYP and Identification for Improvement. Part 3 continues with: School-Level RD Analysis in Two States; and Part 4 presents: RD Analysis of Student-Level Achievement Data in Three Large Districts. Part 5 concludes with: Summary and Implications. Appended are: (1) Supplemental Tables for Subgroups of Students, and (2) Selection of Sites Included in This Report. (Contains 23 exhibits.)

New Leaders for New Schools, a non-profit organization committed to training high-quality urban public school principals, received grants from the U.S. Department of Education in 2006 and 2007 to support the development of innovative teacher compensation strategies. New Leaders is partnering with four urban school districts and a consortium of charter schools to implement the Effective Practices Incentive Community (EPIC), offering two types of financial awards to educators: (1) a reward for principals and instructional staff in schools that are effective in raising student achievement and (2) a financial incentive for teachers who are identified as effective and are willing to document and share their practices. New Leaders contracted with Mathematica Policy Research, Inc. (MPR) to help design methods for identifying effective schools and teachers. The approach used for each partner differs, depending on the priorities of the partner and the type of information available to measure school and teacher performance. This report presents the method used to identify effective schools in the Memphis City Schools (MCS), one of the partner school districts, during the first year of this project. The methodology focuses on a "value-added model" that measures how much a school contributes to test score improvements for their students rather than average test score levels or the percentage of students who meet state proficiency levels. The model "handicaps" both the students' starting point and factors that affect their growth over the year, to account for factors that systematically affect the academic growth of different types of students and initial student performance differences across schools. MPR will work with New Leaders and MCS to revise the model in future years, and to incorporate any additional data that become available. The results of this work were given to New Leaders but are not presented here in order to maintain the confidentiality of the individual schools. The identification of effective teachers will be addressed in later reports. An appendix presents technical details of the value-added model (estimation sample; dosage variables for students who attended multiple schools during the 2006-07 school year; controlling for measurement error; the value-added model; and estimating effects by grade and subject). (Contains 1 footnote and 1 table.) [This report was submitted by Mathematica Policy Research, Inc. to New Leaders for New Schools.]

This report presents findings about the relationship between participation in the Title I school choice and supplemental educational services options and student achievement from the National Longitudinal Study of "No Child Left Behind" (NLS-"NCLB"). A key component of the "No Child Left Behind Act of 2001" ("NCLB") was to provide options to parents whose children had been attending Title I schools identified for improvement, corrective action, or restructuring due to failure to achieve adequate yearly progress toward meeting state standards for two or more years. Under "NCLB," parents have the option of: (1) transferring their children to another school in the district that is not in need of improvement; or (2) enrolling their children in supplemental education services (e.g., tutoring, remediation, or other academic instruction) in addition to instruction provided during the school day. This study used data from nine large, urban school districts to examine the characteristics of students participating in the two options and the resulting impact on student achievement. The study found the following: that participation was highest in elementary grades; that African-American students had the highest participation rates of all racial and ethnic groups; that participating students had lower achievement levels than eligible but nonparticipating students; that students who transferred tended to transfer to higher-achieving, racially balanced schools; and that there was no statistically significant (positive or negative) effect on achievement among students participating in the two options. In sum, although participation rates were not high, the users of the two Title I parental options came from the disadvantaged populations that "NCLB" is intended to target. Appendix A provides a description of the nine-district data set. Appendix B presents the full results of the alternative analyses of the school choice option, comparing achievement gains of current and future choosers. Appendix C presents a meta-analysis of effects of Title I school choice and supplemental educational services. The information in this report was provided through the congressionally mandated National Longitudinal Study of "No Child Left Behind" (NLS-"NCLB"), which was conducted by the RAND Corporation and the American Institutes for Research. (Contains 23 exhibits.) [This report was prepared for Policy and Program Studies Service, Office of Planning, Evaluation and Policy Development, US Department of Education.]

KIPP (Knowledge is Power Program) is a national network of public charter schools whose stated mission is to help underserved students enroll in and graduate from college. Prior studies (see Tuttle et al. 2013) have consistently found that attending a KIPP middle school positively affects student achievement, but few have addressed longer-term outcomes and no rigorous research exists on impacts of KIPP schools at levels other than middle school. In this first high-quality study to rigorously examine the impacts of the network of KIPP public charter schools at all elementary and secondary grade levels, Mathematica found that KIPP schools have positive impacts on student achievement, particularly at the elementary and middle school levels. In addition, the study found positive impacts on student achievement for new entrants to the KIPP network in high school. For students continuing from a KIPP middle school, KIPP high schools' impacts on student achievement are not statistically significant, on average (in comparison to students who did not have the option to attend a KIPP high school and instead attended a mix of other non-KIPP charter, private, and traditional public high schools). Among these continuing students, KIPP high schools have positive impacts on several aspects of college preparation, including more discussions about college, increased likelihood of applying to college, and more advanced coursetaking. This brief presents key findings from the full report. [For the full report, see ED560079; for the executive summary, see ED560080.]

Over the past decade, charter schools have been among the fastest-growing segments of the K-12 education sector in Chicago and across the country. This report addresses several key issues related to charter schools using student-level data provided by Chicago Public Schools. Students leaving traditional public schools for charter schools in Chicago tend to look much like the peers they left behind, in both demographic characteristics and student achievement. Transfers to charter schools tend to slightly reduce racial stratification across the schools. Achievement trajectories suggest that, on average, charter schools' performance in raising student achievement is approximately on par with traditional public schools--except that charter schools do not do well in raising student achievement in their first year of operation. Chicago's charter high schools may produce substantial positive effects on ACT scores, the probability of graduating, and the probability of enrolling in college--but these positive effects are solidly evident only in the multi-grade charter high schools (those that include middle-school grades). The large, positive attainment results in Chicago suggest remarkable promise for (at least) multi-grade charter high schools and demonstrate that evaluations limited to test scores may fail to capture important benefits of charter schools. If charter schools (or other multi-grade high schools) have positive effects on graduation and college entry, they may make a substantial, long-term difference in the life prospects of their students. (Contains 11 footnotes and 14 tables.) [Additional support for the research describe in this report was provided by the Stranahan Foundation. This report supersedes ED501834.]

Over the past decade, charter schools have been among the fastest-growing segments of the K-12 education sector in Chicago and across the country. In this report, the authors examine the following issues related to charter schools (1) Are charter schools attracting high- or low-achieving students and do transfers affect the racial mix in the city's public schools: (2) Are charter schools producing achievement gains for the students who attend them, relative to district-run schools in CPS, as measured by state test scores? And (3) Are Chicago's charter high schools (HSs) increasing their students' likelihood of graduating, their scores on the ACT[R] college-admission exam, and their probability of enrolling in college relative to district-run CPS schools? Information provided uses student-level data provided by Chicago Public Schools (CPS). Major findings reported include: (1) On average, the prior achievement levels of students transferring to charter schools differ only slightly from the citywide average and from the achievement levels of peers in the district-managed CPS schools they exited; (2) Transfers from traditional public schools to charter schools in Chicago do not increase racial stratification across the schools; (3) Consistent with similar studies in other locations, only small differences in average achievement gains were found between charter schools and CPS schools, and these differences do not point in consistent directions (the only strong finding regarding achievement is that charter schools do not do well in raising student achievement in their first year of operation); (4) Evidence was found that Chicago's charter HSs may produce positive effects on ACT scores, the probability of graduating, and the probability of enrolling in college, but these positive effects are solidly evident only in the charter HSs that also included middle school grades. Further research is recommended before it can be definitively stated that charter HSs produce positive effects in conventional 9-12 grade configurations, and conversely, whether district-run schools can produce positive effects by incorporating middle school and perhaps elementary grades. Supporting data is appended. (Contains 14 tables.) [Additional support for this report was provided by the Stranahan Foundation. For related Research Brief, see ED501763. This report has been superseded by ED504521.]

Although Missouri has had a career ladder program for teachers since 1987, there has been little research examining the program's effects. This paper examines the program's effect on student achievement using longitudinal data on district math and reading scores for 524 Missouri school districts over a nine-year period. Our primary specification compares achievement levels in participating districts with a matched group of non-participating districts. We also examine alternative specifications, including controlling for prior district scores and using variation in district participation over time to look at within-district effects. Across the range of specifications, the estimated effects range from small positive effects to no effect in both subjects. We conclude that if the Career Ladder has a positive impact on test scores, it is probably very small. [This research brief describes work published by the National Center on Performance Incentives in "Missouri's Teacher Career Ladder Program" by Kevin Booker and Steven Glazerman, Working Paper 2008-15.]

This research brief summarizes an analysis of the two educational options provided under the No Child Left Behind Act (NCLB) to students in Title I schools that are identified for improvement: school choice and supplemental educational services. Among the key findings: (1) The students participating in supplemental educational services generally improved their performance in both reading and math. (2) For students who took advantage of the school choice option and moved to higher-achieving schools, there was no observable achievement gain as a result of changing schools. (3) Overall, those who took advantage of school choice and supplemental services generally came from the disadvantaged populations that NCLB is intended to target.

Chicago's "multi-grade" charter high schools (those that include middle school grades) appear to be increasing students' rates of graduation and college entry. The overall performance of Chicago's charter schools in raising student test scores is approximately on par with that of traditional public schools in Chicago. Charter schools in Chicago are not having major effects on the sorting of students by race, ethnicity, or achievement. [This research brief describes work documented in "Achievement and Attainment in Chicago Charter Schools" by Kevin Booker, Brian Gill, Ron Zimmer, and Timothy R. Sass, 2008.]