Search

We propose a novel estimator for use in a fuzzy regression discontinuity setting. The estimator can be thought of as extrapolating the traditional fuzzy regression discontinuity estimate or as an observational study that adjusts for endogenous selection into treatment using information at the discontinuity. We show that it can be motivated as being the least complex model consistent with the data or as an estimator that is preferable to both a traditional regression discontinuity design and an observational study. We further show theoretically that no other estimators consistently generate better estimates than our proposed estimator. We then use this approach to examine the effects of early grade retention beyond the compliers around the retention cutoff. We show that the benefits of early grade retention policies are larger for students with lower baseline achievement and smaller for low-performing students who are exempt from retention. These findings imply that: (1) the benefits of early grade retention policies are larger than have been estimated using traditional fuzzy regression discontinuity designs; and (2) retaining additional students would have a limited effect on student outcomes.

Teachers, like the students they serve, never stop learning. In-service teacher professional development (PD) gives educators opportunities to learn more about pedagogy and improve their own instruction methods to boost students' academic and social and emotional outcomes. Districts make a significant financial investment to provide teacher PD, and research on the impact of teacher PD on teacher instructional practices and student education outcomes has been mixed. In addition, there are only a few studies that examine the impact of teacher PD using rigorous empirical evaluation designs (such as randomized control trials) and consider PD across multiple contexts (public versus charter schools). The authors evaluated the Chicago Collaborative, a teacher PD program that is aligned to Common Core Standards and implemented by Leading Educators, a national nonprofit organization that partners with districts and charter management organizations to help teachers develop the leadership skills that they need to successfully transition from leading students to leading their peers. The authors conducted a randomized control trial evaluation using data from 40 schools across three school districts in the Chicago area during the 2018-2019 and 2019-2020 school years. They examined how the Chicago Collaborative program was implemented and whether the program impacted student achievement. The authors found that the Chicago Collaborative was successfully delivered, despite the challenges posed by the coronavirus pandemic at the end of the research period in 2020. The authors also found robust evidence that the Chicago Collaborative increased student test scores. [This research report was published by the RAND Corporation (ED621808).]

Most states' teacher evaluation systems have changed substantially in the past decade. New evaluation systems typically require school leaders to observe teachers' classrooms two to three times a school year instead of once (Doherty & Jacobs, 2015). The feedback that school leaders provide to teachers after these observations is a key but understudied step in the teacher evaluation cycle. The feedback and subsequent professional development are intended to help teachers change their instructional practices and improve student achievement (Correnti & Rowan, 2007; DeNisi & Sonesh, 2011; Taylor & Tyler, 2012). However, little is known about the feedback that school leaders provide to teachers following classroom observations or about how to train leaders to make that feedback more effective. This study examined the impact of disseminating a detailed checklist intended to structure an effective feedback conference between a school leader and a teacher following a classroom observation. The feedback conference checklist is a modified version of one created by the Carnegie Foundation for the Advancement of Teaching (Tang & Chow, 2007). The checklist, along with short testimonial videos, was a low-cost, low-intensity intervention provided to a randomly selected half of 339 participating New Mexico principals in fall 2015 by the study team. These principals' schools constituted the treatment group. Principals in the treatment group schools received an email with an attachment containing a guide and a 24-item feedback conference checklist, plus a hyperlink to a three-minute testimonial video featuring a principal. Principals were encouraged to distribute the checklist to other school leaders and to use the checklist in all their feedback conferences in the 2015/16 school year. Principals were also asked to distribute the checklist to all their teachers in order to promote greater teacher participation in the feedback conference. The study team also emailed the same checklist plus a hyperlink to a three-minute testimonial video featuring a teacher to up to 10 randomly sampled teachers in each treatment group school. The other half of the principals in the study schools formed the control group. Each of the control group principals received a two-page principal guide as an email attachment in fall 2015. The two-page guide reprised the five tips about feedback included in the summer 2015 New Mexico Public Education Department--sponsored professional development for principals and informed principals about the study. In addition, the study team sent up to 10 randomly sampled teachers in each control group school a two-page teacher guide summarizing the teacher evaluation system (Skandera, 2013) and teachers' right to receive post-observation feedback. All principals and teachers in both the treatment group and the control group who consented to be in the study were asked to complete an online survey (one for principals, another for teachers) in spring 2015 and again in spring 2016. The main outcomes of the study were principals' and teachers' reports of the impacts of the checklist and testimonial video on the perceived quality of feedback conferences following formal classroom observations; principals' recommendations for and teachers' take-up of professional development; and the quality of teachers' subsequent instructional practices as measured by principals' formal classroom observation scores and teachers' self-reported scores. Additional exploratory outcomes included the impact of the checklist on student achievement (school-average math and English language arts scores on the Partnership for Assessment of Readiness for College and Careers assessment) and school report card grades (reported as an A, B, C, D, or F of multiple measures of a school's student achievement) compiled annually by the New Mexico Public Education Department. The study also documented how many recipients reported using the checklist and what they thought about it. The checklist had few clear impacts on the quality of feedback, professional development outcomes, instructional practice, or student achievement. There were two exceptions: teachers who received the checklist reported that their principals were less likely to dominate the feedback conferences, and they reported that they were more likely to follow their principal's professional development recommendations. Use of the checklist in the treatment group was moderate: 77 percent of principals surveyed who received the checklist reported viewing it, and 58 percent said they used it with one or more teachers. At the same time, 29 percent of control group principals (who were not emailed the checklist) reported that they had seen the checklist, and 10 percent reported using it with one or more teachers. The relatively moderate use of the checklist by treatment group principals, combined with the reports by some control group school leaders that they were using it, implies that the estimated impacts of using the checklist would be larger than the estimated impacts of receiving it. Though distribution of the feedback conference checklist to principals and teachers had a few modest impacts, this study indicates that distributing the checklist is unlikely by itself to substantially alter feedback conferences, teachers' classroom practices, or student achievement, at least during the first school year in which the checklist is used. This study suggests that only a fraction of school leaders are likely to use the checklist if it is distributed in the low-cost manner followed in this study. But the checklist may also have failed to help principals overcome common barriers to effective feedback, such as providing critical comments to teachers or recommending appropriate professional development. The study results raise the possibility that additional (or different) investments might be necessary to improve school leaders' feedback conferences with teachers--for example, pairing training with written guidance.

Researchers often include covariates when they analyze the results of randomized controlled trials (RCTs), valuing the increased precision of the estimates over the potential of inducing small-sample bias when doing so. In this paper, we develop a sufficient condition which ensures that the inclusion of covariates does not induce small-sample bias in the effect estimates. Using this result as a building block, we develop a novel approach that uses machine learning techniques to reduce the variance of the average treatment effect estimates while guaranteeing that the effect estimates remain unbiased. The framework also highlights how researchers can use data from outside the study sample to improve the precision of the treatment effect estimate by using the auxiliary data to better model the relationship between the covariates and the outcomes. We conclude with a simulation, which highlights the value of using the proposed approach.

With the launch of the New York City Community Schools Initiative (NYC-CS) in 2014, the New York City Department of Education (NYCDOE) has increased its focus on the implementation of a holistic strategy of education reform to address the social consequences of poverty as a means to improving student outcomes. NYC-CS is a strategy to organize resources in schools and share leadership among stakeholders so that academics, health and wellness, youth development, and family engagement are integrated into the fabric of each school. New York City is implementing this strategy at a scale unmatched nationally. In this study, the authors assessed the impact of the NYC-CS through the 2017-2018 school year. The authors assessed the effects along seven outcome domains and explored the extent to which there is heterogeneity in programmatic impact based on student- and school-level characteristics. The authors leveraged innovative quasi-experimental methodology to determine whether students in the community schools are performing better than they would be had their schools not been designated as Community Schools. The findings of this report will contribute to the emerging evidence base on the efficacy of the community school strategy and will be useful for other school district- and state-level policymakers interested in developing or refining similar interventions that support students' and communities' academic, social, and emotional well-being. [The research described in this report was prepared for the New York City Mayor's Office for Economic Opportunity (NYC Opportunity).]

These appendixes provide supplemental detail about the analyses and findings presented in Preparing School Leaders for Success: Evaluation of New Leaders' Aspiring Principals Program, 20120-2017 (Gates et al., 2019). This material will be of interest to technically oriented readers who seek more information about the information presented in the main report. Appendix A provides profiles of New Leaders partner districts. Appendix B contains detailed information on our analysis of school-level outcomes. Appendix C presents technical details about our analysis of student achievement outcomes using student fixed-effects models. Appendix D describes our analysis of principal retention. Appendix E provides details about our analysis of correlations between Aspiring Principals program competency metrics and outcomes. [For the full report, see ED605722.]

Having good teachers in the classroom matters, and in recent years better data have helped researchers more accurately measure individual teachers' effects on their pupils. A novel RAND study goes further, finding that teachers' effectiveness (or lack thereof) ripples out beyond their own students and affects the educational experiences of their pupils' future peers. The research also found that ignoring a teacher's effect on his or her students' future peers can underestimate a good teacher's value by anywhere from 30 to 90 percent, depending on which assumptions are made about the dynamic nature of the comparisons and the precise definition of the peer group. To better understand how a teacher's impact may extend beyond the students in her classroom, a new RAND study, "Does Helping John Help Sue? Evidence of Spillovers in Education," highlights the potential spillover effects of a good teacher by separating out the effect of a student's peers' test scores from the effect of other aspects of peers' backgrounds. This research brief summarizes findings from this study and suggests future research questions that could further broaden the understanding of how good teachers can indirectly have a positive influence on their students' future peers.