Search

These are the appendices for the report "Study of Teacher Coaching Based on Classroom Videos: Impacts on Student Achievement and Teachers' Practices. Evaluation Report." This report examined a promising strategy for individualized coaching: professional coaches--rather than district or school staff--providing feedback to teachers based on videos of their instruction. Feedback based on videos gives teachers the opportunity to observe and reflect on their own teaching and allows coaches to show teachers specific moments from their teaching when providing feedback. For this study, 107 elementary schools were randomly divided into three groups: one that received fewer highly structured cycles of focused professional coaching during a single school year (five cycles), one that received more (eight cycles), and one that continued with its usual strategies for supporting teachers. The study compared teachers' experiences and student achievement across the three groups to determine the effectiveness of the two versions of the coaching. This document provides additional details on the coaching provided for the study, the approach to carrying out the study, and the findings presented in the report. The following three appendices are included: (1) The Study's Video-Base Coaching for Teachers; (2) Study Design, Data Collection, and Analytic Methods; and (3) Supplemental Exhibits and Information on Study Findings. [For the full report, see ED619739. For the Study Highlights, see ED619740.]

Helping teachers become more effective in the classroom is a high priority for educators and policymakers. A growing body of evidence suggests that individualized coaching focused on general teaching practices can improve teachers' instruction and student achievement. However, little is known about the benefits of specific approaches to coaching, including who is doing the coaching, how coaches observe teachers' instruction, and how or how often coaches provide feedback to teachers. This study examined one promising strategy for individualized coaching: professional coaches--rather than district or school staff--providing feedback to teachers based on videos of their instruction. Feedback based on videos gives teachers the opportunity to observe and reflect on their own teaching and allows coaches to show teachers specific moments from their teaching when providing feedback. For this study, about 100 elementary schools were randomly divided into three groups: one that received fewer highly structured cycles of focused professional coaching during a single school year (five cycles), one that received more (eight cycles), and one that continued with its usual strategies for supporting teachers. The study compared teachers' experiences and student achievement across the three groups to determine the effectiveness of the two versions of the coaching. Key findings include: (1) Five coaching cycles based on videos of teachers' instruction improved students' achievement, including for novice teachers and those with weaker classroom practices at the start of the study; (2) Eight cycles of coaching was not effective. Eight cycles of the coaching did not affect student achievement, perhaps because teachers had less time during each cycle to work on the practices being addressed; and (3) The study's coaching changed the type of feedback that teachers received. Compared to those who did not receive the study's coaching, teachers who received the coaching were more likely to report receiving feedback that focused on specific teaching practices, included strategies to use in their classrooms, and provided opportunities to observe and reflect on their teaching. [For the Study Highlights, see ED619740. For the appendix, see ED619742.]

The Wallace Foundation's Principal Supervisor Initiative (PSI) was a four-year effort to redefine principal supervision in six urban school districts. Launched in 2014, the PSI aimed to help these districts transform the principal supervisor position from a role that traditionally focused on managerial tasks to one dedicated to developing and supporting principals to be effective instructional leaders. The PSI encouraged these districts to revise the principal supervisors' job description to focus on instructional leadership, reduce principal supervisors' span of control (the number of principals they oversee), train supervisors and develop their capacity to support principals, create systems to identify and train new supervisors, and strengthen central office structures to support and sustain changes in the principal supervisor's role. This report presents findings from a 2018 survey of principal supervisors in 54 urban school districts, including the 6 PSI districts. The districts were all members of the Council of the Great City Schools, a coalition of the nation's largest urban public school systems. In total, 343 principal supervisors responded to the survey, including 50 in the 6 PSI districts (a 96 percent response rate) and 293 in the 48 other urban districts (a 64 percent response rate). This report compares principal supervision in the PSI districts with that in other urban districts. It compares: (1) the "structures" districts put in place to support the principal supervisor role (including principal supervisors' span of control and districts' approaches to selecting and training supervisors); and (2) principal supervisors' "professional practices." This comparison can provide greater context for the changes that the PSI districts made during the initiative, highlighting areas where the PSI districts' work aligned with broader national trends and areas where it surpassed or lagged these trends. It can also provide a roadmap for other districts seeking to strengthen the principal supervisor role to better support principals as instructional leaders, by highlighting aspects of principal supervision that districts and states may find easier to change and aspects that may require more deliberate action and focus.

In 2014, The Wallace Foundation launched the Principal Supervisor Initiative (PSI), a four-year, $24 million effort to redefine principal supervision in six urban school districts. The PSI aimed to help districts overhaul a position traditionally focused on administration, operations, and compliance to one dedicated to developing and supporting principals to be effective instructional leaders in their schools. Principal supervisors are a potential point of leverage for supporting and developing principals' effectiveness, but little is known about the effectiveness of this approach. The overarching hypothesis of the PSI was that changing the role of principal supervisors from overseeing operations to developing principals' instructional leadership practices could drive improvement in principal effectiveness. This report describes districts' experiences with the PSI across the years of the initiative and analyzes its effects on teachers' perceptions of principals' performance. Specifically, the report addresses the following questions: (1) What is the theory of change for revising the principal supervisor's role?; (2) What were districts' and principal supervisors' experiences with the PSI?; (3) What were principals' experiences with the PSI?; (4) How did the initiative affect teachers' perceptions of principals' performance, and how did teachers' perceptions of principals' performance and the effects of the PSI vary across districts and principal supervisors?; (5) How do key components of the PSI relate to changes in teachers' perceptions of principals' performance?; and (6) How did the principal supervisor role in PSI districts compare with that in other urban districts? The report also discusses the sustainability of changes made during the PSI and lessons learned for revising the principal supervisor role. The analyses of data from site visits, interviews, and surveys focused on districts', supervisors', and principals' experiences, successes, and challenges throughout the initiative; lessons learned; and plans for sustaining the accomplishments of the PSI after the initiative ended. The authors integrated the descriptive survey results with the qualitative data. The PSI brought substantial changes to all six districts. The initiative was successful in changing the principal supervisor role to one that focused on providing support and guidance to schools and principals. Principals' perceptions of their supervisors' effectiveness improved over the course of the initiative. Districts restructured their central offices to better support schools and meet their needs. Changes districts made during the PSI led to a shared understanding of the principal supervisor role and equipped principal supervisors to better support principals. However, the PSI did not improve teachers' perceptions of principals' performance, on average.

Helping principals improve their leadership practices is a common use of federal funds and one way to improve instruction and student achievement. This study sought to better understand the effectiveness of an intensive principal professional development program focused primarily on helping principals conduct structured observations of teachers' classroom instruction and provide targeted feedback based on those observations. The report found that the program did not change principal practices as intended or improve students' achievement. However, improvements in some of the targeted principal practices were positively associated with improved student achievement. Attention on how to change these practices might be a potential avenue for improving the program. [For the one page highlights to this study, "The Effects of a Principal Professional Development Program Focused on Instructional Leadership. Study Highlights. NCEE 2020-0002," see ED599490. For the appendices to this study, "The Effects of a Principal Professional Development Program Focused on Instructional Leadership: Appendices. NCEE 2020-0002," see ED599491.]

The four appendices in this report accompany the full report "The Effects of a Principal Professional Development Program Focused on Instructional Leadership. NCEE 2020-0002," (ED599489). They include: (1) The Study's Principal Professional Development Program; (2) Study Design, Data Collection, and Analytic Methods; (3) Supplemental Tables; and (4) Interpreting Study Findings: Alternatives to P-Values and Statistical Significance. [For the study highlights to this report, "The Effects of a Principal Professional Development Program Focused on Instructional Leadership. Study Highlights. NCEE 2020-0002," see ED599490.]

In 2014, The Wallace Foundation launched the Principal Supervisor Initiative (PSI), a four-year, $24 million-dollar effort to redefine principal supervision in six urban school districts. The initiative sought to help districts transform a position traditionally focused on administration, operations, and compliance to one dedicated to developing and supporting principals to improve instruction in schools. The initiative was motivated by an effort to increase student learning and achievement by improving principal effectiveness. Research has shown that strong principals are integral to strong schools and to raising the quality of teaching. Numerous studies have pointed to the importance of effective leaders for teacher satisfaction, teacher retention, school climate, parent engagement, and student achievement. Principal supervisors are a potential point of leverage for supporting and developing principals, but relatively few districts have invested in such efforts. The motivating hypothesis of the PSI is that changing the role of principal supervisors from overseeing administration and operations to providing instructional leadership can drive improvement in principal effectiveness. The report presents analyses of data from semistructured interviews with central office personnel, principal supervisors, and principals, as well as data from surveys of supervisors and principals in each of the six PSI districts throughout the United States.