Search

The recent turn toward core practices and practice-based teacher education has been accompanied by a growing literature on the definitions, pedagogies to teach, and assessments of core practices. Despite these developments, the field lacks core practices performance assessments designed to be used across course sections, courses, and subjects. This paper provides an existence proof of this type of assessment and investigates the affordances and constraints of the approach. The study describes three types of mixed-reality simulation-based performance tasks of three core practices. More than 400 novices in 64 teacher preparation programs in the United States reported that they were able to use the simulation environment and believed the tasks measure important teaching skills. Scores on the tasks were positively related to novices' prior academic and teacher education experiences. Implications for the formative use of such simulations are discussed.

This report on the second year of data collection in the Understanding Consequential Assessment Systems of Teaching (UCAST) study describes administrators' learning during the first year that Los Angeles Unified School District's (LAUSD) observations were part of a consequential teacher evaluation system for teachers. Drawing on mixed methods, the report provides an overview of the Teacher Growth and Development Cycle (TGDC) implementation during the 2013-2014 school year. It summarizes how much time administrators spent on TGDC-related activities and presents their reflections on the strengths and weaknesses of the TGDC implementation. The report also documents how administrators used the TGDC process with their teachers, how their use of the protocol changed over two school years, and when the observation process did and did not work well. We also review data from two in-depth cases that reflect wider themes in the sample. Three appendices are included: (1) How Was This Report Prepared?; (2) Focus Administrator Surveys; and (3) Focus Administrator Interviews.

Evidence from a handful of large-scale studies suggests that although observers can be trained to score reliably using observation protocols, there are concerns related to initial training and calibration activities designed to keep observers scoring accurately over time (e.g., Bell, et al, 2012; BMGF, 2012). Studies offer little insight into how educational practitioners understand and score observation protocols. This lack of clarity on the factors that facilitate and constrain educators' learning and use of observation systems makes it difficult to implement training and quality control processes at scale. In order to effectively train administrators at scale, it is critical to understand how administrators learn to complete two major tasks--learning to create accurate scores and learning to have conversations around those scores that support instructional improvement. This study takes the perspective that scoring observations of classroom interactions is a complex socio-cognitive process that must be understood in order to improve observer training, and ultimately, score quality. This study suggests administrators have a great deal of knowledge they bring to bear on the observation process. They also bring a commitment to improving instruction to their observation work. In other words, they are not blank slates as they go through observer training. That said, certification data suggest observers have much to learn about how to accurately score lessons according to the protocol. It is unclear whether the knowledge and commitments observers bring is supportive of high quality scores or the improvement of instruction. Future studies should investigate whether the way in which principals use observation protocols results in better, more useful observation scores. Tables are appended.

A variety of factors have converged in the last decade to focus attention on the need for more rigorous designs of professional development (PD). Teacher development is increasingly viewed as one of the primary levers for improving teaching quality and ultimately student achievement. The authors present data from the Teacher Knowledge Assessment System (TKAS), which is designed to administer the Learning Mathematics for Teaching (LMT) measures. TKAS is being widely adopted in the evaluation of PD programs with over 500 separate program administrations and 16,000 teachers representing every major region in the country. The study findings provided potential guidance for the design of studies using teacher knowledge as an outcome measure. Also, the results provide context relevant guidelines (e.g., test outcome, characteristics of teachers, school or program) that researchers may use individually or in combination to design studies and/or assess the practical impact of teacher PD interventions.

The last decade produced numerous studies that show that students learn more from high-quality teachers than they do from lower quality teachers. If instruction is to improve through the use of more rigorous teacher evaluation systems, the implementation of these systems must provide consistent and interpretable information about which aspects of teaching practice need improvement and how those improvements can be accomplished. A primary concern for using observation systems in teacher evaluation is the challenge of training observers to score in valid and reliable ways. In this report the authors will present first-year findings from the "Understanding Consequential Assessment Systems for Teachers" (UCAST) study, which investigates how administrators in a large urban school district learn to use a standardized observation protocol. The relevant research questions are as follows: (1) To what extent do administrator characteristics, beliefs, and expectations predict training satisfaction?; (2) To what extent do administrator characteristics, beliefs, and expectations predict certification success?; and (3) What components of the observation protocol are most challenging for observers to certify, and what accounts for these challenges? This project takes place in the Los Angeles Unified School District (LAUSD). This study is designed to investigate observer thinking and performance as it occurs in practice, and our current analyses draw on quantitative performance and perception data that comes from more than 700 administrators trained by the district during year 1 of the study. To improve the quality of instruction in schools, districts across the country are investing great hopes and resources in the implementation of teacher evaluation systems. Because observation offers a direct measure of instruction and it can point to areas for teacher improvement, additional research on how to implement observation protocols at scale is imperative and will be highly useful to school districts and other education stakeholders. Tables are appended.