Search

In most U.S. schools, teachers are evaluated using observation of teaching practice (OTP). This study investigates rater effects on OTP ratings among 421 principals in an authentic teacher evaluation system. Many-facet Rasch analysis (MFR) using a block of shared ratings revealed that principals generally (a) differentiated between more and less effective teachers, (b) rated their teachers with leniency (i.e., overused higher rating categories), and (c) differentiated between teaching practices (e.g., "Cognitive Engagement" vs. "Classroom Management") with minimal halo effect. Individual principals varied significantly in degree of leniency, and approximately 12% of principals exhibited severe rater bias. Implications for use of OTP ratings for evaluating teachers' effectiveness are discussed. Strengths and limitations of MFR to analyze rater effects in OTP are also discussed.

Teacher evaluation systems commonly rely on observation of teaching practice (OTP) by school principals. However, the value of OTP as evidence of teacher effectiveness depends on its psychometric quality. In this study, we address a key aspect of the psychometric quality of principals' OTP ratings. Specifically, we investigate the degree to which rating scale categories have a consistent interpretation across teaching episodes and practices. Results suggest that the 1,324 principals' use of the rating scale categories functioned as intended overall. However, we also found that the midpoint category is underutilized and that rating categories do not always reflect similar levels of teaching effectiveness across teaching episodes and practices. When such discrepancies occur, we cannot assume principals' ratings reflect a consistent level of teacher effectiveness within and across classrooms. This is a critical component of validity evidence that can inform the interpretation of OTP ratings and point to areas for improvement in both the rubrics and in principals' training for classroom observations.

The authors argue that the prospect of performance can motivate students to learn. He uses a study of the Above and Beyond program at a K-5 school, where students appeared to be more interested in their subjects and learn more effectively when given the opportunity to display or perform in front of an audience.

Research on factors that may promote engagement is hampered by the absence of a measure of classroom-level engagement. Literature has suggested that engagement may have 3 dimensions--affective, behavioral, and cognitive. No existing engagement scales measure all 3 dimensions at the classroom level. The Classroom Engagement Inventory (CEI) was developed to fill this gap. In Study 1, exploratory and confirmatory factor analyses were conducted on data from 3,481 students from the 4th to 12th grade. The results suggested a 4-factor model of the CEI. Using these results, in Study 2 several items were revised and data were collected 1 year later from 4th to 12th grade students in the same school district as Study 1. Analyses were conducted on data from 3,560 students after data cleaning. A series of potential models was tested. The final results suggest a 5-factor 24-item CEI: (1) Affective Engagement, (2) Behavioral Engagement-Compliance, (3) Behavioral Engagement-Effortful Class Participation, (4) Cognitive Engagement, and (5) Disengagement. Results advance understanding of the construct of classroom engagement. The CEI fills a significant gap in measurement of engagement. The CEI is classroom level, measures multiple dimensions of engagement, uses self-report, is relatively short, and can be readily administered in classrooms from the 4th to 12th grade., (PsycINFO Database Record (c) 2014 APA, all rights reserved.)