Sources of Variance in Special Educator Observation Rubrics

This study describes the development and initial psychometric evaluation of a Recognizing Effective Special Education Teachers (RESET) teacher observation instrument. Specifically, the study uses generalizability theory to compare two versions of a rubric, one with general descriptors of performance levels and one with item-specific descriptors of performance levels, to evaluate special education teacher implementation of explicit instruction. Eight raters participated in viewing and scoring videos of special education instruction. Data collected from raters were analyzed in a three facet, crossed, mixed-model design to estimate the variance components and reliability indices. Results show lower unwanted sources of variance and higher indices of reliability with the rubric with item-specific descriptors of performance levels. Contributions to the field of teacher evaluation are discussed. [This paper will be published in "Assessment for Effective Intervention" under the title "Variance and Reliability in Special Educator Observation Rubrics."]