Search

This report describes efforts by a group of science teachers, teacher educators, researchers, and content specialists to conceptualize, develop, and pilot practice-based assessment items designed to measure elementary science teachers' content knowledge for teaching (CKT). The report documents the framework used to specify the content-specific teaching practices and instructional tools that are critical to the work that elementary science teachers engage in with students, curriculum, and instruction. Drawing on this framework, the report details the development process for practice-based assessment items designed to measure CKT elementary science in three content areas: (a) structure and properties of matter, (b) ecosystems, and (c) Earth's place in the universe. These practice-based assessment items address the various content challenges elementary science teachers face in their work and were designed to be used as a foundation for building large-scale assessments of elementary science teachers' CKT science. This report presents initial validity evidence examining the practice-based CKT science item characteristics using results from online administrations of these items to 250 upper elementary science teachers. Findings reveal that the majority of these new assessment items capture variability in elementary science teachers' performance and that a large proportion of the items differentiate moderately well, supporting a beginning proof-of-concept for the conceptualization and design of these practice-based CKT elementary science assessment items.

This report provides a description of a cognitive interview study investigating validity of assessments designed to measure "content knowledge for teaching" (CKT). The report is intended both to provide information on the validity of the CKT measures and to provide guidance to researchers interested in replicating the design. The study takes an argument-based approach to investigating validity by first articulating interpretive arguments that are central to the CKT measurement theory and then using the cognitive interview data to evaluate these arguments (Kane, 2006). The study is based on 30 interviews of elementary mathematics teachers and 30 interviews of elementary English language arts teachers. Teachers were selected using previous CKT assessment scores to represent high- and low-scoring groups for each subject. The cognitive interviews were conducted separately for each subject and responses were coded and then analyzed to investigate the scoring and extrapolation inferences for the validity argument. Findings strongly support the scoring inference, providing evidence that the item keying for the items is correct. Results also indicate that the participants reasoned about the item in ways that conformed with the reasoning outlined in the "task design rationales" (TDR) for each item. These TDRs represent what reasoning should look like for each of these items for a respondent drawing on the desired CKT knowledge. As such, conformity with the TDRs supports the extrapolation inference, providing evidence that the reasoning used by the participant represents the underlying knowledge and skill domain we intend to measure through CKT assessments. The study design, instruments, methods, and results are described in detail, with discussion included to support researchers interested in replicating or capitalizing on the study design.