Analysis of Two-Tier Question Scoring Methods: A Case Study on the Lawson's Classroom Test of Scientific Reasoning

The Lawson's Classroom Test of Scientific Reasoning (LCTSR) is a popular instrument that measures the development of students' scientific reasoning skills. The instrument has a two-tier question design, which has led to multiple ways of scoring and interpretation. In this research, a method of pattern analysis was proposed and applied to analyze two-tier item pairs on the subskill of Control-of-Variable (COV) of LCTSR. The data were collected from students in grade 4 through college in both the United States and China. Students' response patterns to two combined item pairs were analyzed and compared against students at different grade levels and reasoning development levels. Six performance levels were established based on students' response patterns, serving as indicators of COV reasoning development levels. With the new method, a relation was obtained between students' level of COV skills and grade level, as well as their level of overall reasoning development. It can provide useful information on the possible developmental levels of students' reasoning skills.