Search

In test equating, one critical equating property is the group invariance property which indicates that the equating function used to convert performance on each alternate form to the reporting scale should be the same for various subgroups. To mitigate the impact of disrupted learning on the item parameters during the COVID-19 pandemic, a pre-equated approach was frequently implemented for student scoring in the spring 2021 administration of large-scale assessments. This study examines how much the group invariance property was satisfied for various subgroups when pre-equating was applied to large-scale assessments in spring 2021. While most studies for invariance property have been for post-equating, this study investigated the invariance property with pre-equating. Due to the potential impact on live calibration and equating, many large-scale assessment programs have opted to reuse previously administered forms and applied pre-equating rather than developing new forms and post-equating in 2021. This study included grades 6 and 8 ELA and Math from a spring 2021 large-scale assessment program and examined the invariance property for four subgroups: gender, SES, ethnicity, and LEP. Three different scoring methods were studied: unrounded equated scores, rounded equated scores, and MLE as IRT pattern scoring. As evaluation criteria, RMSD and REMSD indices were included (Dorans & Holland, 2000). The effect size was also used for evaluating the equating function differences between each subgroup's categories and the population. Invariance property is often studied when post-equating is performed with alternate forms under common (i.e., anchor) item design. In this case, the quality and number of anchor items between alternate forms influence the equating results and invariance property. In general, if anchors are close to a minimal test version, equating results can be better with many anchor items. Pre-equating design fully satisfies the requirement of the maximum number of anchor items because all items can be used as anchor items. Therefore invariance property can be better achieved than post-equating with some common items under the same condition. RMSD and REMSD values for ELA, where students' performances were not much impacted by the pandemic, showed the same results. However, there were some deviations in the invariance property for Math where students' performances appeared to be impacted.

The purpose of the current research study was to investigate the extent to which student performance was influenced by the disruption to learning that occurred due to the COVID-19 pandemic. To explain observed differences in performance, we compared outcomes from a state-wide summative testing program between students who were administered the test in 2019 and 2021. Because the background and demographic characteristics of the two testing populations differed on several variables, including race/ethnicity, locale, and subgroup membership, propensity score matching was used to develop comparable groups for analyses. Independent samples t-tests were then conducted to assess whether there were significant differences in mean scaled scores and domain subscores between the two matched groups. In addition, the same procedures were used to develop and analyze differences for specific student subgroups (students with disabilities, economically disadvantaged students, and English language learners). Overall, results indicated that the 2021 test-takers earned lower scores than students who tested in 2019 for all subjects and grade levels, except English Language Arts (ELA) grade 8. Moreover, larger score differences were observed for mathematics than ELA. Lastly, results from subgroup analyses indicated that in mathematics all subgroups earned significantly lower scores in 2021 than 2019, whereas in ELA grades 5 through 7, English Language Learners and economically disadvantaged subgroups earned lower scores in 2021 than 2019. The results of this study suggest that after controlling for differences between the 2021 and 2019 populations, student performance was likely influenced by the disruption to learning due to pandemic-related factors.