The Impact of the Pandemic on IRT Model/Data Fit

The application of item response theory (IRT) is almost universal in the development, implementation, and maintenance of large-scale assessments. Therefore, establishing the fit of IRT models to data is essential as the viability of calibration and equating implementations depend on it. In a typical test administration situation, measurement disturbances that influence model data fit are expected. Unfortunately, test administrations nationwide experienced new measurement disturbances because of the COVID-19 pandemic. Given the substantial disruption in education, did the response patterns of test takers change enough that model data fit is threatened and the degree of confidence in applying IRT analyses diminished? Using data from a large-scale state assessment system's 2019 and 2021 administration of the same test forms, model and data fit statistics for items and test takers were evaluated. The summary item fit index Q[subscript 1] (Yen, 1993) and the person fit statistic l[subscript z] (Choi, 2010; Drasgow et. al., 1985) were used for the analyses. Results from the study provide evidence that there wasn't a greater risk to the use of IRT models in 2021 than in previous years, despite the measurement disturbances introduced by the COVID-19 pandemic.