Applying a Multiple Comparison Control to IRT Item-fit Testing.

We used simulation techniques to assess the item-level and familywise Type I error control and power of an IRT item-fit statistic, the S-X². Previous research indicated that the S-X² has good Type I error control and decent power, but no previous research examined familywise Type I error control. We varied percentage of misfitting items, sample size, and test length, and computed familywise Type I error with no correction, a Bonferroni correction, and a Benjamini-Hochberg correction. The S-X² controlled item-level and familywise Type I errors when corrections were applied to conditions with no misfitting items. In the presence of misfitting items, the S-X² exhibited inflated item-level and familywise false hit rates in many conditions, even with familywise Type I error corrections. Lastly, power was low and negatively impacted when either of the familywise Type I error corrections was applied. We suggest using the S-X² with no familywise Type I error control in conjunction with other methods of assessing item fit (e.g., visual analysis). [ABSTRACT FROM AUTHOR]