Back to Search
Start Over
Is the Area Under Curve Appropriate for Evaluating the Fit of Psychometric Models?
- Source :
-
Educational & Psychological Measurement . Jun2023, Vol. 83 Issue 3, p586-608. 23p. - Publication Year :
- 2023
-
Abstract
- In the literature of modern psychometric modeling, mostly related to item response theory (IRT), the fit of model is evaluated through known indices, such as χ2, M2, and root mean square error of approximation (RMSEA) for absolute assessments as well as Akaike information criterion (AIC), consistent AIC (CAIC), and Bayesian information criterion (BIC) for relative comparisons. Recent developments show a merging trend of psychometric and machine learnings, yet there remains a gap in the model fit evaluation, specifically the use of the area under curve (AUC). This study focuses on the behaviors of AUC in fitting IRT models. Rounds of simulations were conducted to investigate AUC's appropriateness (e.g., power and Type I error rate) under various conditions. The results show that AUC possessed certain advantages under certain conditions such as high-dimensional structure with two-parameter logistic (2PL) and some three-parameter logistic (3PL) models, while disadvantages were also obvious when the true model is unidimensional. It cautions researchers about the dangers of using AUC solely in evaluating psychometric models. [ABSTRACT FROM AUTHOR]
- Subjects :
- *COMPUTER simulation
*STRUCTURAL equation modeling
*COMPUTER software
*SAMPLE size (Statistics)
*MATHEMATICAL models
*MULTIVARIATE analysis
*CALIBRATION
*PSYCHOMETRICS
*THEORY
*STATISTICAL hypothesis testing
*RESEARCH funding
*DESCRIPTIVE statistics
*RECEIVER operating characteristic curves
*DIAGNOSTIC errors
*SENSITIVITY & specificity (Statistics)
*PROBABILITY theory
*ALGORITHMS
Subjects
Details
- Language :
- English
- ISSN :
- 00131644
- Volume :
- 83
- Issue :
- 3
- Database :
- Academic Search Index
- Journal :
- Educational & Psychological Measurement
- Publication Type :
- Academic Journal
- Accession number :
- 163453838
- Full Text :
- https://doi.org/10.1177/00131644221098182