1. Predictive validities: figures of merit or veils of deception?
- Author
-
Peter H. Schonemann and Moritz Heene
- Subjects
FOS: Computer and information sciences ,validity ,DOAJ:Psychology ,test bias ,media_common.quotation_subject ,PsyArXiv|Social and Behavioral Sciences|Quantitative Methods|Mathematical Psychology ,lcsh:BF1-990 ,bepress|Social and Behavioral Sciences|Psychology|Quantitative Psychology ,classification errors ,Social and Behavioral Sciences ,Statistics - Applications ,Methodology (stat.ME) ,PsyArXiv|Social and Behavioral Sciences|Quantitative Methods|Experimental Design and Sample Surveys ,PsyArXiv|Social and Behavioral Sciences|Quantitative Methods|Quantitative Psychology ,Statistics ,Range (statistics) ,Econometrics ,Psychology ,PsyArXiv|Social and Behavioral Sciences|Quantitative Methods|Statistical Methods ,Multiple correlation ,Applications (stat.AP) ,test construction ,Statistics - Methodology ,media_common ,IQ tests ,Life Sciences ,PsyArXiv|Social and Behavioral Sciences|Quantitative Methods|Computational Modeling ,Quantitative Psychology ,Deception ,Test theory ,Test bias ,PsyArXiv|Social and Behavioral Sciences|Quantitative Methods|Psychometrics ,Test (assessment) ,Zero (linguistics) ,FOS: Psychology ,Unexpected finding ,PsyArXiv|Social and Behavioral Sciences ,lcsh:Psychology ,test theory ,lcsh:B ,bepress|Social and Behavioral Sciences ,PsyArXiv|Social and Behavioral Sciences|Quantitative Methods ,DOAJ:Social Sciences ,lcsh:Philosophy. Psychology. Religion - Abstract
The ETS has recently released new estimates of validities of the GRE for predicting cumulative graduate GPA. They average in the middle thirties – twice as high as those previously reported by a number of independent investigators.It is shown in the first part of this paper that this unexpected finding can be traced to a flawed methodology that tends to inflate multiple correlation estimates, especially those of populations values near zero.Secondly, the issue of upward corrections of validity estimates for restriction of range is taken up. It is shown that they depend on assumptions that are rarely met by the data.Finally, it is argued more generally that conventional test theory, which is couched in terms of cor- relations and variances, is not only unnecessarily abstract but, more importantly, incomplete, since the practical utility of a test does not only depend on its validity, but also on base-rates and admission quotas. A more direct and conclusive method for gauging the utility of a test involves misclassification rates, and entirely dispenses with questionable assumptions and post-hoc "corrections".On applying this approach to the GRE, it emerges (1) that the GRE discriminates against ethnic and economic minorities, and (2) that it often produces more erroneous decisions than a purely random admissions policy would.
- Published
- 2009