Validity of Item Performance Estimates from an Angoff Standard Setting Study.

Judgmental standard setting methods, such as the W. H. Angoff (1971) method, use item performance estimates as the basis for determining the minimum passing score (MPS). Therefore, the accuracy of these item performance estimates is crucial to the validity of the resulting MPS. Recent researchers, (L. A. Shephard 1994; J. Impara, 1997) have called into question the ability of the judges to make accurate item performance estimates for the target subgroups of candidates, such as minimally competent candidates (MCCs). The purpose of this study was to examine the validity of the judges' estimates of MCC performance that are used to determine a minimum passing score in a standard setting study context. Results from the operational standard setting workshops for a certification program in financial management that used 29 judges in 1996 and 30 in 1997 provide evidence that item performance estimates were valid. Factors that might have influenced this high degree of validity in the item performance estimates in the standard setting study are discussed. (Contains 10 references.) (SLD)