Establishing a Mantel-Haenszel Alpha Cutscore through a Multiple Method Procedure.

The results of classifying test items on the basis of their Mantel-Haenszel (MH) alpha estimates were compared to the results of classifying these items using an item response theory (IRT) based procedure involving the comparison of item difficulties in the interest of identifying the alpha value that maximized the decision concordance between the two methods. The data consisted of candidates' responses to 299 scored items on an examination for professional licensure. A total of 68,458 candidates took this examination in 1988. The candidates' ethnicity was determined by their self-classifications into one of seven categories. A total of 47,573 candidates classified themselves in Ethnic Group 1; 6,486 in Ethnic Group 2; 5,466 in Ethnic Group 3; 2,004 in Ethnic Group 4; 1,014 in Ethnic Group 5; 486 in Ethnic Group 6; 307 in Ethnic Group 7; 726 as "other"; and the remaining 4,396 candidates did not specify their ethnicity. All candidates except the 4,396 who did not classify themselves were used for the MH analyses; for the IRT analyses, random samples of candidates were drawn from the four largest ethnic groups, while all candidates from the smaller groups were used. Candidates were divided into score groups. In both analyses, the majority ethnic group became the reference group and the other ethnic groups were designated focal. The MH estimate of alpha indicated that no differential item functioning was detected. Statistics resulting from IRT methods correlated highly with MH alpha values. Using the MH method in conjunction with IRT methods resulted in cutscores with a high level of decision concordance with advantages over traditional methods of using a significance level to establish an alpha criterion. Eight tables and eight figures provide study data. (SLD)