Construct Validation of Minimum Competence in Standard Setting. Revised.

Studies of the Angoff method of standard setting suggest that judges agree in their estimates of the relative difficulties of test questions for minimally competent examinees and that each judge's estimates correlate well with the observed item difficulties for examinees whose total test scores are near the judge's personal standard (G. E. DeMauro, 1991). This finding suggests that Angoff estimates contain additive item-related and judge-related components, varying both from judge to judge and from estimated to observed performance by constants. Since, in homogeneous tests, observed performance on items also varies by constants over ability levels, the observed convergence of each judge's estimates on item performance near an individual standard is really a special case of convergence of all judges on item performance near a common deliberated standard. Data from the New Jersey High School Proficiency Test (NJHSPT) standard setting study supported this hypothesis. The convergence of the judges on a construct of minimal competence was studied for the standard setting study of multiple-choice items for three tests of the NJHSPT for grade 11. In all, 78 judges were involved. (Contains 3 tables and 14 references.) (Author/SLD)