A comparison of standard-setting procedures for an OSCE in undergraduate medical education.

Purpose: To compare four standard-setting procedures for an objective structure clinical examination (OSCE).
Methods: A 12-station OSCE was administered to 84 students in each of the final (fourth-) year medical classes of 1996 and 1997 at Dalhousie University Faculty of Medicine. Four standard-setting procedures (Angoff, borderline, relative, and holistic) were applied to the data to establish a cutoff score for a pass/fail decision.
Results: The procedures yielded highly inconsistent results. The Angoff and borderline procedures gave similar results; however, the relative and holistic methods gave widely divergent results. The Angoff procedure yielded results reliable enough to use in decision making for a high-stakes examination, but would have required more judges or more stations.
Conclusions: The Angoff and borderline procedures provide reasonable and defensible approaches to standard setting and are practical to apply by non-psychometricians in medical schools. Further investigation of the other procedures is needed.