A Qualitative Examination of Teachers' Conception of the Just Competent Examinee in Angoff (1971) Workshops.

A key component of the Angoff (W. Angoff, 1971) method for setting cut scores is the target examinee. Expert judges are asked, following training and discussion, to consider the ability and the likely performance of some subset of examinees and to then estimate, item by item, the likely performance of such examinees on the test for which a cut score is desired. Questions of interest for this study were: (1) how teachers who serve as judges on cut score panels describe the target examinee; (2) how teachers' descriptions of the target examinee compare to the definitions provided by workshop facilitators as a frame for the discussion of target performance; (3) how teachers' descriptions of the target examinee compare to the behavioral characteristics arrived at during the workshop training; and (4) how descriptions of the target examinee compare across workshops. The results of two studies involving panel groups of 22 fourth-grade teachers and 20 ninth-grade teachers suggest that teachers who serve as judges in cut score setting processes are influenced by the definitions and training used in the standard setting study. Further, it appears that some common notion of the just competent examinee might be at work. Future studies should examine teachers' conception of the just competent students (given the purpose of the intended use of the cut score) before, as well as after, Angoff workshops. An appendix contains salient phrases extracted from teacher responses to the study prompts. (SLD)