Assessing senior residents' knowledge and performance: an integrated evaluation program.

Background: Surgical residents have traditionally been evaluated by ward evaluations and multiple-choice testing; overall resident performance has seldom been objectively evaluated. We developed a comprehensive evaluation program for senior residents. The purpose of this study was to determine whether the several different assessment methods included in this program provide similar information about senior residents' clinical knowledge and performance.
Methods: Sixteen senior general surgery residents were evaluated by subjective faculty ward evaluations, a structured oral examination, the American Board of Surgery In-Training Examination, and an objective structured clinical examination (OSCE). The OSCE was divided into two parts: part A required the resident to obtain a directed history or to perform a physical examination; part B required the resident to answer questions about the patient seen in part A.
Results: The various evaluation methods differed in their estimates of clinical competence (ward evaluations rated residents highest; the OSCE, lowest). The American Board of Surgery In-Training Examination correlated with both the structured oral examination and OSCE part B, indicating that all are adequate measures of knowledge. Neither the total OSCE score nor OSCE part A correlated with other measures, but the fact that both are highly reliable suggests that they evaluate different clinical skills. Faculty ward evaluations are inflated and do not correlate with other measures.
Conclusions: We conclude that developing a comprehensive program for evaluating resident competence is desirable and feasible.