Comprehensive assessment of thyroidectomy skills development: a pilot project

Objectives/Hypothesis: To test the validity, reliability, and feasibility of an evaluation tool designed to measure the development of trainees' surgical skills in the operating room for thyroid surgery. Study Design: Prospective validation study. Methods: A modified Delphi technique was employed to develop a new Objective Structured Assessment of Technical Skills–based instrument for thyroid surgery. During a 1-year period, 16 otolaryngology–head and neck surgery residents (ranging from postgraduate year 2 to 6) and one endocrine surgery fellow were evaluated by one faculty member obtaining a total of 94 evaluations. Performance was rated using a task-based checklist (TBC) and a global rating scale (GRS). The TBC measured trainees' thyroidectomy technical skills, and the GRS assessed their overall surgical performance. Results: Based on four clinical levels (junior, intermediate, senior, and surgical fellow) our tool demonstrated construct validity for both components of the assessment instrument, specifically for the TBC showing a mean difference of 0.9 (95% confidence interval: 0.5-1.3, P < .001) between the contiguous clinical levels senior versus intermediate. Cronbach α, a measure of internal consistency, was 0.96 for both components of the instrument. The correlation between the TBC and GRS was also high within trainee (r = 0.62, n = 94, P < .001) and across trainees (r = 0.96, n = 17, P < .001). Conclusions: Our tool proved to be a valid, reliable, and feasible instrument for assessing competency in thyroid surgery. It is effective in providing timely formative feedback during and upon the conclusion of the surgical procedure by identifying procedural tasks for which additional training is necessary. In addition, it enables longitudinal tracking of residents' surgical performance, thus ensuring their appropriate development. Laryngoscope, 122:103–109, 2012