Search

Objective To investigate whether training otorhinolaryngology residents to criterion performance levels (proficiency) on the Endoscopic Sinus Surgery Simulator produces individuals whose performance in the operating room is at least equal to those who are trained by performing a fixed number of surgical procedures. Design Prospective cohort. Setting Two academic medical centers in New York City. Participants Otorhinolaryngology junior residents composed of 8 experimental subjects and 6 control subjects and 6 attending surgeons. Intervention Experimental subjects achieved benchmark proficiency criteria on the Endoscopic Sinus Surgery Simulator; control subjects repeated the surgical procedure twice. Main Outcome Measures Residents completed validated objective tests to assess baseline abilities. All subjects were videotaped performing an initial standardized surgical procedure. Residents were videotaped performing a final surgery. Videotapes were assessed for metrics by an expert panel. Results Attendings outperformed the residents in most parameters on the initial procedure. Experimental and attending groups outperformed controls in some parameters on the final procedure. There was no difference between resident groups in initial performance, but the experimental subjects outperformed the control subjects in navigation in the final procedure. Most important, there was no difference in final performance between subgroups of the experimental group on the basis of the number of trials needed to attain proficiency. Conclusions Simulator training can improve resident technical skills so that each individual attains a proficiency level, despite the existence of an intrinsic range of abilities. This proficiency level translates to at least equal, if not superior, operative performance compared with that of current conventional training with finite repetition of live surgical procedures.

Objectives To establish discriminant validity of the endoscopic sinus surgery simulator (ES3) (Lockheed Martin, Akron, Ohio) between various health care provider experience levels and to define benchmarking criteria for skills assessment. Design Prospective multi-institutional comparison study. Setting University-based tertiary care institution. Participants Ten expert otolaryngologists, 14 otolaryngology residents, and 10 medical students. Interventions Subjects completed the ES3's virtual reality curriculum (10 novice mode, 10 intermediate mode, and 3 advanced mode trials). Performance scores were recorded on each trial. Performance differences were analyzed using analysis of variance for repeated measures (experience level as between-subjects factor). Main Outcome Measures Simulator performance scores, accuracy, time to completion, and hazard disruption. Results The novice mode accurately distinguished the 3 groups, particularly at the onset of training (mean scores: senior otolaryngologists, 66.0; residents, 42.7; students, 18.3; for the paired comparisons between groups 1 and 2 and groups 1 and 3, P = .04 and .03, respectively). Subjects were not distinguished beyond trial 5. The intermediate mode only discriminated students from other subjects ( P = .008). The advanced mode did not show performance differences between groups. Scores on the novice mode predicted those on the intermediate mode, which predicted advanced mode scores ( r = 0.687), but no relationship was found between novice and advanced scores. All groups performed equally well and with comparable consistency at the outset of training. Expert scores were used to define benchmark criteria of optimal performance. Conclusions This study completes the construct validity assessment of the ES3 by demonstrating its discriminant capabilities. It establishes expert surgeon benchmark performance criteria and shows that the ES3 can train novice subjects to attain those. The refined analysis of trial performance scores could serve educational and skills assessment purposes. Current studies are evaluating the transfer of surgical skills acquired on the ES3 to the operating room (predictive validity).