Summative Evaluation of Vaginal Surgery Skills: Setting A Pass-Fail Score.

Introduction and Hypothesis: We developed a summative assessment tool to evaluate competent performance on three procedure-specific low fidelity simulation models for vaginal surgery. Our purpose was to determine a pass-fail score for each model.
Methods: We enrolled participants (2011-2023, three Canadian academic centers) and grouped them according to operative competency in vaginal procedures. Novice operators were medical students recruited through targeted advertisement to clerkship level medical students. Proficient operators consisted of gynecology residents from the intervention arm of a randomized controlled trial, trained to competence in the use of the models; urogynecology fellows and attending gynecologic surgeons recruited through departmental rounds. All participants were asked to perform the three procedures on the models, were videotaped, and their performance assessed by evaluators familiar with the procedure and the scoring system, blinded to operator identity. A total performance score (range 0-400) assessed timing and errors. Basic skill deductions were set a priori. We calculated sensitivity and specificity scores and obtained an optimal cutoff based on Youden's J statistic.
Results: For anterior repair, we rated 46 novice and 16 proficient videos. The pass-fail score was 170/400. For posterior repair, we rated 54 novice and 14 proficient videos. The pass-fail score was 140/400. For vaginal hysterectomy, we rated 47 novice and 12 proficient videos. The pass-fail score was 180/400. Scores of proficient operators were significantly better than those of novice participants (p < 0.001 for all).
Conclusions: A pass-fail score can distinguish between novice and proficient operators and can be used for summative assessment of surgical skill.
(© 2024. The International Urogynecological Association.)