Discriminatory ability of milestones: An analysis of milestone variability by obstetrics and gynecology subspecialty

Background: Little evidence exists regarding Accreditation Council for Graduate Medical Education (ACGME) milestone discriminatory ability. This short report describes variability in milestone scores by category to determine their utility in discerning high and low performers in a single Obstetrics and Gynecology residency. Methods: A Clinical Competency Committee (CCC) design was implemented with four subcommittees, each responsible for a predetermined milestones subset: Obstetrics, Gynecology, Ambulatory Practice, and Professional Activities. Milestone scores for 44 residents per year over four biannual evaluation cycles (2014-2016) were evaluated, for a total of 176 independent evaluations. Results: Findings indicate that discriminatory ability, assessed by variability between resident scores, differed by subcommittee. Subcommittees that were primarily tasked with evaluating clinical- and procedural-based milestones demonstrated lower discriminatory ability among trainees. Conclusion: Greater Professional Activity milestone variability indicates better differentiation; future research should determine correlation of these findings with other professionalism performance metrics and novel intervention strategies.