The Congenital Heart Technical Skill Study: Rationale and Design.

Background: We report the rationale and design for a peer-evaluation protocol of attending congenital heart surgeon technical skill using direct video observation.
Methods: All surgeons contributing data to The Society of Thoracic Surgeons-Congenital Heart Surgery Database (STS-CHSD) are invited to submit videos of themselves operating, to rate peers, or both. Surgeons may submit Norwood procedures, complete atrioventricular canal repairs, and/or arterial switch operations. A HIPPA-compliant website allows secure transmission/evaluation. Videos are anonymously rated using a modified Objective Structured Assessment of Technical Skills score. Ratings are linked to five years of contemporaneous outcome data from the STS-CHSD and surgeon questionnaires. The primary outcome is a composite for major morbidity/mortality.
Results: Two hundred seventy-six surgeons from 113 centers are eligible for participation: 83 (30%) surgeons from 53 (45%) centers have agreed to participate, with recruitment ongoing. These surgeons vary considerably in years of experience and outcomes. Participants, both early and late in their careers, describe the process as "very rewarding" and "less time consuming than anticipated." An initial subset of 10 videos demonstrated excellent interrater reliability (interclass correlation = 0.85).
Conclusions: This study proposes to evaluate the technical skills of attending pediatric cardiothoracic surgeons by video observation and peer-review. It is notable that over a quarter of congenital heart surgeons, across a range of experiences, from almost half of United States centers have already agreed to participate. This study also creates a mechanism for peer feedback; we hypothesize that feedback could yield broad and meaningful quality improvement.