Search

Background: Video-Assisted Thoracoscopic Surgery (VATS) lobectomy is an advanced procedure and to maximize patient safety it is important to ensure the competency of thoracic surgeons before performing the procedure. The objective of this study was to investigate validity evidence for a virtual reality simulator-based test including multiple lobes of the lungs. Method: VATS experts from the department of Cardiothoracic Surgery at Rigshospitalet, Copenhagen, Denmark, worked with Surgical Science (Gothenburg, Sweden) to develop VATS lobectomy modules for the LapSim®virtual reality simulator covering all five lobes of the lungs. Participants with varying experience in VATS were recruited and classified as either novice, intermediate, or experienced surgeons. Each participant performed VATS lobectomy on the simulator for three different randomly chosen lobes. Nine predefined simulator metrics were automatically recorded on the simulator. Results: Twenty-two novice, ten intermediate, and nine experienced surgeons performed the test resulting in a total of 123 lobectomies. Analysis of Variances (ANOVA) found significant differences between the three groups for parameters: blood loss (p< 0.001), procedure time (p< 0.001), and total instrument path length (p= 0.03). These three metrics demonstrated high internal consistency and significant test–retest reliability was found between each of them. Relevant pass/fail levels were established for each of the three metrics, 541 ml, 30 min, and 71 m, respectively. Conclusion: This study provides validity evidence for a simulator-based test of VATS lobectomy competence including multiple lobes of the lungs. The test can be used to ensure basic competence at the end of a simulation-based training program for thoracic surgery trainees.

Objectives: To determine the number of procedures and expert raters necessary to provide a reliable assessment of competence in Video-Assisted Thoracoscopic Surgery (VATS) lobectomy. Methods: Three randomly selected VATS lobectomies were performed on a virtual reality simulator by participants with varying experience in VATS. Video recordings of the procedures were independently rated by three blinded VATS experts using a modified VATS lobectomy assessment tool (VATSAT). The unitary framework of validity was used to describe validity evidence, and generalizability theory was used to explore the reliability of different assessment options. Results: Forty-one participants (22 novices, 10 intermediates, and 9 experienced) performed a total of 123 lobectomies. Internal consistency reliability, inter-rater reliability, and test–retest reliability were 0.94, 0.85, and 0.90, respectively. Generalizability theory found that a minimum of two procedures and four raters or three procedures and three raters were needed to ensure the overall reliability of 0.8. ANOVA showed significant differences in test scores between the three groups (P< 0.001). A pass/fail level of 19 out of 25 points was established using the contrasting groups’ standard setting method, leaving one false positive (one novice passed) and zero false negatives (all experienced passed). Conclusion: We demonstrated validity evidence for a VR simulator test with different lung lobes, and a credible pass/fail level was identified. Our results can be used to implement a standardized mastery learning training program for trainees in VATS lobectomies that ensures that everyone reaches basic competency before performing supervised operations on patients.