Search

Background: Competency-based training has gained ground in surgical training and with it assessment tools to ensure that training objectives are met. Very few assessment tools are available for evaluating performance in thoracoscopic procedures. Video recordings would provide the possibility of blinded assessment and limited rater bias. This study aimed to provide validity evidence for a newly developed and dedicated tool for assessing competency in Video-Assisted Thoracoscopic Surgery (VATS) lobectomy., Methods: Participants with varying experience with VATS lobectomy were included from different countries. Video recordings from participants' performance of a VATS right upper lobe lobectomy on a virtual reality simulator were rated by three raters using a modified version of a newly developed VATS lobectomy assessment tool (the VATSAT) and analyzed in relation to the unitary framework (content, response process, internal structure, relation to other variables, and consequences of testing)., Results: Fifty-three participants performed two consecutive simulated VATS lobectomies on the virtual reality simulator, leaving a total of 106 videos. Content established in previously published studies. Response process Standardized data collection was ensured by using an instructional element, uniform data collection, a special rating program, and automatic generation of the results to a database. Raters were carefully instructed in using the VATSAT, and tryout ratings were carried out. Internal structure Inter-rater reliability was calculated as intra-class correlation coefficients, to 0.91 for average measures (p < 0.001). Test/re-test reliability was calculated as Pearson's r of 0.70 (p < 0.001). G-coefficient was calculated to be 0.79 with two procedures and three raters. By performing D-theory was found that either three procedures rated by two raters or five procedures rated by one rater were enough to reach an acceptable G-coefficient of ≥ 0.8. Relation to other variables Significant differences between groups were found (p < 0.001). The participants' VATS lobectomy experience correlated significantly to their VATSAT score (p = 0.016). Consequences of testing The pass/fail score was found to be 14.9 points by the contrasting groups' method, leaving five false positive (29%) and six false negatives (43%)., Conclusion: Validity evidence was provided for the VATSAT according to the unitary framework. The VATSAT provides supervisors and assessors with a procedure-specific assessment tool for evaluating VATS lobectomy performance and aids with the decision of when the trainee is ready for unsupervised performance.

Background: The societies of thoracic surgery are working to incorporate simulation and competency-based assessment into specialty training. One challenge is the development of a simulation-based test, which can be used as an assessment tool. The study objective was to establish validity evidence for a virtual reality simulator test of a video-assisted thoracoscopic surgery (VATS) lobectomy of a right upper lobe., Methods: Participants with varying experience in VATS lobectomy were included. They were familiarized with a virtual reality simulator (LapSim ® ) and introduced to the steps of the procedure for a VATS right upper lobe lobectomy. The participants performed two VATS lobectomies on the simulator with a 5-min break between attempts. Nineteen pre-defined simulator metrics were recorded., Results: Fifty-three participants from nine different countries were included. High internal consistency was found for the metrics with Cronbach's alpha coefficient for standardized items of 0.91. Significant test-retest reliability was found for 15 of the metrics (p-values <0.05). Significant correlations between the metrics and the participants VATS lobectomy experience were identified for seven metrics (p-values <0.001), and 10 metrics showed significant differences between novices (0 VATS lobectomies performed) and experienced surgeons (>50 VATS lobectomies performed). A pass/fail level defined as approximately one standard deviation from the mean metric scores for experienced surgeons passed none of the novices (0 % false positives) and failed four of the experienced surgeons (29 % false negatives)., Conclusion: This study is the first to establish validity evidence for a VATS right upper lobe lobectomy virtual reality simulator test. Several simulator metrics demonstrated significant differences between novices and experienced surgeons and pass/fail criteria for the test were set with acceptable consequences. This test can be used as a first step in assessing thoracic surgery trainees' VATS lobectomy competency.

Background: Testing stimulates learning, improves long-term retention, and promotes technical performance. No purpose-orientated test of competence in the theoretical aspects of VATS lobectomy has previously been presented. The purpose of this study was, therefore, to develop and gather validity evidence for a theoretical test on VATS lobectomy consisting of multiple-choice questions., Methods: Four European VATS lobectomy experts were interviewed to explore their views on important theoretical VATS lobectomy knowledge (step 1). This information was used to construct the test items in compliance with existing guidelines for multiple-choice questions (step 2). The experts rated the relevance of the items to confirm content validity in a modified Delphi approach (step 3). Finally, the test was administered to physicians, who were categorised into different experience levels based on their experience in VATS procedures overall and in VATS lobectomies specifically. Their answers were used to achieve construct validity (step 4)., Results: Initially, 81 items were constructed and two Delphi iterations reduced the test to 50 items. Item analysis led to the exclusion of 19 items and the mean discrimination index of the 31 final items was 0.26. Cronbach's alpha for internal consistency was 0.75. The mean item difficulty was calculated to 0.63. According to performed VATS procedures, significantly different test performances were detected when comparing the group performances (p = 0.002) and the experts performed significantly better than the novices (p < 0.001) and intermediates (p = 0.01). In the category of performed VATS lobectomies, significant group performances were also found. In this category, the experts were also significantly better than the novices (p < 0.001), the trainees (p = 0.002), and the intermediates (p = 0.01)., Conclusions: This study led to the development of a theoretical test on VATS lobectomy consisting of multiple-choice questions. Both content and construct validity evidence were established.

Background: As we move toward competency-based education in medicine, we have lagged in developing competency-based evaluation methods. In the era of minimally invasive surgery, there is a need for a reliable and valid tool dedicated to measure competence in video-assisted thoracoscopic surgery. The purpose of this study is to create such an assessment tool, and to explore its reliability and validity., Methods: An expert group of physicians created an assessment tool consisting of 10 items rated on a five-point rating scale. The following factors were included: economy and confidence of movement, respect for tissue, precision of operative technique, creation and placement of ports, localization of pathologic tissue, use of staplers, retrieval of tissue in bag and placement of chest tube. Fifty consecutive thoracoscopic wedge resections were recorded and assessed blindly and independently by two experts using the tool., Results: Four residents, four fellows and five consultants performed 1-10 (median 4) operations each. The fellows performed significantly better than the residents (P = 0.03; effect size, ES = 0.72). The consultants scored 11% higher than the fellows, but this difference was not significant (P = 0.10, ES = 0.64). The inter-rater reliability was acceptable (Cronbach's alpha 0.71)., Conclusions: This tool for assessing performance in thoracoscopy is reliable and valid. It can provide unbiased feedback to trainees, and can be used to evaluate new teaching curricula, i.e. simulation-based training. Furthermore, it has potential to aid in certification of new thoracic surgeons.

Background: Lobectomy using video-assisted thoracoscopic surgery (VATS) still is a controversial operation despite its many observed benefits. The controversy may be due to difficulties performing the procedure. This study addresses a standardized anterior approach facilitating the operation., Methods: This report describes 156 VATS lobectomies performed with a uniform anterior three-port technique. The surgeon and the assistant stand on the abdominal side of the patient, and the setup is independent of the lobe to be resected., Results: From 2005 to 2007, 156 VATS lobectomies and 13 conversions (7.7%) were performed with an anterior three-port technique. No mortality occurred among the VATS lobectomies during a 30-day period. The major comorbidity rate was 78%. The comorbidities included former thoracic surgery (9%) or a history of another cancer (22%), indicating potentially difficult surgery and a nonselected population. The majority (92%) of the patients had lung cancer. The major postoperative problem was air leakage, which usually prolongs the tube time and the in-hospital stay, but a "no-touch fissure" technique significantly reduced the median tube time to 4 days and the postoperative hospital stay to 5 days., Conclusions: This study shows that VATS lobectomies can be performed using a standardized anterior approach with low rates of conversion and morbidity. A "no-touch fissure" technique lowers the postoperative tube time, the in-hospital stay, and the number of patients with prolonged air leakage. A uniform anterior approach facilitates VATS lobectomy and makes this advanced procedure more easily adaptable by many surgeons familiar with an open anterior approach. The findings show that VATS lobectomy can be performed safely for a large proportion of lung cancer patients.