Search

Background: Competence in transurethral resection of bladder tumors (TURB) is critical in bladder cancer management and should be ensured before independent practice. Objective: To develop an assessment tool for TURB and explore validity evidence in a clinical context. Design, Setting, and Participants: From July 2019 to March 2021, a total of 33 volunteer doctors from three hospitals were included after exemption from the regional ethics committee (REG-008-2018). Participants performed two TURB procedures on patients with bladder tumors. A newly developed assessment tool (Objective Structured Assessment for Transurethral Resection of Bladder Tumors Skills, OSATURBS) was used for direct observation assessment (DOA), self-assessment (SA), and blinded video assessment (VA). Outcome Measurements and Statistical Analysis: Cronbach's alpha and Pearson's r were calculated for across items internal consistency reliability, inter-rater reliability, and test-retest reliability. Correlation between OSATURBS scores and the operative experience was calculated with Pearson's r and a pass/fail score was established. Differences in assessment scores were explored with paired t -test and independent samples t -test. Results and Limitations: The internal consistency reliability across items Cronbach's alpha was 0.94 ( n  = 260, p  < 0.001). Inter-rater reliability was 0.80 ( n  = 64, p  < 0.001). Test-retest correlation was high, r  = 0.71 ( n  = 32, p  < 0.001). Relationship with TURB experience was high, r  = 0.71 ( n  = 32, p  < 0.001). Pass/fail score was 19 points. DOAs were strongly correlated with video ratings ( r  = 0.85, p  < 0.001) but with a significant social bias with lower scores for inexperienced and higher scores for experienced participants. Participants tended to overestimate their own performances. Conclusions: OSATURBS tool for TURB can be used for assessment of surgical proficiency in the clinical setting. DOA and SA are biased, and blinded VA of TURB performances is advised. Clinical Trials NCT03864302.

Objectives: A prospective observational trial to develop and gather validity evidence using Messick's framework for a simulator-based test in TURB. Methods: Forty-nine doctors were recruited from urology departments (Herlev/Gentofte University Hospital, Rigshospitalet Copenhagen University Hospital and Zealand University Hospital Roskilde) and enrolled from April to September 2018. The TURB Mentor™ virtual reality (VR) simulator was assessed at an expert meeting selecting clinically relevant cases and metrics. Test sessions were done on identical simulators at two university hospitals in Denmark. All participants performed three TURB procedures on the VR simulator. Simulator metrics were analysed with analysis of variance (ANOVA) and metrics with the ability to discriminate between groups were combined in a total simulator score. Finally, a pass/fail score was identified using the contrasting groups' method. Results: Eleven simulator metrics were found eligible and four had significant discrimination ability between competency levels: resected pathology (%) ( p  = 0.008); cutting in bladder wall ( n ) ( p  = 0.004); time (s) ( p  = 0.034); and inspection of the bladder wall (%) ( p  = 0.002). The internal structure of the total simulator score [(resected pathology*inspection of the bladder wall)/time] was high with the intraclass correlation coefficient, Cronbach's alpha = 0.85. The mean total simulator score was significantly lower in the novice group than in the intermediate, 15.9 and 25.6, respectively (mean difference = 9.7, p  = 0.011) and experienced group, 30.6 (mean difference = 14.7, p  < 0.001). A pass/fail score of 22 was identified. Conclusion: We found validity evidence for a newly developed VR simulator-based test and establised a pass/fail score identifying surgical skills in TURB. The TURBEST test can be used in a proficiency-based TURB simulator training programme for accreditation prior to supervised procedures on patients.