Search

Background: Feedback is important for surgical trainees but it can be biased and time-consuming. We examined crowd-sourced assessment as an alternative to experienced surgeons' assessment of robot-assisted radical prostatectomy (RARP)., Methods: We used video recordings (n = 45) of three RARP modules on the RobotiX, Simbionix simulator from a previous study in a blinded comparative assessment study. A group of crowd workers (CWs) and two experienced RARP surgeons (ESs) evaluated all videos with the modified Global Evaluative Assessment of Robotic Surgery (mGEARS)., Results: One hundred forty-nine CWs performed 1490 video ratings. Internal consistency reliability was high (0.94). Inter-rater reliability and test-retest reliability were low for CWs (0.29 and 0.39) and moderate for ESs (0.61 and 0.68). In an Analysis of Variance (ANOVA) test, CWs could not discriminate between the skill level of the surgeons (p = 0.03-0.89), whereas ES could (p = 0.034)., Conclusion: We found very low agreement between the assessments of CWs and ESs when they assessed robot-assisted radical prostatectomies. As opposed to ESs, CWs could not discriminate between surgical experience using the mGEARS ratings or when asked if they wanted the surgeons to perform their robotic surgery., (© 2023. The Author(s).)

Objectives: To assess the resection quality of transurethral bladder tumour resection (TURBT) and the association to surgeon experience depending on the presence of detrusor muscle., Methods: A retrospective study on 640 TURBT procedures performed at Zealand University Hospital, Denmark, from 1 January 2015 - 31 December 2016. Data included patient characteristics, procedure type, surgeon category, supervisor presence, surgical report data, pathological data, complications data and recurrence data. Analysis was performed using simple and multiple logistic regression on the association between surgeon experience and the presence of detrusor muscle in resected tissue from TURBT., Results: Supervised junior residents had significant lower detrusor muscle presence (73%) compared with consultants (83%) (OR = 0.4, 95% CI = 0.21-0.83). Limitations were the retrospective design and the diversity of included TURBT., Conclusions: It was found that surgical experience predicts detrusor muscle presence and supervised junior residents performing TURBT on patients resulted in less detrusor muscle than consultants.

Background: Competence in transurethral resection of bladder tumors (TURB) is critical in bladder cancer management and should be ensured before independent practice. Objective: To develop an assessment tool for TURB and explore validity evidence in a clinical context. Design, Setting, and Participants: From July 2019 to March 2021, a total of 33 volunteer doctors from three hospitals were included after exemption from the regional ethics committee (REG-008-2018). Participants performed two TURB procedures on patients with bladder tumors. A newly developed assessment tool (Objective Structured Assessment for Transurethral Resection of Bladder Tumors Skills, OSATURBS) was used for direct observation assessment (DOA), self-assessment (SA), and blinded video assessment (VA). Outcome Measurements and Statistical Analysis: Cronbach's alpha and Pearson's r were calculated for across items internal consistency reliability, inter-rater reliability, and test-retest reliability. Correlation between OSATURBS scores and the operative experience was calculated with Pearson's r and a pass/fail score was established. Differences in assessment scores were explored with paired t -test and independent samples t -test. Results and Limitations: The internal consistency reliability across items Cronbach's alpha was 0.94 ( n  = 260, p  < 0.001). Inter-rater reliability was 0.80 ( n  = 64, p  < 0.001). Test-retest correlation was high, r  = 0.71 ( n  = 32, p  < 0.001). Relationship with TURB experience was high, r  = 0.71 ( n  = 32, p  < 0.001). Pass/fail score was 19 points. DOAs were strongly correlated with video ratings ( r  = 0.85, p  < 0.001) but with a significant social bias with lower scores for inexperienced and higher scores for experienced participants. Participants tended to overestimate their own performances. Conclusions: OSATURBS tool for TURB can be used for assessment of surgical proficiency in the clinical setting. DOA and SA are biased, and blinded VA of TURB performances is advised. Clinical Trials NCT03864302.

Background: Transurethral resection of bladder tumours (TURBT) is the initial diagnostic treatment for patients with bladder cancer. TURBT is not an easy procedure to master and simulator training may play a role in improving the learning curve., Objective: To implement a national training programme for simulation-based mastery learning in TURBT and explore operating theatre performance after training., Design Setting and Participants: From June 2019 to March 2021, 31 doctors at urology departments in Denmark performed two pretraining TURBT procedures on patients, followed by proficiency-based mastery learning on a virtual reality simulator and then two post-training TURBTs on patients., Outcome Measurements and Statistical Analyses: Operating theatre performances were video-recorded and assessed by two independent, blinded raters using the Objective Structured Assessment for Transurethral Resection of Bladder Tumours Skills (OSATURBS) assessment tool. Paired-sample t tests were used to compare pretraining and post-training analyses and independent t tests for between-group comparisons. This trial is registered at ClinicalTrials.gov as NCT03864302., Results and Limitations: Before training, novices had significantly lower performance scores in comparison to those with intermediate experience ( p = 0.017) and experienced doctors ( p < 0.001). After training, novices significantly improved their clinical performance score (from 11.4 to 17.1; p = 0.049, n = 10). Those with intermediate experience and experienced doctors did not benefit significantly from simulator training ( p = 0.9 and p = 0.8, respectively)., Conclusions: Novices improved their TURBT performance in the operating theatre after completing a proficiency-based training programme on a virtual reality simulator., Patient Summary: We trained surgeons in an operation to remove bladder tumours using a virtual reality simulator. Novice doctors improved their performance significantly after the training, but the training effects for more experienced doctors were minimal. Therefore, we suggest the introduction of mandatory simulator training in the residency programme for urologists., (© 2022 The Author(s).)

Purpose: To investigate validity evidence for a simulator-based test in robot-assisted radical prostatectomy (RARP). Materials and Methods: The test consisted of three modules on the RobotiX Mentor VR-simulator: Bladder Neck Dissection, Neurovascular Bundle Dissection, and Ureterovesical Anastomosis . Validity evidence was investigated by using Messick's framework by including doctors with different RARP experience: novices (who had assisted for RARP), intermediates (robotic surgeons, but not RARP surgeons), or experienced (RARP surgeons). The simulator metrics were analyzed, and Cronbach's alpha and generalizability theory were used to explore reliability. Intergroup comparisons were done with mixed-model, repeated measurement analysis of variance and the correlation between the number of robotic procedures and the mean test score were examined. A pass/fail score was established by using the contrasting groups' method. Results: Ten novices, 11 intermediates, and 6 experienced RARP surgeons were included. Six metrics could discriminate between groups and showed acceptable internal consistency reliability, Cronbach's alpha = 0.49, p  < 0.001. Test-retest reliability was 0.75, 0.85, and 0.90 for one, two, and three repetitions of tests, respectively. Six metrics were combined into a simulator score that could discriminate between all three groups, p  = 0.002, p  < 0.001, and p  = 0.029 for novices vs intermediates, novices vs experienced, and intermediates vs experienced, respectively. Total number of robotic operations and the mean score of the three repetitions were significantly correlated, Pearson's r  = 0.74, p  < 0.001. Conclusion: This study provides validity evidence for a simulator-based test in RARP. We determined a pass/fail level that can be used to ensure competency before proceeding to supervised clinical training.

Objectives: A prospective observational trial to develop and gather validity evidence using Messick's framework for a simulator-based test in TURB. Methods: Forty-nine doctors were recruited from urology departments (Herlev/Gentofte University Hospital, Rigshospitalet Copenhagen University Hospital and Zealand University Hospital Roskilde) and enrolled from April to September 2018. The TURB Mentor™ virtual reality (VR) simulator was assessed at an expert meeting selecting clinically relevant cases and metrics. Test sessions were done on identical simulators at two university hospitals in Denmark. All participants performed three TURB procedures on the VR simulator. Simulator metrics were analysed with analysis of variance (ANOVA) and metrics with the ability to discriminate between groups were combined in a total simulator score. Finally, a pass/fail score was identified using the contrasting groups' method. Results: Eleven simulator metrics were found eligible and four had significant discrimination ability between competency levels: resected pathology (%) ( p  = 0.008); cutting in bladder wall ( n ) ( p  = 0.004); time (s) ( p  = 0.034); and inspection of the bladder wall (%) ( p  = 0.002). The internal structure of the total simulator score [(resected pathology*inspection of the bladder wall)/time] was high with the intraclass correlation coefficient, Cronbach's alpha = 0.85. The mean total simulator score was significantly lower in the novice group than in the intermediate, 15.9 and 25.6, respectively (mean difference = 9.7, p  = 0.011) and experienced group, 30.6 (mean difference = 14.7, p  < 0.001). A pass/fail score of 22 was identified. Conclusion: We found validity evidence for a newly developed VR simulator-based test and establised a pass/fail score identifying surgical skills in TURB. The TURBEST test can be used in a proficiency-based TURB simulator training programme for accreditation prior to supervised procedures on patients.

Background: Both clinicians and patients use medical mobile phone apps. Anyone can publish medical apps, which leads to contents with variable quality that may have a serious impact on human lives. We herein provide an overview of the prevalence of expert involvement in app development and whether or not app contents adhere to current medical evidence., Objective: To systematically review studies evaluating expert involvement or adherence of app content to medical evidence in medical mobile phone apps., Methods: We systematically searched 3 databases (PubMed, The Cochrane Library, and EMBASE), and included studies evaluating expert involvement or adherence of app content to medical evidence in medical mobile phone apps. Two authors performed data extraction independently. Qualitative analysis of the included studies was performed., Results: Based on inclusion criteria, 52 studies were included in this review. These studies assessed a total of 6520 apps. Studies dealt with a variety of medical specialties and topics. As much as 28 studies assessed expert involvement, which was found in 9-67% of the assessed apps. Thirty studies (including 6 studies that also assessed expert involvement) assessed adherence of app content to current medical evidence. Thirteen studies found that 10-87% of the assessed apps adhered fully to the compared evidence (published studies, recommendations, and guidelines). Seventeen studies found that none of the assessed apps (n=2237) adhered fully to the compared evidence., Conclusions: Most medical mobile phone apps lack expert involvement and do not adhere to relevant medical evidence.