Search

The objective of this study is to compare automated performance metrics (APM) and surgical gestures for technical skills assessment during simulated robot-assisted radical prostatectomy (RARP). Ten novices and six experienced RARP surgeons performed simulated RARPs on the RobotiX Mentor (Surgical Science, Sweden). Simulator APM were automatically recorded, and surgical videos were manually annotated with five types of surgical gestures. The consequences of the pass/fail levels, which were based on contrasting groups' methods, were compared for APM and surgical gestures. Intra-class correlation coefficient (ICC) analysis and a Bland-Altman plot were used to explore the correlation between APM and surgical gestures. Pass/fail levels for both APM and surgical gesture could fully distinguish between the skill levels of the surgeons with a specificity and sensitivity of 100%. The overall ICC (one-way, random) was 0.70 (95% CI: 0.34-0.88), showing moderate agreement between the methods. The Bland-Altman plot showed a high agreement between the two methods for assessing experienced surgeons but disagreed on the novice surgeons' skill level. APM and surgical gestures could both fully distinguish between novices and experienced surgeons in a simulated setting. Both methods of analyzing technical skills have their advantages and disadvantages and, as of now, those are only to a limited extent available in the clinical setting. The development of assessment methods in a simulated setting enables testing before implementing it in a clinical setting., (© 2024. The Author(s).)

Introduction: Simulation-based training (SBT) aids healthcare providers in acquiring the technical skills necessary to improve patient outcomes and safety. However, since SBT may require significant resources, training all skills to a comparable extent is impractical. Hence, a strategic prioritization of technical skills is necessary. While the European Training Requirements in Neonatology provide guidance on necessary skills, they lack prioritization. We aimed to identify and prioritize technical skills for a SBT curriculum in neonatology., Methods: A three-round modified Delphi process of expert neonatologists and neonatal trainees was performed. In round one, the participants listed all the technical skills newly trained neonatologists should master. The content analysis excluded duplicates and non-technical skills. In round two, the Copenhagen Academy for Medical Education and Simulation Needs Assessment Formula (CAMES-NAF) was used to preliminarily prioritize the technical skills according to frequency, importance of competency, SBT impact on patient safety, and feasibility for SBT. In round three, the participants further refined and reprioritized the technical skills. Items achieving consensus (agreement of ≥75%) were included., Results: We included 168 participants from 10 European countries. The response rates in rounds two and three were 80% (135/168) and 87% (117/135), respectively. In round one, the participants suggested 1964 different items. Content analysis revealed 81 unique technical skills prioritized in round two. In round three, 39 technical skills achieved consensus and were included., Conclusion: We reached a European consensus on a prioritized list of 39 technical skills to be included in a SBT curriculum in neonatology., (© 2024 S. Karger AG, Basel.)

Background: Artificial intelligence (AI) is becoming increasingly used in medical education, but our understanding of the validity of AI-based assessments (AIBA) as compared with traditional clinical expert-based assessments (EBA) is limited. In this study, the authors aimed to compare and contrast the validity evidence for the assessment of a complex clinical skill based on scores generated from an AI and trained clinical experts, respectively., Methods: The study was conducted between September 2020 to October 2022. The authors used Kane's validity framework to prioritise and organise their evidence according to the four inferences: scoring, generalisation, extrapolation and implications. The context of the study was chorionic villus sampling performed within the simulated setting. AIBA and EBA were used to evaluate performances of experts, intermediates and novice based on video recordings. The clinical experts used a scoring instrument developed in a previous international consensus study. The AI used convolutional neural networks for capturing features on video recordings, motion tracking and eye movements to arrive at a final composite score., Results: A total of 45 individuals participated in the study (22 novices, 12 intermediates and 11 experts). The authors demonstrated validity evidence for scoring, generalisation, extrapolation and implications for both EBA and AIBA. The plausibility of assumptions related to scoring, evidence of reproducibility and relation to different training levels was examined. Issues relating to construct underrepresentation, lack of explainability, and threats to robustness were identified as potential weak links in the AIBA validity argument compared with the EBA validity argument., Conclusion: There were weak links in the use of AIBA compared with EBA, mainly in their representation of the underlying construct but also regarding their explainability and ability to transfer to other datasets. However, combining AI and clinical expert-based assessments may offer complementary benefits, which is a promising subject for future research., (© 2023 The Authors. Medical Education published by Association for the Study of Medical Education and John Wiley & Sons Ltd.)

Background: Training and assessment of operator competence for the less invasive surfactant administration (LISA) procedure vary. This study aimed to obtain international expert consensus on LISA training (LISA curriculum (LISA-CUR)) and assessment (LISA assessment tool (LISA-AT))., Methods: From February to July 2022, an international three-round Delphi process gathered opinions from LISA experts (researchers, curriculum developers, and clinical educators) on a list of items to be included in a LISA-CUR and LISA-AT (Round 1). The experts rated the importance of each item (Round 2). Items supported by more than 80% consensus were included. All experts were asked to approve or reject the final LISA-CUR and LISA-AT (Round 3)., Results: A total of 153 experts from 14 countries participated in Round 1, and the response rate for Rounds 2 and 3 was >80%. Round 1 identified 44 items for LISA-CUR and 22 for LISA-AT. Round 2 excluded 15 items for the LISA-CUR and 7 items for the LISA-AT. Round 3 resulted in a strong consensus (99-100%) for the final 29 items for the LISA-CUR and 15 items for the LISA-AT., Conclusions: This Delphi process established an international consensus on a training curriculum and content evidence for the assessment of LISA competence., Impact: This international consensus-based expert statement provides content on a curriculum for the less invasive surfactant administration procedure (LISA-CUR) that may be partnered with existing evidence-based strategies to optimize and standardize LISA training in the future. This international consensus-based expert statement also provides content on an assessment tool for the LISA procedure (LISA-AT) that can help to evaluate competence in LISA operators. The proposed LISA-AT enables standardized, continuous feedback and assessment until achieving proficiency., (© 2023. The Author(s).)

Purpose: To explore how multiple stakeholder groups contribute to the understanding of trainee competence., Method: The authors conducted a constructivist qualitative study in 2015 using focus group discussions to explore the perceptions of different stakeholder groups (patients, nurses/nurse practitioners, supervisors/senior physicians, leaders/administrators, trainees) regarding trainee competence in the emergency department. The authors used a conventional content analysis, a comparative analysis of supervisors'/senior physicians' versus other stakeholders' perspectives, and a directed analysis informed by stakeholder theory to analyze the focus group transcripts., Results: Forty-six individuals participated in nine focus groups. Four categories of competence were identified: Core Clinical Activities, Patient Centeredness, Aligning Resources, and Code of Conduct. Stakeholders generally agreed in their overall expectations regarding trainee competence. Within individual categories, each stakeholder group identified new considerations, details, and conflicts, which were a replication, elaboration, or complication of a previously identified theme. All stakeholders stressed those aspects of trainee competence that were relevant to their work or values. Trainees were less aware of the patient perspective than that of the other stakeholder groups., Conclusions: Considering multiple stakeholder perspectives enriched the description and conceptualization of trainee competence. It also can inform the development of curricula and assessment tools and guide learning about inter- and intradisciplinary conflicts. Further research should explore how trainees' perceptions of value are influenced by their organizational context and, in particular, how trainees adapt their learning goals in response to the divergent demands of key stakeholders.

Introduction: There is no international consensus on the orientation of transvaginal ultrasound images and no evidence exists to support the superiority of one image orientation over the other. The aim of this study was to compare learning curves and skills transfer in a group of novices randomized to top-down or bottom-up image orientation, and to determine whether individual preferences for image orientation affect learning and skills transfer., Material and Methods: 60 senior medical students, with no prior ultrasound experience, were randomized to orient the image top-down or bottom-up during training on an ultrasound simulator until attaining expert levels of performance. Participants then completed a transfer test involving a systematic ultrasound examination on a physical mannequin using real ultrasound equipment. Performance was assessed during the transfer test by two independent raters using the objective structured assessment of ultrasound skills (OSAUS) score and a global rating score., Results: The bottom-up group reached the expert level with significantly fewer attempts than did the top-down group [median ± interquartile range: 4 ± 2 vs 5 ± 3] (U = 285.5, P = 0.014). The bottom-up group used less time to achieve the expert level (median ± interquartile range: 3 h 2 m ± 1 h 14 m vs 3 h 28 m ± 2 h 21 m) (U = 301.5, P = 0.029). The two groups performed similarly during the transfer test with respect to their OSAUS scores (top-down 56.7% vs bottom-up 53.2%, P = 0.13). The global rating scores were higher in the top-down group (top-down 57.1% vs bottom-up 50.0%, P = 0.02)., Conclusions: Orientation of the images bottom-up rather than top-down, led to a steeper learning curve, but had little or no impact on the subsequent transfer of skills., (© 2018 Nordic Federation of Societies of Obstetrics and Gynecology.)

Objective: To explore the effect of adding simulation-based transvaginal ultrasound training to trainees' clinical training compared with only clinical training on quality of and efficiency of care., Background: Simulation-based ultrasound training may be an effective adjunct to clinical training, but no studies have examined its effects on quality and efficiency of care., Methods: Trainees from 4 University Hospitals in East Denmark were included (N = 54). Participants were randomized to either simulation-based ultrasound training and clinical training (intervention group, n = 28), or to clinical training only (control group, n = 26).The primary outcome was patient-reported discomfort during transvaginal ultrasound examinations performed by study participants. Secondary outcomes included patient-reported perceived safety and confidence in ultrasound provider. Finally, the need for trainee supervision or repeated patient examinations was recorded., Results: In total, 1150 patient ratings were collected. The intervention was associated with a reduction of patient discomfort by 18.5% [95% confidence interval (CI), 10.7-25.5; P < 0.001), and with a 7.9% (95% CI, 0.5-14.7; P = 0.04) increase in perceived safety. The intervention group participants received 11.1% (95% CI, 2.5-18.9) higher scores on patients' confidence compared with control group participants (P = 0.01). When the number of days of clinical training was doubled, the odds for trainee supervision or repeated patient examination was reduced by 45.3% (95% CI, 33.5-55.1) and 19.8% (95% CI, 4.1-32.9) in the intervention and control group, respectively (P = 0.005)., Conclusions: Simulation-based ultrasound training improved quality of care and reduced the need for repeated patient examination and trainee supervision.

Context: Traditionally, trainees are instructed to practise with as few errors as possible during simulation-based training. However, transfer of learning may improve if trainees are encouraged to commit errors. The aim of this study was to assess the effects of error management instructions compared with error avoidance instructions during simulation-based ultrasound training., Methods: Medical students (n = 60) with no prior ultrasound experience were randomised to error management training (EMT) (n = 32) or error avoidance training (EAT) (n = 28). The EMT group was instructed to deliberately make errors during training. The EAT group was instructed to follow the simulator instructions and to commit as few errors as possible. Training consisted of 3 hours of simulation-based ultrasound training focusing on fetal weight estimation. Simulation-based tests were administered before and after training. Transfer tests were performed on real patients 7-10 days after the completion of training. Primary outcomes were transfer test performance scores and diagnostic accuracy. Secondary outcomes included performance scores and diagnostic accuracy during the simulation-based pre- and post-tests., Results: A total of 56 participants completed the study. On the transfer test, EMT group participants attained higher performance scores (mean score: 67.7%, 95% confidence interval [CI]: 62.4-72.9%) than EAT group members (mean score: 51.7%, 95% CI: 45.8-57.6%) (p < 0.001; Cohen's d = 1.1, 95% CI: 0.5-1.7). There was a moderate improvement in diagnostic accuracy in the EMT group compared with the EAT group (16.7%, 95% CI: 10.2-23.3% weight deviation versus 26.6%, 95% CI: 16.5-36.7% weight deviation [p = 0.082; Cohen's d = 0.46, 95% CI: -0.06 to 1.0]). No significant interaction effects between group and performance improvements between the pre- and post-tests were found in either performance scores (p = 0.25) or diagnostic accuracy (p = 0.09)., Conclusions: The provision of error management instructions during simulation-based training improves the transfer of learning to the clinical setting compared with error avoidance instructions. Rather than teaching to avoid errors, the use of errors for learning should be explored further in medical education theory and practice., (© 2016 John Wiley & Sons Ltd and The Association for the Study of Medical Education.)

Objectives: The aim of the study was to explore whether learning curves on a virtual-reality (VR) sonographic simulator can be used to predict subsequent learning curves on a physical mannequin and learning curves during clinical training., Methods: Twenty midwives completed a simulation-based training program in transvaginal sonography. The training was conducted on a VR simulator as well as on a physical mannequin. A subgroup of 6 participants underwent subsequent clinical training. During each of the 3 steps, the participants' performance was assessed using instruments with established validity evidence, and they advanced to the next level only after attaining predefined levels of performance. The number of repetitions and time needed to achieve predefined performance levels were recorded along with the performance scores in each setting. Finally, the outcomes were correlated across settings., Results: A good correlation was found between time needed to achieve predefined performance levels on the VR simulator and the physical mannequin (Pearson correlation coefficient .78; P < .001). Performance scores on the VR simulator correlated well to the clinical performance scores (Pearson correlation coefficient .81; P = .049). No significant correlations were found between numbers of attempts needed to reach proficiency across the 3 different settings. A post hoc analysis found that the 50% fastest trainees at reaching proficiency during simulation-based training received higher clinical performance scores compared to trainees with scores placing them among the 50% slowest (P = .025)., Conclusions: Performances during simulation-based sonography training may predict performance in related tasks and subsequent clinical learning curves., (© 2016 by the American Institute of Ultrasound in Medicine.)

Objective: The aim of this systematic review was to identify studies on hysteroscopic training and assessment., Design: PubMed, Excerpta Medica, the Cochrane Library, and Web of Science were searched in January 2015. Manual screening of references and citation tracking were also performed. Studies on hysteroscopic educational interventions were selected without restrictions on study design, populations, language, or publication year. A qualitative data synthesis including the setting, study participants, training model, training characteristics, hysteroscopic skills, assessment parameters, and study outcomes was performed by 2 authors working independently. Effect sizes were calculated when possible. Overall, 2 raters independently evaluated sources of validity evidence supporting the outcomes of the hysteroscopy assessment tools., Results: A total of 25 studies on hysteroscopy training were identified, of which 23 were performed in simulated settings. Overall, 10 studies used virtual-reality simulators and reported effect sizes for technical skills ranging from 0.31 to 2.65; 12 used inanimate models and reported effect sizes for technical skills ranging from 0.35 to 3.19. One study involved live animal models; 2 studies were performed in clinical settings. The validity evidence supporting the assessment tools used was low. Consensus between the 2 raters on the reported validity evidence was high (94%)., Conclusions: This systematic review demonstrated large variations in the effect of different tools for hysteroscopy training. The validity evidence supporting the assessment of hysteroscopic skills was limited., (Copyright © 2016 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.)

Objectives: This study is designed to provide an overview of why, how, when and for whom collaborative learning of clinical skills may work in health professions education., Why: Collaborative learning of clinical skills may influence learning positively according to the non-medical literature. Training efficiency may therefore be improved if the outcomes of collaborative learning of clinical skills are superior or equivalent to those attained through individual learning., How: According to a social interaction perspective, collaborative learning of clinical skills mediates its effects through social interaction, motivation, accountability and positive interdependence between learners. Motor skills learning theory suggests that positive effects rely on observational learning and action imitation, and negative effects may include decreased hands-on experience. Finally, a cognitive perspective suggests that learning is dependent on cognitive co-construction, shared knowledge and reduced cognitive load., When and for Whom: The literature on the collaborative learning of clinical skills in health science education is reviewed to support or contradict the hypotheses provided by the theories outlined above. Collaborative learning of clinical skills leads to improvements in self-efficacy, confidence and performance when task processing is observable or communicable. However, the effects of collaborative learning of clinical skills may decrease over time as benefits in terms of shared cognition, scaffolding and cognitive co-construction are outweighed by reductions in hands-on experience and time on task., Conclusions: Collaborative learning of clinical skills has demonstrated promising results in the simulated setting. However, further research into how collaborative learning of clinical skills may work in clinical settings, as well as into the role of social dynamics between learners, is required., (© 2015 John Wiley & Sons Ltd.)

Context: Dyad practice may be as effective as individual practice during clinical skills training, improve students' confidence, and reduce costs of training. However, there is little evidence that dyad training is non-inferior to single-student practice in terms of skills transfer., Objectives: This study was conducted to compare the effectiveness of simulation-based ultrasound training in pairs (dyad practice) with that of training alone (single-student practice) on skills transfer., Methods: In a non-inferiority trial, 30 ultrasound novices were randomised to dyad (n = 16) or single-student (n = 14) practice. All participants completed a 2-hour training programme on a transvaginal ultrasound simulator. Participants in the dyad group practised together and took turns as the active practitioner, whereas participants in the single group practised alone. Performance improvements were evaluated through pre-, post- and transfer tests. The transfer test involved the assessment of a transvaginal ultrasound scan by one of two clinicians using the Objective Structured Assessment of Ultrasound Skills (OSAUS)., Results: Thirty participants completed the simulation-based training and 24 of these completed the transfer test. Dyad training was found to be non-inferior to single-student training: transfer test OSAUS scores were significantly higher than the pre-specified non-inferiority margin (delta score 7.8%, 95% confidence interval -3.8-19.6%; p = 0.04). More dyad (71.4%) than single (30.0%) trainees achieved OSAUS scores above a pre-established pass/fail level in the transfer test (p = 0.05). There were significant differences in performance scores before and after training in both groups (pre- versus post-test, p < 0.01) with large effect sizes (Cohen's d = 3.85) and no significant interactions between training type and performance (p = 0.59). The dyad group demonstrated higher training efficiency in terms of simulator score per number of attempts compared with the single-student group (p = 0.03)., Conclusion: Dyad practice improves the efficiency of simulation-based training and is non-inferior to individual practice in terms of skills transfer., (© 2015 John Wiley & Sons Ltd.)

Background: This study explored individual, team, and setting factors associated with the quality of management of in-hospital emergency situations experienced by former Advanced Life Support (ALS) course participants., Methods: This study was a survey of former ALS course participants' long-term experience of management of in-hospital, emergency situations. The survey was carried out in 2012 in Denmark and Norway., Results: A questionnaire was send to 526 potential responders and (281/479 × 100) 58.7% responded. The results demonstrated that 75% of the emergency situations were perceived as "managed well". In general, the responders' confidence in being ALS providers was high, mean 4.3 (SD 0.8), scale 1-5. Significant differences between the perceived "well" and "not well" managed situations were found for all questions, p<0.001. The largest differences related to perception of co-workers' ability to apply ALS principles, the team atmosphere and communication. Responders' ratings of quality of management of emergency situations increased with intensity of setting. However, the 'clinical setting' was rated significantly lower as attributor to ability to apply ALS principles compared to 'co-workers familiarity with ALS principles', 'own confidence as ALS-provider' and 'own social/inter-personal skills'., Conclusion: The results of this survey emphasise that ALS providers' perceived ability to apply ALS skills were substantially affected by teamwork skills and co-workers' skills. Team related factors associated with successful outcome were related to clear role distribution, clear inter-personal communication and attentive listening, as well as respectful behaviour and positive team atmosphere. Although intensity of setting was attributed to ability to apply ALS principles, this did not affect management of emergency situations to the same extent as individual and team factors., (Copyright © 2014 Elsevier Ireland Ltd. All rights reserved.)

According to the Danish Ministry of Health recommendations, workplace-based assessment (WPBA) should be a central component in postgraduate specialty training. In this review the potential impact of WPBA with regard to patient safety, medical licensing and effects on doctors' learning is explored. The utility of WPBA instruments is reviewed with respect to their reliability, validity, cost, acceptability, and educational impact. Finally, the concept of Entrusted Professional Activities is discussed in view of the utility framework, and recommendations for future practice are provided.

Background: The rising number of medical students and the impact this has on students' learning of clinical skills is a matter of concern. Cooperative learning in pairs, called dyad training, might help address this situation., Objective: The aim of this study was to evaluate the effect of dyad training on students' patient encounter skills., Design: Experimental, randomized, observer-blinded trial., Participants: Forty-nine pre-clerkship medical students without prior clinical experience., Intervention: All students underwent a 4-h course on how to manage patient encounters. Subsequently, the students were randomized into a dyad practice group (n = 24) or a single practice group (n = 25). Both groups practiced for 4 h on four different case scenarios, using simulated patients. Students in the dyad group practiced together and took turns as the active participant, whereas students in the single group practiced alone., Main Measure: Performance tests of patient encounter skills were conducted 2 weeks after the training by two blinded raters. Students had no clinical training during those weeks. A questionnaire-based evaluation surveyed students' confidence in their patient management skills., Key Results: The dyad group scored significantly higher on the performance test, mean 40.7 % (SD 6.6), than the single group, mean 36.9 % (SD 5.8), P = 0.04, effect size 0.61. Inter-rater reliability was 0.69. The dyad group expressed significantly higher confidence in managing future clinical patient encounters than the single group, mean 7.6 (SD 0.9) vs. mean 6.5 (SD 1.1), respectively, P < 0.001, effect size 1.16., Conclusion: Dyad training of pre-clerkship medical students' patient encounter skills is effective, efficient, and prompts higher confidence in managing future patient encounters compared to training alone. This training format may help maintain high-quality medical training in the face of an increasing number of students in medical schools.

Aim: The aim of this study is to compare student teachers and clinical associate professors regarding the quality of procedural skills teaching in terms of participants' technical skills, knowledge and satisfaction with the teaching., Methods: This is an experimental, randomized, controlled study comparing the teaching of student teachers and associate professors regarding participants' learning outcome and satisfaction with the teaching. Two skills are chosen for the experiment, i.v.-access and bladder catheterization. Learning outcome is assessed by a pre- and post testing of the participants' knowledge and skills. Participants evaluate satisfaction with teaching on nine statements immediately after the teaching., Results: In total 59 first year medical students are included as participants in the experiment. The students taught by student teachers perform just as well as the students taught by associate professors and in one skill--catheterization--they perform even better, mean post- minus pre-test scores 65.5 (SD 12.9) vs. 35.0 (SD 23.3), One-way ANOVA, p < 0.0001, effect size 1.62. Student teachers receive significantly more positive evaluations than associate professors on several statements., Conclusion: Trained student teachers can be as good as associate professors in teaching clinical skills.

Over the past decade, ultrasound simulators have been increasingly used as an adjunct to clinical training in Obstetrics and Gynaecology. Studies have shown that providing initial simulation-based ultrasound training leads to sustained improvements in clinical performances1 including improved diagnostic accuracy,2 reduced need for supervised practice, and decreased patient discomfort.3. [ABSTRACT FROM AUTHOR]