Search

This study aimed to develop a deep learning model to assess the quality of fetal echocardiography and to perform prospective clinical validation. The model was trained on data from the 18-22-week anomaly scan conducted in seven hospitals from 2008 to 2018. Prospective validation involved 100 patients from two hospitals. A total of 5363 images from 2551 pregnancies were used for training and validation. The model's segmentation accuracy depended on image quality measured by a quality score (QS). It achieved an overall average accuracy of 0.91 (SD 0.09) across the test set, with images having above-average QS scoring 0.97 (SD 0.03). During prospective validation of 192 images, clinicians rated 44.8% (SD 9.8) of images as equal in quality, 18.69% (SD 5.7) favoring auto-captured images and 36.51% (SD 9.0) preferring manually captured ones. Images with above average QS showed better agreement on segmentations (p

Introduction: Simulation-based training (SBT) aids healthcare providers in acquiring the technical skills necessary to improve patient outcomes and safety. However, since SBT may require significant resources, training all skills to a comparable extent is impractical. Hence, a strategic prioritization of technical skills is necessary. While the European Training Requirements in Neonatology provide guidance on necessary skills, they lack prioritization. We aimed to identify and prioritize technical skills for a SBT curriculum in neonatology. Methods: A three-round modified Delphi process of expert neonatologists and neonatal trainees was performed. In round one, the participants listed all the technical skills newly trained neonatologists should master. The content analysis excluded duplicates and non-technical skills. In round two, the Copenhagen Academy for Medical Education and Simulation Needs Assessment Formula (CAMES-NAF) was used to preliminarily prioritize the technical skills according to frequency, importance of competency, SBT impact on patient safety, and feasibility for SBT. In round three, the participants further refined and reprioritized the technical skills. Items achieving consensus (agreement of ≥75%) were included. Results: We included 168 participants from 10 European countries. The response rates in rounds two and three were 80% (135/168) and 87% (117/135), respectively. In round one, the participants suggested 1964 different items. Content analysis revealed 81 unique technical skills prioritized in round two. In round three, 39 technical skills achieved consensus and were included. Conclusion: We reached a European consensus on a prioritized list of 39 technical skills to be included in a SBT curriculum in neonatology., INTRODUCTION: Simulation-based training (SBT) aids healthcare providers in acquiring the technical skills necessary to improve patient outcomes and safety. However, since SBT may require significant resources, training all skills to a comparable extent is impractical. Hence, a strategic prioritization of technical skills is necessary. While the European Training Requirements in Neonatology provide guidance on necessary skills, they lack prioritization. We aimed to identify and prioritize technical skills for a SBT curriculum in neonatology.METHODS: A three-round modified Delphi process of expert neonatologists and neonatal trainees was performed. In round one, the participants listed all the technical skills newly trained neonatologists should master. The content analysis excluded duplicates and non-technical skills. In round two, the Copenhagen Academy for Medical Education and Simulation Needs Assessment Formula (CAMES-NAF) was used to preliminarily prioritize the technical skills according to frequency, importance of competency, SBT impact on patient safety, and feasibility for SBT. In round three, the participants further refined and reprioritized the technical skills. Items achieving consensus (agreement of ≥75%) were included.RESULTS: We included 168 participants from 10 European countries. The response rates in rounds two and three were 80% (135/168) and 87% (117/135), respectively. In round one, the participants suggested 1964 different items. Content analysis revealed 81 unique technical skills prioritized in round two. In round three, 39 technical skills achieved consensus and were included.CONCLUSION: We reached a European consensus on a prioritized list of 39 technical skills to be included in a SBT curriculum in neonatology.

Background: Artificial intelligence (AI) is becoming increasingly used in medical education, but our understanding of the validity of AI-based assessments (AIBA) as compared with traditional clinical expert-based assessments (EBA) is limited. In this study, the authors aimed to compare and contrast the validity evidence for the assessment of a complex clinical skill based on scores generated from an AI and trained clinical experts, respectively. Methods: The study was conducted between September 2020 to October 2022. The authors used Kane's validity framework to prioritise and organise their evidence according to the four inferences: scoring, generalisation, extrapolation and implications. The context of the study was chorionic villus sampling performed within the simulated setting. AIBA and EBA were used to evaluate performances of experts, intermediates and novice based on video recordings. The clinical experts used a scoring instrument developed in a previous international consensus study. The AI used convolutional neural networks for capturing features on video recordings, motion tracking and eye movements to arrive at a final composite score. Results: A total of 45 individuals participated in the study (22 novices, 12 intermediates and 11 experts). The authors demonstrated validity evidence for scoring, generalisation, extrapolation and implications for both EBA and AIBA. The plausibility of assumptions related to scoring, evidence of reproducibility and relation to different training levels was examined. Issues relating to construct underrepresentation, lack of explainability, and threats to robustness were identified as potential weak links in the AIBA validity argument compared with the EBA validity argument. Conclusion: There were weak links in the use of AIBA compared with EBA, mainly in their representation of the underlying construct but also regarding their explainability and ability to transfer to other datasets. However, combining AI and clinical e

To collect validity evidence for the assessment of surgical competence through the classification of general surgical gestures for a simulated robot-assisted radical prostatectomy (RARP). We used 165 video recordings of novice and experienced RARP surgeons performing three parts of the RARP procedure on the RobotiX Mentor. We annotated the surgical tasks with different surgical gestures: dissection, hemostatic control, application of clips, needle handling, and suturing. The gestures were analyzed using idle time (periods with minimal instrument movements) and active time (whenever a surgical gesture was annotated). The distribution of surgical gestures was described using a one-dimensional heat map, snail tracks. All surgeons had a similar percentage of idle time but novices had longer phases of idle time (mean time: 21 vs. 15 s, p < 0.001). Novices used a higher total number of surgical gestures (number of phases: 45 vs. 35, p < 0.001) and each phase was longer compared with those of the experienced surgeons (mean time: 10 vs. 8 s, p < 0.001). There was a different pattern of gestures between novices and experienced surgeons as seen by a different distribution of the phases. General surgical gestures can be used to assess surgical competence in simulated RARP and can be displayed as a visual tool to show how performance is improving. The established pass/fail level may be used to ensure the competence of the residents before proceeding with supervised real-life surgery. The next step is to investigate if the developed tool can optimize automated feedback during simulator training.