Search

Background: Artificial intelligence (AI) is becoming increasingly used in medical education, but our understanding of the validity of AI‐based assessments (AIBA) as compared with traditional clinical expert‐based assessments (EBA) is limited. In this study, the authors aimed to compare and contrast the validity evidence for the assessment of a complex clinical skill based on scores generated from an AI and trained clinical experts, respectively. Methods: The study was conducted between September 2020 to October 2022. The authors used Kane's validity framework to prioritise and organise their evidence according to the four inferences: scoring, generalisation, extrapolation and implications. The context of the study was chorionic villus sampling performed within the simulated setting. AIBA and EBA were used to evaluate performances of experts, intermediates and novice based on video recordings. The clinical experts used a scoring instrument developed in a previous international consensus study. The AI used convolutional neural networks for capturing features on video recordings, motion tracking and eye movements to arrive at a final composite score. Results: A total of 45 individuals participated in the study (22 novices, 12 intermediates and 11 experts). The authors demonstrated validity evidence for scoring, generalisation, extrapolation and implications for both EBA and AIBA. The plausibility of assumptions related to scoring, evidence of reproducibility and relation to different training levels was examined. Issues relating to construct underrepresentation, lack of explainability, and threats to robustness were identified as potential weak links in the AIBA validity argument compared with the EBA validity argument. Conclusion: There were weak links in the use of AIBA compared with EBA, mainly in their representation of the underlying construct but also regarding their explainability and ability to transfer to other datasets. However, combining AI and clinical expert‐based assessments may offer complementary benefits, which is a promising subject for future research. Is artificial intelligence (AI) ready to replace clinicians in clinical skills assessment? In this paper, Johnsson et al. use Kane's framework to compare validity evidence between the two. [ABSTRACT FROM AUTHOR]

Introduction: During a health crisis, hospitals must prioritise activities and resources, which can compromise clerkship‐based learning. We explored how health crises affect clinical clerkships using the COVID‐19 pandemic as an example. Methods: In a constructivist qualitative study, we conducted 22 semi‐structured interviews with key stakeholders (i.e. medical students and doctors) from two teaching hospitals and 10 different departments. We used thematic analysis to investigate our data and used stakeholder theory as a sensitising concept. Results: We identified three themes: (1) emotional triggers and reactions; (2) negotiation of legitimacy; and (3) building resilience. Our results suggest that the health crisis accentuated already existing problems in clerkships, such as students' feelings of low legitimacy, constant negotiation of roles, inconsistencies navigating rules and regulations and low levels of active participation. Medical students and doctors adapted to the new organisational demands by developing increased resilience. Students responded by reaching out for guidance and acceptance to remain relevant in the clinical clerkships. Doctors developed a behaviour of closing in and focused on managing themselves and their patients. This created tension between these two stakeholder groups. Conclusion: A health crisis can critically disrupt the hierarchical structure within the clinical clerkships and exacerbate existing conflicts between stakeholder groups. When medical students are not perceived as legitimate stakeholders in clinical clerkships during a health crisis, their attendance is perceived as unnecessary or even a nuisance. Despite increased student proactiveness and resilience, their roles inevitably shift from being doctors‐to‐be to students‐to‐be‐managed. This study reports that when medical students are not perceived as legitimate stakeholders in clinical clerkships during a health crisis, their roles shift from being doctors‐to‐be to students‐to‐be‐managed. [ABSTRACT FROM AUTHOR]

Introduction: Dyad learning occurs when two students work together to acquire new skills and knowledge. Several studies have provided evidence to support the educational rationale for dyad learning in the controlled simulated setting. However, the role of dyad learning in the clinical setting remains uncertain. Unlike the simulated setting, learning in the clinical setting depends on a complex interplay between medical students, doctors, nurses and patients potentially making dyad learning less valuable in clerkships. The objective of this study was to explore how key stakeholders perceive the value of implementing dyad learning during medical students' clinical clerkships. Methods: In a constructivist qualitative study, we conducted 51 semi‐structured interviews with 36 key stakeholders involved in dyad learning, including 10 medical students, 12 doctors, five nurses and nine patients. Data were coded inductively using thematic analysis, then coded deductively using stakeholder theory as a theoretical framework. Results: We found that stakeholders generally perceived the educational impact of dyad learning in the clinical setting similarly but disagreed on its value. Students emphasised that dyad learning made them participate more actively during patient encounters and patients did not mind having two students present. Doctors and nurses considered dyad learning disruptive to the balance between service and training and reported that it did not resonate with their perception of good patient care. Conclusion: Dyad learning enables students to be more active during their clinical clerkships, but it easily disrupts the balance between service and training. This disruption may be exacerbated by the shifted balance in priorities and values between different stakeholder groups, as well as by making implicit teaching obligations more explicit for supervising doctors and nurses. Consequently, implementing dyad learning may not be perceived as valuable by doctors and nurses in the clinical setting, regardless of its pedagogical rationale. The authors demonstrate that collaborative learning between pairs of medical students is perceived and valued differently by different stakeholders in the clinical setting, particularly with respect to tge patient perspective. [ABSTRACT FROM AUTHOR]

Context: A long‐standing myth in medical education research is a divide between two different poles: research aiming to advance theory with little focus on practical applications ('ivory tower' research) and practically oriented research aiming to serve educators and decision‐makers with little focus on advancing theory ('in‐the‐trenches' practice). We explored this myth in a sample of randomised medical education studies using Stokes' four‐quadrant framework for the classification of research perspective. Methods: We searched MEDLINE, Embase, CINAHL, PsychINFO, ERIC, Web of Science and Scopus for studies in medical education using a randomised design that were published between 1 January 2018 and 31 December 2018. We used Stokes' four‐quadrant framework to categorise the studies according to their use of theory, concepts and their justification for practical use. We compared medical education research published in medical education journals and clinical journals. Results: A total of 150 randomised studies were included in the analysis. The largest segment of studies (46.7%) was categorised as use‐inspired basic research (Pasteur's Quadrant), closely followed by pure applied research (40.7%, Edison's Quadrant). Only a few studies were categorised as aiming to advance knowledge with no thought for practical educational application (2.0%, Bohr's Quadrant). The proportion of studies that included educational concepts and theory differed according to publication in clinical journals or medical education journals: 40.5% vs 71.8%, respectively, P <.001. There were no differences between journals with regard to the proportion of studies that included a practical educational or clinical rationale (P =.99). Conclusion: In a large sample of studies using randomised designs, we found no evidence to support the myth that medical education research divides between two singular poles represented by 'ivory tower research' and 'in‐the‐trenches practice'. We did confirm prevailing assumptions regarding an emphasis on non‐theoretical medical education research in clinical journals. Countering claims of tension between theory and practice, Tolsgaard et al. empirically address the myth that health professional education divide easily into 'ivory tower research' or 'in‐the‐trenches practice' [ABSTRACT FROM AUTHOR]

Introduction: There is no international consensus on the orientation of transvaginal ultrasound images and no evidence exists to support the superiority of one image orientation over the other. The aim of this study was to compare learning curves and skills transfer in a group of novices randomized to top-down or bottom-up image orientation, and to determine whether individual preferences for image orientation affect learning and skills transfer.Material and Methods: 60 senior medical students, with no prior ultrasound experience, were randomized to orient the image top-down or bottom-up during training on an ultrasound simulator until attaining expert levels of performance. Participants then completed a transfer test involving a systematic ultrasound examination on a physical mannequin using real ultrasound equipment. Performance was assessed during the transfer test by two independent raters using the objective structured assessment of ultrasound skills (OSAUS) score and a global rating score.Results: The bottom-up group reached the expert level with significantly fewer attempts than did the top-down group [median ± interquartile range: 4 ± 2 vs 5 ± 3] (U = 285.5, P = 0.014). The bottom-up group used less time to achieve the expert level (median ± interquartile range: 3 h 2 m ± 1 h 14 m vs 3 h 28 m ± 2 h 21 m) (U = 301.5, P = 0.029). The two groups performed similarly during the transfer test with respect to their OSAUS scores (top-down 56.7% vs bottom-up 53.2%, P = 0.13). The global rating scores were higher in the top-down group (top-down 57.1% vs bottom-up 50.0%, P = 0.02).Conclusions: Orientation of the images bottom-up rather than top-down, led to a steeper learning curve, but had little or no impact on the subsequent transfer of skills. [ABSTRACT FROM AUTHOR]

Context Traditionally, trainees are instructed to practise with as few errors as possible during simulation-based training. However, transfer of learning may improve if trainees are encouraged to commit errors. The aim of this study was to assess the effects of error management instructions compared with error avoidance instructions during simulation-based ultrasound training. Methods Medical students ( n = 60) with no prior ultrasound experience were randomised to error management training ( EMT) ( n = 32) or error avoidance training ( EAT) ( n = 28). The EMT group was instructed to deliberately make errors during training. The EAT group was instructed to follow the simulator instructions and to commit as few errors as possible. Training consisted of 3 hours of simulation-based ultrasound training focusing on fetal weight estimation. Simulation-based tests were administered before and after training. Transfer tests were performed on real patients 7-10 days after the completion of training. Primary outcomes were transfer test performance scores and diagnostic accuracy. Secondary outcomes included performance scores and diagnostic accuracy during the simulation-based pre- and post-tests. Results A total of 56 participants completed the study. On the transfer test, EMT group participants attained higher performance scores (mean score: 67.7%, 95% confidence interval [ CI]: 62.4-72.9%) than EAT group members (mean score: 51.7%, 95% CI: 45.8-57.6%) (p < 0.001; Cohen's d = 1.1, 95% CI: 0.5-1.7). There was a moderate improvement in diagnostic accuracy in the EMT group compared with the EAT group (16.7%, 95% CI: 10.2-23.3% weight deviation versus 26.6%, 95% CI: 16.5-36.7% weight deviation [p = 0.082; Cohen's d = 0.46, 95% CI: −0.06 to 1.0]). No significant interaction effects between group and performance improvements between the pre- and post-tests were found in either performance scores (p = 0.25) or diagnostic accuracy (p = 0.09). Conclusions The provision of error management instructions during simulation-based training improves the transfer of learning to the clinical setting compared with error avoidance instructions. Rather than teaching to avoid errors, the use of errors for learning should be explored further in medical education theory and practice. [ABSTRACT FROM AUTHOR]

Context Concerns have been raised over the gap between education theory and practice and how research can contribute to inform decision makers on their choices and priorities. Little is known about how educational theories and research outcomes produced under optimal conditions in highly controlled settings generalise to the real-life education context. One way of bridging this gap is applying the concept of practical trials in medical education. In this paper we elaborate on characteristics of practical trials and based on examples from medical education we discuss the challenges, limitations and future directions for this kind of research. Current State Practical trials have the overall aim of informing decision makers. They are carried out in real-life settings and are characterised by (i) comparison of viable alternative education strategies, (ii) broad inclusion criteria regarding participants across several settings and (iii) multiple outcome measures with long-term follow-up to evaluate both benefits and risks. Questions posed by practical trials may be proactive in applying theory in the development of educational innovations or reactive to educational reforms and innovations. Non-inferiority or equivalence designs are recommended when comparing viable alternatives and the use of crossover designs, cluster randomisation or stepped wedge trial designs are feasible when studying implementations across several settings. Outcome measures may include variables related to learners, teachers, educational administration, quality of care, patient outcomes and cost. Conclusions Practical trials in medical education may contribute to bridge the gap between education theory and practice and aid decision makers in making evidence-based choices and priorities. Conducting practical trials is not without challenges and rigorous design and methods must be applied. Of concern is that the practical focus may lead to failure to include a sound theoretical basis in the research questions and the interventions studied, and that authors fail to obtain informed consent from their participants. [ABSTRACT FROM AUTHOR]

Objectives: The aim of the study was to explore whether learning curves on a virtual‐reality (VR) sonographic simulator can be used to predict subsequent learning curves on a physical mannequin and learning curves during clinical training. Methods: Twenty midwives completed a simulation‐based training program in transvaginal sonography. The training was conducted on a VR simulator as well as on a physical mannequin. A subgroup of 6 participants underwent subsequent clinical training. During each of the 3 steps, the participants' performance was assessed using instruments with established validity evidence, and they advanced to the next level only after attaining predefined levels of performance. The number of repetitions and time needed to achieve predefined performance levels were recorded along with the performance scores in each setting. Finally, the outcomes were correlated across settings. Results: A good correlation was found between time needed to achieve predefined performance levels on the VR simulator and the physical mannequin (Pearson correlation coefficient .78; P < .001). Performance scores on the VR simulator correlated well to the clinical performance scores (Pearson correlation coefficient .81; P = .049). No significant correlations were found between numbers of attempts needed to reach proficiency across the 3 different settings. A post hoc analysis found that the 50% fastest trainees at reaching proficiency during simulation‐based training received higher clinical performance scores compared to trainees with scores placing them among the 50% slowest (P = .025). Conclusions: Performances during simulation‐based sonography training may predict performance in related tasks and subsequent clinical learning curves. [ABSTRACT FROM AUTHOR]

Objectives This study is designed to provide an overview of why, how, when and for whom collaborative learning of clinical skills may work in health professions education. Why Collaborative learning of clinical skills may influence learning positively according to the non-medical literature. Training efficiency may therefore be improved if the outcomes of collaborative learning of clinical skills are superior or equivalent to those attained through individual learning. How According to a social interaction perspective, collaborative learning of clinical skills mediates its effects through social interaction, motivation, accountability and positive interdependence between learners. Motor skills learning theory suggests that positive effects rely on observational learning and action imitation, and negative effects may include decreased hands-on experience. Finally, a cognitive perspective suggests that learning is dependent on cognitive co-construction, shared knowledge and reduced cognitive load. When and for whom The literature on the collaborative learning of clinical skills in health science education is reviewed to support or contradict the hypotheses provided by the theories outlined above. Collaborative learning of clinical skills leads to improvements in self-efficacy, confidence and performance when task processing is observable or communicable. However, the effects of collaborative learning of clinical skills may decrease over time as benefits in terms of shared cognition, scaffolding and cognitive co-construction are outweighed by reductions in hands-on experience and time on task. Conclusions Collaborative learning of clinical skills has demonstrated promising results in the simulated setting. However, further research into how collaborative learning of clinical skills may work in clinical settings, as well as into the role of social dynamics between learners, is required. [ABSTRACT FROM AUTHOR]

Objective To provide a model for conducting cost-effectiveness analyses in medical education. The model was based on a randomised trial examining the effects of training midwives to perform cervical length measurement (CLM) as compared with obstetricians on patients' waiting times. ( CLM), as compared with obstetricians. Methods The model included four steps: (i) gathering data on training outcomes, (ii) assessing total costs and effects, (iii) calculating the incremental cost-effectiveness ratio ( ICER) and (iv) estimating cost-effectiveness probability for different willingness to pay ( WTP) values. To provide a model example, we conducted a randomised cost-effectiveness trial. Midwives were randomised to CLM training (midwife-performed CLMs) or no training (initial management by midwife, and CLM performed by obstetrician). Intervention-group participants underwent simulation-based and clinical training until they were proficient. During the following 6 months, waiting times from arrival to admission or discharge were recorded for women who presented with symptoms of pre-term labour. Outcomes for women managed by intervention and control-group participants were compared. These data were then used for the remaining steps of the cost-effectiveness model. Results Intervention-group participants needed a mean 268.2 (95% confidence interval [ CI], 140.2-392.2) minutes of simulator training and a mean 7.3 (95% CI, 4.4-10.3) supervised scans to attain proficiency. Women who were scanned by intervention-group participants had significantly reduced waiting time compared with those managed by the control group ( n = 65; mean difference, 36.6 [95% CI 7.3-65.8] minutes; p = 0.008), which corresponded to an ICER of 0.45 EUR minute−1. For WTP values less than EUR 0.26 minute−1, obstetrician-performed CLM was the most cost-effective strategy, whereas midwife-performed CLM was cost-effective for WTP values above EUR 0.73 minute−1. Conclusion Cost-effectiveness models can be used to link quality of care to training costs. The example used in the present study demonstrated that different training strategies could be recommended as the most cost-effective depending on administrators' willingness to pay per unit of the outcome variable. [ABSTRACT FROM AUTHOR]

Over the past decade, ultrasound simulators have been increasingly used as an adjunct to clinical training in Obstetrics and Gynaecology. Studies have shown that providing initial simulation-based ultrasound training leads to sustained improvements in clinical performances1 including improved diagnostic accuracy,2 reduced need for supervised practice, and decreased patient discomfort.3. [ABSTRACT FROM AUTHOR]

Liv and Tolsgaard examine the literature on transfer and suggest that context is more than a barrier to be overcome. [ABSTRACT FROM AUTHOR]

Context Dyad practice may be as effective as individual practice during clinical skills training, improve students' confidence, and reduce costs of training. However, there is little evidence that dyad training is non-inferior to single-student practice in terms of skills transfer. Objectives This study was conducted to compare the effectiveness of simulation-based ultrasound training in pairs (dyad practice) with that of training alone (single-student practice) on skills transfer. Methods In a non-inferiority trial, 30 ultrasound novices were randomised to dyad ( n = 16) or single-student ( n = 14) practice. All participants completed a 2-hour training programme on a transvaginal ultrasound simulator. Participants in the dyad group practised together and took turns as the active practitioner, whereas participants in the single group practised alone. Performance improvements were evaluated through pre-, post- and transfer tests. The transfer test involved the assessment of a transvaginal ultrasound scan by one of two clinicians using the Objective Structured Assessment of Ultrasound Skills ( OSAUS). Results Thirty participants completed the simulation-based training and 24 of these completed the transfer test. Dyad training was found to be non-inferior to single-student training: transfer test OSAUS scores were significantly higher than the pre-specified non-inferiority margin (delta score 7.8%, 95% confidence interval −3.8-19.6%; p = 0.04). More dyad (71.4%) than single (30.0%) trainees achieved OSAUS scores above a pre-established pass/fail level in the transfer test (p = 0.05). There were significant differences in performance scores before and after training in both groups (pre- versus post-test, p < 0.01) with large effect sizes (Cohen's d = 3.85) and no significant interactions between training type and performance (p = 0.59). The dyad group demonstrated higher training efficiency in terms of simulator score per number of attempts compared with the single-student group (p = 0.03). Conclusion Dyad practice improves the efficiency of simulation-based training and is non-inferior to individual practice in terms of skills transfer. [ABSTRACT FROM AUTHOR]

The author discusses the costs and economic aspects of medical education. Topics include the costs to stakeholders such as government, students, and educational institutions associated with failure in health profession education, the use of summative evaluations to determine whether educational programs work and formative evaluations to determine how to improve educational programs, and the consideration of a program's value when performing a cost analysis.

An introduction is presented in which the author discusses various reports within the issue on topics including the opportunities and challenges related on medical education, clarity as an essential tool in communication and the proper ways to write an article.

A letter to the editor is presented which discusses improving the methodological rigor of medical education trials.