Search

The evaluation of teaching can be an essential driver for curriculum development. Instruments for teaching evaluation are not only used for the purpose of quality assurance but also in the context of medical education research. Therefore, they must meet the common requirements for reliability and validity. This position paper from the GMA Teaching Evaluation Committee discusses strategic and methodological aspects of evaluation in the context of undergraduate medical education and related courses; and formulates recommendations for the further development of evaluation. First, a four-step approach to the design and implementation of evaluations is presented, then methodological and practical aspects are discussed in more detail. The focus here is on target and confounding variables, survey instruments as well as aspects of implementation and data protection. Finally, possible consequences from evaluation data for the four dimensions of teaching quality (structural and procedural aspects, teachers and outcomes) are discussed.

Background: Serious games are risk-free environments training various medical competencies, such as clinical reasoning, without endangering patients' safety. Furthermore, serious games provide a context for training situations with unpredictable outcomes. Training these competencies is particularly important for healthcare professionals in emergency medicine., Methods: Based on these considerations, we designed, implemented, and evaluated a serious game in form of an emergency department, containing the features of a virtual patient generator, a chatbot for medical history taking with self-formulated questions, artificially generated faces based on an artificial intelligence algorithm, and feedback for students. The development process was based on an already existing framework resulting in an iterative procedure between development and evaluation. The serious game was evaluated using the System Usability Scale and the User Experience Questionnaire., Results: The System Usability Scale provided a substantial result for the usability. In terms of the user experience, four scales yielded positive results, whereas two scales yielded neutral results., Conclusion: The evaluation of both usability and user experience yielded overall positive results, while simultaneously identifying potential areas for improvement. Further studies will address the implementation of additional game design elements, and testing student learning outcome., (© 2024. The Author(s).)

Background: Time-since-death (TSD) diagnostics are crucial in forensic medical casework. The compound method by Henssge and Madea, which combines temperature and non-temperature-based techniques, is widely used to estimate TSD. This study aims to validate the predictive ability of this method in a cohort of 76 deceased individuals with known times of death (TOD)., Methods: A convenience sample of 76 deceased individuals was examined at the Institute of Legal Medicine, University Medical Center Hamburg-Eppendorf. The study included individuals who died at the hospital and those with sudden death in public. Exclusion criteria included age under 18, known infection or sepsis, polytrauma, bleeding, and hyperthermia. The TSD interval was calculated using the Deathtime software., Results: The overall agreement between the actual TOD and the 95% prediction interval for the TSD was 36.8% (95% CI: 26.1 to 48.7). Warm-stored corpses showed a higher agreement (61.9% [95% CI: 38.4 to 81.9]) compared to cold-stored corpses (27.3% [95% CI: 16.1 to 41.0]). Factors such as body mass index (BMI) and body surface area (BSA) were found to influence the odds of agreement. Assuming a plausible range of ambient temperatures between death and admission improved the agreement in cold-stored cases., Conclusion: The study found low to moderate agreement between the actual TOD and the 95% prediction interval using the Henssge method. Incorporating BMI and BSA could improve the predictive accuracy of TSD estimations. Further research with larger sample sizes and external validation is recommended to refine the model., (© 2024. The Author(s).)

Background: The Masterplan Medicine 2020 adopted in 2017 entails many changes to the medical studies curriculum. The new structure affects the content of the coursework and its main focus. A major goal of this masterplan is to prepare young physicians by teaching the skills that are essential for the future profession. The National Competence-Based Learning Objectives Catalog for Medicine (NKLM) provides the basis for the teaching content., Methods: The Working Group Teaching of the German Society of Ophthalmology (Arbeitsgemeinschaft Lehre der DOG) actively supports this transformation. In cross-site collaboration, teaching materials for various teaching formats have been compiled by relying on the NKLM (e.g., recordings of operations, slides for small group instruction, image galleries, case studies). An online library named the DOG-EyeTeacher was then created., Results: The aim of the DOG-EyeTeacher is to relieve the training clinics and to establish basic standards in teaching materials, thereby enabling the necessary focus on medical education. The provision of these teaching materials should deepen the interest in ophthalmology among future doctors., Conclusion: The DOG-EyeTeacher is our response to the challenges associated with the planned restructuring of the medical curriculum. Since October 2023, any DOG member involved in teaching can create an account free of charge to use our materials., (© 2024. The Author(s).)

Background: Despite the challenging curriculum, medicine is a popular study program. We propose McClelland's Motive Disposition Theory (MDT) as a possible theory for explaining medical students motivation. The theory describes how individuals differ in their behaviour due to their varying manifestations of certain motives. The three motives can thus influence the students behaviour and academic success. Using these motives, complimented with an altruism- and a freedom motive, this study was aimed at investigating young adults' explicit motives to study medicine. In addition, we also wanted to find out whether there are gender differences in motives and other variables such as empathy, emotional intelligence and academic self-concept., Methods: Over 20 universities across Germany were contacted and asked to share the online study with their first semester medical students in the winter term 2022/23, which resulted in a final N = 535. We used validated and reliable measurements, including a self-created and piloted questionnaire covering medicine-specific explicit motives., Results: Comparing the mean scores between motives, we found that the altruism motive was the strongest motive (M = 5.19), followed by freedom (M = 4.88), affiliation (M = 4.72) and achievement (M = 4.59). The power motive achieved the lowest score (M = 3.92). Male students scored significantly higher for power (M = 4.24) than females did (M = 3.80, p < .001), while female students found affiliation more important (M = 4.81) than male students did (M = 4.59, p = .016). Female participants scored significantly higher for emotional intelligence (p = .010) and several personality aspects, including empathy (p < .001), but showed a significantly lower academic self-concept (p = .033), compared to their male colleagues. Nonetheless, the effect sizes were mostly small to medium., Conclusions: Our findings suggest that first-year medical students are primarily motivated by humanitarian factors to study medicine, compared to motives related to money or power. This is mostly in line with earlier studies using qualitative approaches, showing that MDT can be applied to explain explicit motives in medical students., Trial Registration: The longitudinal project, which this study was part of, was registered via OSF ( https://archive.org/details/osf-registrations-mfhek-v1 ) on the 28th of September 2022 under the title "Transformation of emotion and motivation factors in medical students during the study progress: A multicenter longitudinal study"., (© 2024. The Author(s).)

Purpose: Many curricula promote frontal teaching approaches, potentially decreasing interaction and motivation - also within infection prevention & control and infectious diseases (IPC/ID). We aimed to investigate the implementation of three innovative teaching methods (ITM) within IPC/ID education: game-based learning (GBL), peer-teaching (PT) and misinformation detection (MID)., Methods: Multi-phase study involving third-year medical students was conducted. Phase-1 included a cross-sectional survey, assessing previous ITM-experience and interest to participate in phase-2, where the students were divided into teams. Each team prepared a video covering an IPC/ID-topic with deliberately placed misinformation, which had to be identified and corrected by the opposing team, followed by qualitative evaluation (phase-3). Finally, the MID-concept was incorporated into regular curricula in a non-competitive environment (phase-4) and evaluated within a cohort not involved in phases 1-3., Results: 276 students responded to phase-1. 58% expressed interest in participating in phase-2. Roughly 59% [47-71%] CI-95% of respondents without previous PT-experience stated interest in PT, while the interest in GBL and MID was even higher. 19 students participated in phase-2. All topic MID-scores ranged between 6 and 8/10 points, except for emporiatrics (3/10). Post-hoc analysis revealed a positive student-perception of ITM, particularly GBL. Phase-4 received 103 responses with general positive evaluation. Major agreements existed on the usefulness of critical information evaluation for medical practice (82% [75-91%] CI-95%) and of MID during studies (69% [59-79%] CI-95%)., Conclusion: our results hint at a relatively high interest in ITM and show MID applicability in regular IPC/ID curricula, which could be of advantage for the learning environment., (© 2024. The Author(s).)

Problem: Creating medical exam questions is time consuming, but well-written questions can be used for test-enhanced learning, which has been shown to have a positive effect on student learning. The automated generation of high-quality questions using large language models (LLMs), such as ChatGPT, would therefore be desirable. However, there are no current studies that compare students' performance on LLM-generated questions to questions developed by humans., Approach: The authors compared student performance on questions generated by ChatGPT (LLM questions) with questions created by medical educators (human questions). Two sets of 25 multiple-choice questions (MCQs) were created, each with 5 answer options, 1 of which was correct. The first set of questions was written by an experienced medical educator, and the second set was created by ChatGPT 3.5 after the authors identified learning objectives and extracted some specifications from the human questions. Students answered all questions in random order in a formative paper-and-pencil test that was offered leading up to the final summative neurophysiology exam (summer 2023). For each question, students also indicated whether they thought it had been written by a human or ChatGPT., Outcomes: The final data set consisted of 161 participants and 46 MCQs (25 human and 21 LLM questions). There was no statistically significant difference in item difficulty between the 2 question sets, but discriminatory power was statistically significantly higher in human than LLM questions (mean = .36, standard deviation [SD] = .09 vs mean = .24, SD = .14; P = .001). On average, students identified 57% of question sources (human or LLM) correctly., Next Steps: Future research should replicate the study procedure in other contexts (e.g., other medical subjects, semesters, countries, and languages). In addition, the question of whether LLMs are suitable for generating different question types, such as key feature questions, should be investigated., (Copyright © 2023 the Association of American Medical Colleges.)

Background: Artificial intelligence (AI) is becoming increasingly important in healthcare. It is therefore crucial that today's medical students have certain basic AI skills that enable them to use AI applications successfully. These basic skills are often referred to as "AI literacy". Previous research projects that aimed to investigate medical students' AI literacy and attitudes towards AI have not used reliable and validated assessment instruments., Methods: We used two validated self-assessment scales to measure AI literacy (31 Likert-type items) and attitudes towards AI (5 Likert-type items) at two German medical schools. The scales were distributed to the medical students through an online questionnaire. The final sample consisted of a total of 377 medical students. We conducted a confirmatory factor analysis and calculated the internal consistency of the scales to check whether the scales were sufficiently reliable to be used in our sample. In addition, we calculated t-tests to determine group differences and Pearson's and Kendall's correlation coefficients to examine associations between individual variables., Results: The model fit and internal consistency of the scales were satisfactory. Within the concept of AI literacy, we found that medical students at both medical schools rated their technical understanding of AI significantly lower (M MS1 = 2.85 and M MS2 = 2.50) than their ability to critically appraise (M MS1 = 4.99 and M MS2 = 4.83) or practically use AI (M MS1 = 4.52 and M MS2 = 4.32), which reveals a discrepancy of skills. In addition, female medical students rated their overall AI literacy significantly lower than male medical students, t(217.96) = -3.65, p <.001. Students in both samples seemed to be more accepting of AI than fearful of the technology, t(745.42) = 11.72, p <.001. Furthermore, we discovered a strong positive correlation between AI literacy and positive attitudes towards AI and a weak negative correlation between AI literacy and negative attitudes. Finally, we found that prior AI education and interest in AI is positively correlated with medical students' AI literacy., Conclusions: Courses to increase the AI literacy of medical students should focus more on technical aspects. There also appears to be a correlation between AI literacy and attitudes towards AI, which should be considered when planning AI courses., (© 2024. The Author(s).)

Serious games, as a learning resource, enhance their game character by embedding game design elements that are typically used in entertainment games. Serious games in its entirety have already proven their teaching effectiveness in different educational contexts including medical education. The embedded game design elements play an essential role for a game's effectiveness and thus they should be selected based on evidence-based theories. For game design elements embedded in serious games used for the education of medical and healthcare professions, an overview of theories for the selection lacks. Additionally, it is still unclear whether and how single game design elements affect the learning effectiveness. Therefore, the main aim of this systematic review is threefold. Firstly, light will be shed on the single game design elements used in serious games in this area. Second, the game design elements' underlying theories will be worked out, and third, the game design elements' effectiveness on student learning outcome will be assessed. Two literature searches were conducted in November 2021 and May 2022 in six literature databases with keywords covering the fields of educational game design, serious game, and medical education. Out of 1006 initial records, 91 were included after applying predefined exclusion criteria. Data analysis revealed that the three most common game design elements were points, storyline, and feedback. Only four underlying theories were mentioned, and no study evaluated specific game design elements. Since game design elements should be based on theories to ensure meaningful evaluations, the conceptual GATE framework is introduced, which facilitates the selection of evidence-based game design elements for serious games., (© 2024. The Author(s).)