Search

Purpose: To evaluate the effectiveness of ChatGPT-generated feedback compared to expert-written feedback in improving clinical reasoning skills among first-year medical students., Methods: This is a randomized controlled trial conducted at a single medical school and involved 129 first-year medical students who were randomly assigned to two groups. Both groups completed three formative tests with feedback on urinary tract infections (UTIs; uncomplicated, complicated, pyelonephritis) over five consecutive days as a spaced repetition, receiving either expert-written feedback (control, n = 65) or ChatGPT-generated feedback (experiment, n = 64). Clinical reasoning skills were assessed using Key-Features Questions (KFQs) immediately after the intervention and 10 days later. Students' critical approach to artificial intelligence (AI) was also measured before and after disclosing the AI involvement in feedback generation., Results: There was no significant difference between the mean scores of the control (immediate: 78.5 ± 20.6 delayed: 78.0 ± 21.2) and experiment (immediate: 74.7 ± 15.1, delayed: 76.0 ± 14.5) groups in overall performance on Key-Features Questions (out of 120 points) immediately (P = .26) or after 10 days (P = .57), with small effect sizes. However, the control group outperformed the ChatGPT group in complicated urinary tract infection cases (P < .001). The experiment group showed a significantly higher critical approach to AI after disclosing, with medium-large effect sizes., Conclusions: ChatGPT-generated feedback can be an effective alternative to expert feedback in improving clinical reasoning skills in medical students, particularly in resource-constrained settings with limited expert availability. However, AI-generated feedback may lack the nuance needed for more complex cases, emphasizing the need for expert review. Additionally, exposure to the drawbacks in AI-generated feedback can enhance students' critical approach towards AI-generated educational content., (© The Author(s) 2024. Published by Oxford University Press on behalf of Fellowship of Postgraduate Medicine.)

Background: Anatomy is a crucial part of medical education, and there have been attempts to improve this field by utilizing various methods. With the advancement of technology, three-dimensional (3D) materials have gained popularity and become a matter of debate about their effectiveness compared to two-dimensional (2D) sources. This research aims to analyze the effectiveness of 3D PDFs compared to 2D atlases., Methods: This study is a randomized controlled trial involving 87 Year-1 and Year-2 medical students at Gazi University Faculty of Medicine, Turkey. The study was conducted in two steps. In Step-1, students were randomized to watch lecture videos on liver anatomy and male genitalia anatomy supplemented with either a 3D PDF (intervention group) or 2D atlas (control group) images. Following the video lectures, a test (immediate test) was administered. In Step-2, the same test (delayed test) was administered 10 days after the immediate test. The test scores were compared between the intervention and control groups. In addition to the descriptive analyses, Chi-square and Mann-Whitney U tests were performed., Results: In the immediate test, while there was no significant difference between the groups for the liver test (p > 0.05), 3D PDF group's scores (Median = 24.50) was significantly higher than the 2D atlas group's in the genitalia test (Median = 21.00), (p = 0.017). The effect size (Cohen's d) was 0.57. In the delayed test, there was no significant difference between the groups in the liver and genitalia tests (p > 0.05). However, the effect size in the immediate genitalia test was 0.40. Year-1 students' immediate test of genitalia performances were significantly higher in the 3D PDF group (Median = 24.00) than the 2D atlas group (Median = 19.00), (p = 0.016). The effect size was 0.76. Also, Year-1 students' 3D PDF group (Median = 20.50) presented with significantly higher performance than the 2D atlas group (Median = 12.00), (p = 0.044) in the delayed test of genitalia, with the 0.63 effect size., Conclusion: 3D PDF is more effective than 2D atlases in teaching anatomy, especially to initial learners. It is particularly useful for teaching complex anatomical structures, such as male genitalia, compared to the liver. Hence, it may be a valuable tool for medical teachers to utilize during lectures., (© 2023. The Author(s).)