Back to Search
Start Over
Can ChatGPT-3.5 Pass a Medical Exam? A Systematic Review of ChatGPT's Performance in Academic Testing.
- Source :
-
Journal of Medical Education & Curricular Development . 3/13/2024, p1-12. 12p. - Publication Year :
- 2024
-
Abstract
- OBJECTIVE: We, therefore, aim to conduct a systematic review to assess the academic potential of ChatGPT-3.5, along with its strengths and limitations when giving medical exams. METHOD: Following PRISMA guidelines, a systemic search of the literature was performed using electronic databases PUBMED/MEDLINE, Google Scholar, and Cochrane. Articles from their inception till April 4, 2023, were queried. A formal narrative analysis was conducted by systematically arranging similarities and differences between individual findings together. RESULTS: After rigorous screening, 12 articles underwent this review. All the selected papers assessed the academic performance of ChatGPT-3.5. One study compared the performance of ChatGPT-3.5 with the performance of ChatGPT-4 when giving a medical exam. Overall, ChatGPT performed well in 4 tests, averaged in 4 tests, and performed badly in 4 tests. ChatGPT's performance was directly proportional to the level of the questions' difficulty but was unremarkable on whether the questions were binary, descriptive, or MCQ-based. ChatGPT's explanation, reasoning, memory, and accuracy were remarkably good, whereas it failed to understand image-based questions, and lacked insight and critical thinking. CONCLUSION: ChatGPT-3.5 performed satisfactorily in the exams it took as an examinee. However, there is a need for future related studies to fully explore the potential of ChatGPT in medical education. [ABSTRACT FROM AUTHOR]
- Subjects :
- *CHATGPT
*ACADEMIC achievement
*ARTIFICIAL intelligence
*CRITICAL thinking
Subjects
Details
- Language :
- English
- ISSN :
- 23821205
- Database :
- Academic Search Index
- Journal :
- Journal of Medical Education & Curricular Development
- Publication Type :
- Academic Journal
- Accession number :
- 176035164
- Full Text :
- https://doi.org/10.1177/23821205241238641