Search

Abstract Background Artificial intelligence (AI) chatbots have demonstrated proficiency in structured knowledge assessments; however, there is limited research on their performance in scenarios involving diagnostic uncertainty, which requires careful interpretation and complex decision-making. This study aims to evaluate the efficacy of AI chatbots, GPT-4o and Claude-3, in addressing medical scenarios characterized by diagnostic uncertainty relative to Family Medicine residents. Methods Questions with diagnostic uncertainty were extracted from the Progress Tests administered by the Department of Family and Community Medicine at the University of Toronto between 2022 and 2023. Diagnostic uncertainty questions were defined as those presenting clinical scenarios where symptoms, clinical findings, and patient histories do not converge on a definitive diagnosis, necessitating nuanced diagnostic reasoning and differential diagnosis. These questions were administered to a cohort of 320 Family Medicine residents in their first (PGY-1) and second (PGY-2) postgraduate years and inputted into GPT-4o and Claude-3. Errors were categorized into statistical, information, and logical errors. Statistical analyses were conducted using a binomial generalized estimating equation model, paired t-tests, and chi-squared tests. Results Compared to the residents, both chatbots scored lower on diagnostic uncertainty questions (p

The COVID-19 pandemic has led to many undergraduate medical programs shifting their preclinical curricula online— reducing access to clinical skills sessions and ultimately causing gaps in students’ knowledge. This study sought to better understand the impact and role of virtual clinical skills training sessions, using point-of-view (POV) livestreaming wearable technology, in supplementing medical students’ learning. 38 University of Ottawa medical students were recruited to participate in a 1.5-hour virtual clinical teaching session. An abdominal physical examination was broadcasted through two views (chest-mounted smartphone, room overview). Participants completed pre- and post-event questionnaires on their overall impression, satisfaction/challenges, and platform efficacy compared to other learning modalities. Improvements were noted in participants’ perspectives towards event engagement (p=0.042, Cohen’s d=0.48), comparability to in-person encounters (p=