Search

Background & Objective: Teaching and assessment of complex problem solving are a challenge for medical education. Integrating Machine Learning (ML) into medical education has the potential to revolutionize teaching and assessment of these problem-solving processes. In order to demonstrate possible applications of ML to education, we sought to apply ML in the context of a structured Video Commentary (VC) assessment, using ML to predict residents' training level., Setting: A secondary analysis of multi-institutional, IRB approved study. Participants had completed the VC assessment consisting of 13 short (20-40 seconds) operative video clips. They were scored in real-time using an extensive checklist by an experienced proctor in the assessment. A ML model was developed using TensorFlow and Keras. The individual scores of the 13 video clips from the VC assessment were used as the inputs for the ML model as well as for regression analysis., Participants: A total of 81 surgical residents of all postgraduate years (PGY) 1-5 from 7 institutions constituted the study sample., Results: Scores from individual VC clips were strongly positively correlated with PGY level (p = 0.001). Some video clips were identified to be strongly correlated with a higher total score on the assessment; others had significant influence when used to predict trainees' PGY levels. Using a supervised machine learning model to predict trainees' PGY resulted in a 40% improvement over traditional statistical analysis., Conclusions: Performing better in a few select video clips was key to obtaining a higher total score but not necessarily foretelling of a higher PGY level. The use of the total score as a sole measure may fail to detect deeper relationships. Our ML model is a promising tool in gauging learners' levels on an assessment as extensive as VC. The model managed to approximate residents' PGY levels with a lower MAE than using traditional statistics. Further investigations with larger datasets are needed., (Copyright © 2020 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.)

Background: High-income countries have increased the use of simulation-based training and assessment for surgical education. Learners in low- and middle-income countries may have different educational needs and levels of autonomy but they and their patients could equally benefit from the procedural training simulation provides. We sought to characterize the current state of surgical skills simulation in East, Central, and Southern Africa and determine residents' perception and future interest in such activities., Methods: A survey was created via collaboration and revision between trainees and educators with experiences spanning high-income countries and low- and middle-income countries. The survey was administered on paper to 76 trainees (PGY2-3) who were completing the College of Surgeons of East, Central, and Southern Africa (COSECSA) Membership of the College of Surgeons examination in Kampala, Uganda in December 2019. Data from paper responses were summarized using descriptive statistics and frequencies., Results: We received responses from 43 trainees (57%) from 11 countries in sub-Saharan Africa who participated in the examination. Fifty-eight percent of respondents reported having dedicated space for surgical skills simulation training, and most (91%) had participated in some form of simulation activity at some point in their training. However, just 16% used simulation as a regular part of training. The majority of trainees (90%) felt that surgical skills learned in simulation were transferrable to the operating room and agreed it should be a required part of training. Seventy-one percent of trainees felt that simulation could objectively measure technical skills, and 73% percent of respondents agreed that simulation should be integrated into formal assessment. However, residents split on whether proficiency in simulation should be achieved prior to operative experience (54%) and if nontechnical skills could be measured (51%). The most common cited barriers to the integration of surgical simulation into residents' education were lack of suitable tools and models (85%), funding (73%), and maintenance of facilities (49%)., Conclusions: Residents from East, Central, and Southern Africa strongly agree that simulation is a valuable educational tool and ought to be required during their surgical residency. Barriers to achieving this goal include availability of affordable tools, adequate funding and confidence in the value of the educational experience. Trainees affirm further efforts are necessary to make simulation more widely available in these contexts., (Copyright © 2021 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.)