1. Relationship between ratings of performance in the simulated and workplace environments among emergency medicine residents
- Author
-
Warren J Cheung, Meghan McConnell, Nancy Dudek, Michael O'Brien, and Nicholas Prudhomme
- Subjects
Canada ,medicine.medical_specialty ,Critically ill ,business.industry ,Specialty ,Internship and Residency ,030208 emergency & critical care medicine ,03 medical and health sciences ,0302 clinical medicine ,Concordance correlation coefficient ,Emergency medicine ,Emergency Medicine ,Humans ,Medicine ,Clinical Competence ,030212 general & internal medicine ,Workplace ,business - Abstract
ObjectivesThe Emergency Medicine (EM) Specialty Committee of the Royal College of Physicians and Surgeons of Canada (RCPSC) specifies that resuscitation entrustable professional activities (EPAs) can be assessed in the workplace and simulated environments. However, limited validity evidence for these assessments in either setting exists. We sought to determine if EPA ratings improve over time and whether an association exists between ratings in the workplace v. simulation environment.MethodsAll Foundations EPA1 (F1) assessments were collected for first-year residents (n = 9) in our program during the 2018–2019 academic year. This EPA focuses on initiating and assisting in the resuscitation of critically ill patients. EPA ratings obtained in the workplace and simulation environments were compared using Lin's concordance correlation coefficient (CCC). To determine whether ratings in the two environments differed as residents progressed through training, a within-subjects analysis of variance was conducted with training environment and month as independent variables.ResultsWe collected 104 workplace and 36 simulation assessments. No correlation was observed between mean EPA ratings in the two environments (CCC(8) = -0.01; p = 0.93). Ratings in both settings improved significantly over time (F(2,16) = 18.8; p < 0.001; η2= 0.70), from 2.9 ± 1.2 in months 1–4 to 3.5 ± 0.2 in months 9–12. Workplace ratings (3.4 ± 0.1) were consistently higher than simulation ratings (2.9 ± 0.2) (F(2,16) = 7.2; p = 0.028; η2= 0.47).ConclusionsNo correlation was observed between EPA F1 ratings in the workplace v. simulation environments. Further studies are needed to clarify the conflicting results of our study with others and build an evidence base for the validity of EPA assessments in simulated and workplace environments.
- Published
- 2020
- Full Text
- View/download PDF