Search

Background The Emergency Medicine In-Training Examination (EMITE) is one of the few validated instruments for medical knowledge assessment of emergency medicine (EM) residents. The EMITE is administered only once annually, with results available just 2 months before the end of the academic year. An earlier predictor of EMITE scores would be helpful for educators to institute timely remediation plans. A previous single-site study found that only 69% of faculty predictions of EMITE scores were accurate. Objective The goal of this article was to measure the accuracy with which EM faculty at five residency programs could predict EMITE scores for resident physicians. Methods We asked EM faculty at five different residency programs to predict the 2014 EMITE scores for all their respective resident physicians. The primary outcome was prediction accuracy, defined as the proportion of predictions within 6% of the actual scores. The secondary outcome was prediction precision, defined as the mean deviation of predictions from the actual scores. We assessed faculty background variables for correlation with the two outcomes. Results One hundred and eleven faculty participated in the study (response rate 68.9%). Mean prediction accuracy for all faculty was 60.0%. Mean prediction precision was 6.3%. Participants were slightly more accurate at predicting scores of noninterns compared to interns. No faculty background variable correlated with the primary or secondary outcomes. Eight participants predicted scores with high accuracy (>80%). Conclusions In this multicenter study, EM faculty possessed only moderate accuracy at predicting resident EMITE scores. A very small subset of faculty members is highly accurate.

Background The Emergency Medicine In-Training Examination (EMITE) is one of the only valid tools for medical knowledge assessment in current use by emergency medicine (EM) residencies. However, EMITE results return late in the academic year, providing little time to institute potential remediation. Objective The goal of this study was to determine the ability of EM faculty to accurately predict resident EMITE scores prior to results return. Methods We asked EM faculty at the study site to predict the 2012 EMITE scores of the 50 EM residents 2 weeks prior to results being available. The primary outcome was prediction accuracy, defined as the proportion of predictions within 6% of the actual score. The secondary outcome was prediction precision, defined as the mean deviation of predictions from the actual scores. We assessed several faculty background variables, including years of experience, educational leadership status, and clinical hours worked, for correlation with the two outcomes. Results Thirty-two of the 38 faculty (84.2%, 95% confidence interval [CI] 69.6–92.6) participated in the study, rendering a total of 1600 predictions for 50 residents. Mean resident EMITE score was 81.1% (95% CI 79.5–82.8%). Mean prediction accuracy for all faculty participants was 69% (95% CI 65.9–72.1%). Mean prediction precision was 5.2% (95% CI 4.9–5.5%). Education leadership status was the only background variable correlated with the primary and secondary outcomes (Spearman's ρ = 0.51 and −0.53, respectively). Conclusion Faculty possess only moderate accuracy at predicting resident EMITE scores. We recommend a multicenter study to evaluate the generalizability of the present results.

Background Reading emergent electrocardiograms (ECGs) is one of the emergency physician's most crucial tasks, yet no well-validated tool exists to measure resident competence in this skill. Objectives To assess validity of a novel tool measuring emergency medicine resident competency for interpreting, and responding to, critical ECGs. In addition, we aim to observe trends in this skill for resident physicians at different levels of training. Methods This is a multi-center, prospective study of postgraduate year (PGY) 1–4 residents at five emergency medicine (EM) residency programs in the United States. An assessment tool was created that asks the physician to identify either the ECG diagnosis or the best immediate management. Results One hundred thirteen EM residents from five EM residency programs submitted completed assessment surveys, including 43 PGY-1s, 33 PGY-2s, and 37 PGY-3/4s. PGY-3/4s averaged 74.6% correct (95% confidence interval [CI] 70.9–78.4) and performed significantly better than PGY-1s, who averaged 63.2% correct (95% CI 58.0–68.3). PGY-2s averaged 69.0% (95% CI 62.2–73.7). Year-to-year differences were more pronounced in management than in diagnosis. Conclusions Residency training in EM seems to be associated with improved ability to interpret “critical” ECGs as measured by our assessment tool. This lends validity evidence for the tool by correlating with a previously observed association between residency training and improved ECG interpretation. Resident skill in ECG interpretation remains less than ideal. Creation of this sort of tool may allow programs to assess resident performance as well as evaluate interventions designed to improve competency.