Search

ObjectivesThe national implementation of competency-based medical education (CBME) has prompted an increased interest in identifying and tracking clinical and educational outcomes for emergency medicine training programs. For the 2019 Canadian Association of Emergency Physicians (CAEP) Academic Symposium, we developed recommendations for measuring outcomes in emergency medicine training in the context of CBME to assist educational leaders and systems designers in program evaluation.MethodsWe conducted a three-phase study to generate educational and clinical outcomes for emergency medicine (EM) education in Canada. First, we elicited expert and community perspectives on the best educational and clinical outcomes through a structured consultation process using a targeted online survey. We then qualitatively analyzed these responses to generate a list of suggested outcomes. Last, we presented these outcomes to a diverse assembly of educators, trainees, and clinicians at the CAEP Academic Symposium for feedback and endorsement through a voting process.ConclusionAcademic Symposium attendees endorsed the measurement and linkage of CBME educational and clinical outcomes. Twenty-five outcomes (15 educational, 10 clinical) were derived from the qualitative analysis of the survey results and the most important short- and long-term outcomes (both educational and clinical) were identified. These outcomes can be used to help measure the impact of CBME on the practice of Emergency Medicine in Canada to ensure that it meets both trainee and patient needs.

ObjectivesThe Canadian Resident Matching Service (CaRMS) selection process has come under scrutiny due to the increasing number of unmatched medical graduates. In response, we outline our residency program's selection process including how we have incorporated best practices and novel techniques.MethodsWe selected file reviewers and interviewers to mitigate gender bias and increase diversity. Four residents and two attending physicians rated each file using a standardized, cloud-based file review template to allow simultaneous rating. We interviewed applicants using four standardized stations with two or three interviewers per station. We used heat maps to review rating discrepancies and eliminated rating variance using Z-scores. The number of person-hours that we required to conduct our selection process was quantified and the process outcomes were described statistically and graphically.ResultsWe received between 75 and 90 CaRMS applications during each application cycle between 2017 and 2019. Our overall process required 320 person-hours annually, excluding attendance at the social events and administrative assistant duties. Our preliminary interview and rank lists were developed using weighted Z-scores and modified through an organized discussion informed by heat mapped data. The difference between the Z-scores of applicants surrounding the interview invitation threshold was 0.18-0.3 standard deviations. Interview performance significantly impacted the final rank list.ConclusionsWe describe a rigorous resident selection process for our emergency medicine training program which incorporated simultaneous cloud-based rating, Z-scores, and heat maps. This standardized approach could inform other programs looking to adopt a rigorous selection process while providing applicants guidance and reassurance of a fair assessment.

We designed two practical, user-friendly, low-cost, aesthetically pleasing resources, with the goal of introducing residents and observers to a new Competence by Design assessment system based on entrustable professional activities. They included a set of rotation- and stage-specific entrustable professional activities reference cards for bedside use by residents and observers and a curriculum board to organize the entrustable professional activities reference cards by stages of training based on our program's curriculum map. A survey of 14 emergency medicine residents evaluated the utilization and helpfulness of these resources. They had a positive impact on our program's transition to Competence by Design and could be successfully incorporated into other residency programs to support the introduction of entrustable professional activities-based Competence by Design assessment systems.Le groupe a conçu deux documents à la fois pratiques, conviviaux, peu coûteux et agréables à l’œil, dans le but de présenter aux résidents et aux observateurs un nouveau système d’évaluation du modèle d'acquisition des compétences par conception, fondé sur des activités professionnelles confiables. Les documents comprenaient un ensemble de fiches de référence illustrant des activités professionnelles confiables liées aux étapes de formation et aux stages cliniques, à utiliser au chevet par les résidents et les observateurs ainsi qu'un tableau cartonné représentatif du curriculum visant à répartir les fiches de référence liées aux activités professionnelles confiables selon les étapes de formation fondées sur la carte du programme. Quatorze résidents en médecine d'urgence ont évalué l'utilisation et l'utilité de ces documents. L'enquête a révélé que ces derniers avaient facilité la transition vers le modèle d'acquisition des compétences par conception, et se prêteraient bien à d'autres programmes de résidence afin d'aider à la mise en œuvre de systèmes d’évaluation de la nouvelle approche de formation, fondés sur des activités professionnelles confiables.

ObjectivesThe objective of this study was to evaluate the impact of a novel injury prevention intervention designed to prompt patients to initiate an injury prevention discussion with the ED physician, thus enabling injury prevention counselling and increasing bicycle helmet use among patients.MethodsA repeated measures 2 x 3 randomized controlled trial design was used. Fourteen emergency physicians were observed for two shifts each between June and August 2013. Each pair of shifts was randomized to either an injury prevention shift, during which the emergency physician would wear a customized scrub top, or a control shift. The outcomes of interest were physician time spent discussing injury prevention, current helmet use, and self-reported change in helmet use rates at one year. Logistic regression analyses were used to examine the impact of the intervention.ResultsThe average time spent on injury prevention for all patients was 3.3 seconds. For those patients who actually received counselling, the average time spent was 17.0 seconds. The scrub top intervention did not significantly change helmet use rates at one year. The intervention also had no significant impact on patient decisions to change or reinforcement of helmet use.ConclusionsOur study showed that the intervention did not increase physician injury prevention counselling or self-reported bicycle helmet use rates among patients. Given the study limitations, replication and extension of the intervention is warranted.

Innovation Concept: The fairness of the Canadian Residency Matching Service (CaRMS) selection process has been called into question by rising rates of unmatched medical students and reports of bias and subjectivity. We outline how the University of Saskatchewan Royal College emergency medicine program evaluates CaRMS applications in a standardized, rigorous, equitable and defensible manner. Methods: Our CaRMS applicant evaluation methods were first utilized in the 2017 CaRMS cycle, based on published Best Practices, and have been refined yearly to ensure validity, standardization, defensibility, rigour, and to improve the speed and flow of data processing. To determine the reliability of the total application scores for each rater, single measures intraclass correlation coefficients (ICCs) were calculated using a random effects model in 2017 and 2018. Curriculum, Tool or Material: A secure, online spreadsheet was created that includes applicant names, reviewer assignments, data entry boxes, and formulas. Each file reviewer entered data in a dedicated sheet within the document. Each application was reviewed by two staff physicians and two to four residents. File reviewers used a standardized, criterion-based scoring rubric for each application component. The file score for each reviewer-applicant pair was converted into a z-score based on each reviewer's distribution of scores. Z-scores of all reviewers for a single applicant were then combined by weighted average, with the group of staff and group of residents each being weighted to represent half of the final file score. The ICC for the total raw scores improved from 0.38 (poor) in 2017 to 0.52 (moderate) in 2018. The data from each reviewer was amalgamated into a master sheet where applicants were sorted by final file score and heat-mapped to offer a visual aid regarding differences in ratings. Conclusion: Our innovation uses heat-mapped and formula-populated spreadsheets, scoring rubrics, and z-scores to normalize variation in scoring trends between reviewers. We believe this approach provides a rigorous, defensible, and reproducible process by which Canadian residency programs can appraise applicants and create a rank order list.