Search

Background: Creating new written-exam questions is a burdensome task for faculty members. While several guidelines exist, there had not been a previous attempt to streamline them in a user-friendly tool. We created the Written Exam Question Quality tool (WEQQ) and explored potential users’ perception of this tool when writing their exam questions. Methods: We conducted a descriptive study to explore how four Canadian faculty members used the WEQQ. We conducted structured interviews that were analyzed within and across participants to understand the latter’s perceived usefulness and acceptability of the WEQQ. Quantitative data from a short questionnaire on creating exam questions and their psychometric properties were also collected. Results and conclusion: Participants’ perception of the WEQQ was positive, and they were favorable to its use. The WEQQ seemed to represent a user-friendly, easy way to help faculty members in creating multiple-choice or short-answer questions. Time on task remained the same when using the WEQQ. We were able to identify two user profiles, passive and active, which indicated how faculty members use the WEQQ to create exam questions. Future steps would be to further investigate if the WEQQ can increase the quality of written-exam questions and to understand how to promote an active use of the WEQQ when implementing this tool.

Validity as a social imperative foregrounds the social consequences of assessment and highlights the importance of building quality into the assessment development and monitoring processes. Validity as a social imperative is informed by current assessment trends such as programmatic-, longitudinal-, and rater-based assessment, and is one of the conceptualizations of validity currently at play in the Health Professions Education (HPE) literature. This Black Ice is intended to help readers to get a grip on how to embed principles of validity as a social imperative in the development and quality monitoring of an assessment. This piece draws on a program of work investigating validity as a social imperative, key HPE literature, and data generated through stakeholder interviews. We describe eight ways to implement validation practices that align with validity as a social imperative.

Background: Educators now use reported observations when assessing trainees’ performance. Unfortunately, they have little information about how to design and implement assessments based on reported observations. Objective: The purpose of this scoping review was to map the literature on the use of reported observations in judging health professions education (HPE) trainees' performances. Methods: Arksey and O'Malley’s (2005) method was used with four databases (sources: ERIC, CINAHL, MEDLINE, PsycINFO). Eligibility criteria for articles were: documents in English or French, including primary data, and initial or professional training; (2) training in an HPE program; (3) workplace-based assessment; and (4) assessment based on reported observations. The inclusion/exclusion, and data extraction steps were performed (agreement rate > 90%). We developed a data extraction grid to chart the data. Descriptive analyses were used to summarize quantitative data, and the authors conducted thematic analysis for qualitative data. Results: Based on 36 papers and 13 consultations, the team identified six steps characterizing trainee performance assessment based on reported observations in HPE: (1) making first contact, (2) observing and documenting the trainee performance, (3) collecting and completing assessment data, (4) aggregating assessment data, (5) inferring the level of competence, and (6) documenting and communicating the decision to the stakeholders. Discussion: The design and implementation of assessment based on reported observations is a first step towards a quality implementation by guiding educators and administrators responsible for graduating competent professionals. Future research might focus on understanding the context beyond assessor cognition to ensure the quality of meta-assessors’ decisions.

Background: While educators observe gaps in clerkship students’ clinical reasoning (CR) skills, students report few opportunities to develop them. This study aims at exploring how students who used self-explanation (SE) and structured reflection (SR) for CR learning during preclinical training, applied these learning strategies during clerkship. Methods: We conducted an explanatory sequential mixed-methods study involving medical students. With a questionnaire, we asked students how frequently they adopted behaviours related to SE and SR during clerkship. Next, we conducted a focus group with students to explore why they adopted these behaviours. Results: Fifty-two of 198 students answered the questionnaire and five participated in a focus group. Specific behaviours adopted varied from 50% to 98%. We identified three themes about why students used these strategies: as “just in time” learning strategies; to deepen their understanding and identify gaps in knowledge; to develop a practical approach to diagnosis. A fourth theme related to the balance between learning and assessment and its consequence on adopting SE behaviours. Conclusions: Students having experienced SE and SR regularly in preclinical training tend to transpose these strategies into the clerkship providing them with a practical way to reflect deliberately and capture learning opportunities of the unpredictable clinical context.

Background: While developing reflection skills is considered important by educators, the assessment of these skills is often associated with unintended negative consequences. In the context of a mandatory longitudinal course that aims to promote the development of reflection on professional identity, we assessed students’ commitment to reflection. This study explores students’ perception of this assessment by their mentor. Methods: We conducted a qualitative descriptive study using semi-structured interviews with twenty-one 1st and six 2nd year medical students. Thematic analysis was informed by Braun and Clarke’s six-step approach. Results: We identified four main themes: 1- assessment as a motivator, 2- consequences on authenticity, 3- perception of inherent subjectivity, and 4 - relationship with the mentor. Conclusions: In the context of assessing reflection skills in future physicians, we observed that students –when assessed on the process of reflection– experienced high motivation but were ambivalent on the question of authenticity. The subjectivity of the assessment as well as the relationship with their mentor also raises questions. Nevertheless, this assessment approach for reflective skills appears to be promising in terms of limiting the negative consequences of assessment.

OBJECTIVES Based on increasing drug overdose deaths and a shortage of healthcare professionals trained in the management of opioid use disorder (OUD), it is imperative to improve health professional education in addiction medicine. This small group learning exercise and patient panel was designed to provide first year medical students with insights into the lives of people with OUD—through a lens of harm reduction—and to connect biomedical knowledge to the core values and professional themes of their doctoring courses. METHODS Facilitators were assigned to each small group of 8 students for the harm reduction-centered Long and Winding Road small group case exercise. This was followed by a patient panel of 2 to 3 persons with OUD. The small group was conducted with first-year medical students as a virtual training session due to the COVID-19 pandemic. Students completed pre- and post-session surveys about agreement with statements pertaining to the learning objectives. RESULTS The small group and patient panel were delivered over 8 sessions and attended by all first-year medical students (N = 201). Survey response rate was 67%. Post-session, there was significantly greater agreement with knowledge on all learning objectives compared to pre-session. Two relevant multiple-choice questions on the medical student final exam were answered correctly by 79% and 98% of students. CONCLUSION Centering on people with lived experience, we completed small groups and patient panels to introduce concepts of OUD and harm reduction to first year medical students. Pre- and post-session surveys showed short-term achievement of the learning objectives.

Abstract Background Daily rhythms are observed in humans and almost all other organisms. Most of these observed rhythms reflect both underlying endogenous circadian rhythms and evoked responses from behaviours such as sleep/wake, eating/fasting, rest/activity, posture changes and exercise. For many research and clinical purposes, it is important to understand the contribution of the endogenous circadian component to these observed rhythms. Content The goal of this manuscript is to provide guidance on best practices in measuring metrics of endogenous circadian rhythms in humans and promote the inclusion of circadian rhythms assessments in studies of health and disease. Circadian rhythms affect all aspects of physiology. By specifying minimal experimental conditions for studies, we aim to improve the quality, reliability and interpretability of research into circadian and daily (i.e., time‐of‐day) rhythms and facilitate the interpretation of clinical and translational findings within the context of human circadian rhythms. We describe protocols, variables and analyses commonly used for studying human daily rhythms, including how to assess the relative contributions of the endogenous circadian system and other daily patterns in behaviours or the environment. We conclude with recommendations for protocols, variables, analyses, definitions and examples of circadian terminology. Conclusion Although circadian rhythms and daily effects on health outcomes can be challenging to distinguish in practice, this distinction may be important in many clinical settings. Identifying and targeting the appropriate underlying (patho)physiology is a medical goal. This review provides methods for identifying circadian effects to aid in the interpretation of published work and the inclusion of circadian factors in clinical research and practice.

The COVID-19 pandemic rushed licensure and certification institutions, as well as many university programs, to integrate Information and Communication Technologies (ICTs) in their practices to allow for remote administrations of their exams independent of distancing measures. The Black Ice covered in this manuscript is the integration of ICTs to allow remote administration of high-stakes assessments in terms of its development, administration, and monitoring with the aim to promote the validity of score interpretation.

Introduction During the first year of the COVID-19 pandemic, over 93,000 Americans lost their lives to a preventable overdose. Medications for opioid use disorder (OUD) have been shown to decrease mortality in OUD but are underutilized. Through this case-based learning exercise, first-year medical students applied physiologic and pharmacologic principles to the diagnosis and treatment of OUD. Methods Faculty facilitated a case discussion over a 1-hour large-group case-based learning (CBL) session. Facilitators utilized PowerPoint slides to illustrate graphs and figures while discussing the case. To evaluate students on the CBL learning objectives, three pharmacology exam questions were administered; students also evaluated the CBL's effectiveness in meeting educational objectives on three Likert-scale questions and via open-ended feedback. Results First-year medical students (n = 200) completed the CBL. The mean score on the exam questions was 91%. Students agreed or strongly agreed that the CBL was an effective way to learn pharmacology principles (69%), that it reinforced pharmacologic fundamentals (70%), and that it showed how pharmacology fundamentals were important in the real world of clinical medicine (86%). Qualitative feedback on the CBL was generally positive, including satisfaction with the small-group setting and practical applications of pharmacology to clinical practice. Discussion This CBL exercise contains content critical for preparing students to combat the modern opioid epidemic. The exercise provides an opportunity for learners to review fundamental pharmacodynamic and pharmacokinetic principles so as to ready them for clinical clerkships and beyond.

Introduction The overdose crisis remains a critical public health problem, creating an urgent need to train physicians in the treatment and management of opioid use disorder (OUD). Our medicine clerkship module aimed to close this gap by training and assessing students’ motivational interviewing skills, harm reduction knowledge, and use of nonstigmatizing language in the treatment of patients with OUD. Methods We evaluated the impact of a small-group, case-based activity and patient panel on the clinical documentation skills of students in a medicine clerkship. Clinical documentation was based on an observed structured clinical examination of a standardized patient with OUD and was evaluated using a grading rubric that followed the module learning objectives. Students also submitted reflections on the curriculum. Results Qualitative responses (n = 40) from students evaluating the small-group activity and patient panel exercise revealed overall student satisfaction with the patient panel and exposure to patients living with OUD. Three themes emerged from student reflections: (1) humanity, (2) different paths to recovery, and (3) using nonstigmatizing language. For the quantitative test, students’ (n = 39) mean clinical documentation scores before and after the small-group activity and patient panel increased from 10.1 to 11.3 out of 13.5 possible points. There was a significant difference between mean pretest and posttest scores (p < .001). Discussion The medicine clerkship provided an acceptable and feasible opportunity for implementing a multifaceted educational experience for students with significant immediate impact on their evaluation of patients with OUD.

Introduction: Recently, validity as a social imperative was proposed as an emerging conceptualization of validity in the assessment literature in health professions education (HPE). To further develop our understanding, we explored the perceived acceptability and anticipated feasibility of validity as a social imperative with users and leaders engaged with assessment in HPE in Canada. Methods: We conducted a qualitative interpretive description study. Purposeful and snowball sampling were used to recruit participants for semi-structured individual interviews and focus groups. Each transcript was analyzed by two team members and discussed with the team until consensus was reached. Results: We conducted five focus group and eleven interviews with two different stakeholder groups (users and leaders). Our findings suggest that the participants perceived the concept of validity as a social imperative as acceptable. Regardless of group, participants shared similar considerations regarding: the limits of traditional validity models, the concept’s timeliness and relevance, the need to clarify some terms used to characterize the concept, the similarities with modern theories of validity, and the anticipated challenges in applying the concept in practice. In addition, participants discussed some limits with current approaches to validity in the context of workplace-based and programmatic assessment. Conclusion: Validity as a social imperative can be interwoven throughout existing theories of validity and may represent how HPE is adapting traditional models of validity in order to respond to the complexity of assessment in HPE; however, challenges likely remain in operationalizing the concept prior to its implementation.

Elisabeth Boileau,1 Christina St-Onge,2 Marie-Claude Audétat3 1Department of Family and Emergency Medicine, 2Department of Medicine, Université de Sherbrooke, Sherbrooke, QC, Canada; 3Unité des Internistes Généralistes et Pédiatres, Université de Genève, Geneva, Switzerland Abstract: Struggling medical trainees pose a challenge to clinical teachers, since these learners warrant closer supervision that is time-consuming and competes with time spent on patient care. Clinical teachers’ perception that they are ill equipped to address learners’ difficulties efficiently may lead to delays or even lack of remediation for these learners. Because of the paucity of evidence to guide best practices in remediation, the best approach to guide clinical teachers in the field remains to be established. We aimed to present a synthetic review of the empirical evidence and theory that may guide clinical teachers in their daily task of supervising struggling learners, reviewing current knowledge on the challenges and solutions that have been identified and explored. A computerized literature search was performed using Medline, Embase, Education Resources Information Center, and Education Source, after which final articles were selected based on relevance. The literature reviewed provided best evidence for clinical teachers to address learners’ difficulties, which is presented in the order of the four steps inherent to the clinical approach: 1) detecting a problem based on a subjective impression, 2) gathering and documenting objective data, 3) assessing data to make a diagnosis, and 4) planning remediation. A synthesized classification of pedagogical diagnoses is also presented. This review provides an outline of practical recommendations regarding the supervision and management of struggling learners up to the remediation phase. Our findings suggest that future research and faculty development endeavors should aim to operationalize remediation strategies further in response to specific diagnoses, and to make these processes more accessible to clinical teachers in the field. Keywords: clinical supervision, underperformance, struggling learners, clinical teachers, pedagogical diagnosis, remediation

Kathleen Ouellet,1 Robert Sabbagh,2 Linda Bergeron,3 Sandeep Kumar Mayer,2 Christina St-Onge4 1Center for Health Profession Education, Faculty of Medicine and Health Sciences, Université de Sherbrooke, 2Department of Surgery, Faculty of Medicine and Health Sciences, Université de Sherbrooke, 3Research Chair in Medical Education, Paul Grand’Maison of the Société des médecins, Université de Sherbrooke, 4Department of Medicine, Faculty of Medicine and Health Sciences, Université de Sherbrooke, Sherbrooke, Québec, Canada Background: Collaboration is an important competence to be acquired by residents. Although improving residents’ collaboration via interprofessional education has been investigated in many studies, little is known about the residents’ spontaneous collaborative behavior. The purpose of this exploratory study was to describe how residents spontaneously collaborate.Methods: Seven first-year residents (postgraduate year 1; three from family medicine and one each from ear, nose, and throat, obstetrics/gynecology, general surgery, and orthopedic surgery) participated in two collaborative meetings with actors performing the part of other health ­professionals (ie, occupational therapist, physiotherapist, nurse, or social worker). Both meetings were built around an issue or conflict with the patients’ families reported by one professional. The residents were required to lead the meeting to collect proper information to reach a joint decision. Two team members analyzed the video recordings of the meetings using an emerging-theme qualitative methodology.Results: Although the residents spontaneously knew how to successfully communicate with other professionals, they seemed to struggle with the patient-centered approach and the shared decision-making process.Discussion: Even if the residents performed communication-wise in their collaborative role, they seemed to have perceived themselves as decision makers instead of collaborators in the joint decision process. The results of this study can inform future studies on learning strategies to improve behaviors that would more likely need attention in interprofessional education. Keywords: interprofessional education, IPE, communication, collaboration

Background: Challenges associated with the use of the CanMEDS physician competency framework (CanMEDS) have been the subject of several studies. Most of these have focused on the adoption of specific roles in an Anglophone context. This study aims to investigate how Francophone postgraduate medical education (PGME) program directors have integrated the CanMEDS framework into their programs. Methods: We invited Francophone PGME program directors to participate in group interviews aimed at exploring their experiences using the CanMEDS framework. We used an open-ended interview guide and realized a thematic analysis of the transcripts. Results: We held five group interviews between February and December 2014 with 17 Francophone program directors representing 13 out of a maximum of 62 different specialties/subspecialties. Although program directors endorsed the framework, its integration was seen as challenging, particularly the assessment of non-medical expert roles. To overcome these challenges, they relied on common strategies including a longitudinal approach to the framework, improving inter-program collaboration, and subcontracting the teaching of certain roles. Conclusion: While integrating the CanMEDS framework into their programs, Francophone program directors struggled with teaching and assessing non-medical expert roles and ensuring their longitudinal integration over time. Directors relied on various strategies, some of which (e.g., subcontracting) may ultimately limit the adoption of the framework as a whole. ___ Contexte: Les défis associés à l'utilisation du référentiel de compétences CanMEDS pour les médecins ont fait l'objet de plusieurs études. La plupart de ces études ont portées sur l'adoption de rôles spécifiques dans un contexte anglophone. Cette étude vise à explorer comment les directeurs de programmes d’études médicales postdoctorales (EMP) francophones ont intégré CanMEDS dans leurs programmes. Méthodes: Nous avons invité les directeurs de programmes EMP francophones à participer à des entrevues de groupe. Ces entrevues visaient à explorer leur expérience de l’utilisation du référentiel CanMEDS. Nous avons utilisé un guide d'entrevue ouvert et nous avons fait une analyse thématique des transcriptions. Résultats: Nous avons tenu cinq entrevues de groupe entre février et décembre 2014 avec 17 directeurs de programmes de 13 des 62 spécialités/sous-spécialités. Bien que les directeurs de programmes appuient le référentiel, son intégration a été perçue comme un défi, notamment en ce qui a trait à l'évaluation des rôles autres que celui d'expert médical. Pour surmonter ces défis, ils se sont appuyés sur des stratégies communes, notamment une approche longitudinale du référentiel, l'amélioration de la collaboration entre les programmes et la sous-traitance de l'enseignement de certains rôles. Conclusions: À travers le processus d’intégration du référentiel CanMEDS, les directeurs de programmes EMP francophones ont de la difficulté à enseigner et à évaluer les rôles autres que celui d'expert médical ainsi qu’à veiller à leur intégration respective et continue au fil du temps. Ils ont eu recours à diverses stratégies, dont certaines (p. ex., la sous-traitance) pourraient ultimement limiter l'adoption du référentiel dans son ensemble.

The first part of this article focuses on the wide variations in the diagnosis of attention-deficit hyperactivity disorder (ADHD) that are observed between countries and between regions within the same country. Diagnosing ADHD is more problematic than is commonly thought. For instance, younger American children in the same grade at school are 64% more likely to receive the diagnosis; symptoms can result from many underlying causes. Furthermore, ADHD can be confused with many other health issues. As a result it is largely overdiagnosed and overtreated. The second part of the article reviews recent studies showing that anti-ADHD drugs lack long-term effectiveness and come with important adverse events. Overall, and in the long run, the pharmacologic treatment of ADHD is likely to cause more harm than good.