Search

BackgroundSince 2013, US residency programs have used the competency-based framework of the Milestones to report resident progress and to provide feedback to residents. The implementation of Milestones-based assessments, clinical competency committee (CCC) meetings, and processes for providing feedback varies among programs and warrants systematic examination across specialties.ObjectiveWe sought to determine how varying assessment, CCC, and feedback implementation strategies result in different outcomes in resource expenditure and stakeholder engagement, and to explore the contextual forces that moderate these outcomes.MethodsFrom 2017 to 2018, interviews were conducted of program directors, CCC chairs, and residents in emergency medicine (EM), internal medicine (IM), pediatrics, and family medicine (FM), querying their experiences with Milestone processes in their respective programs. Interview transcripts were coded using template analysis, with the initial template derived from previous research. The research team conducted iterative consensus meetings to ensure that the evolving template accurately represented phenomena described by interviewees.ResultsForty-four individuals were interviewed across 16 programs (5 EM, 4 IM, 5 pediatrics, 3 FM). We identified 3 stages of Milestone-process implementation, including a resource-intensive early stage, an increasingly efficient transition stage, and a final stage for fine-tuning.ConclusionsResidency program leaders can use these findings to place their programs along an implementation continuum and gain an understanding of the strategies that have enabled their peers to progress to improved efficiency and increased resident and faculty engagement.

Haptic perception is used in the anatomy laboratory with the handling of three‐dimensional (3D) prosections, dissections, and synthetic models of anatomical structures. Vision‐based spatial ability has been found to correlate with performance on tests of 3D anatomy knowledge in previous studies. The objective was to explore whether haptic‐based spatial ability was correlated with vision‐based spatial ability. Vision‐based spatial ability was measured in a study group of 49 medical graduates with three separate tests: a redrawn Vandenberg and Kuse Mental Rotations Tests in two (MRT A) and three (MRT C) dimensions and a Surface Development Test (SDT). Haptic‐based spatial ability was measured using 18 different objects constructed from 10 cubes glued together. Participants were asked to draw these objects from blind haptic perception, and drawings were scored by two independent judges. The maximum score was 24 for each of MRT A and MRT C, 60 for SDT, and 18 for the drawings. The drawing score based on haptic perception [median = 17 (lower quartile = 16, upper quartile = 18)] correlated with MRT A [14 (9, 17)], MRT C [9 (7, 12)] and SDT [44 (36, 52)] scores with a Spearman's rank correlation coefficient of 0.395 (p = 0.0049), 0.507 (p = 0.0002) and 0.606 (p < 0.0001), respectively. Spatial abilities assessed by vision‐based tests were correlated with a drawing score based on haptic perception of objects. Future research should investigate the contribution of haptic‐based and vision‐based spatial abilities on learning 3D anatomy from physical models. [ABSTRACT FROM AUTHOR]

Background: The conceptualisation of medical competence is central to its use in competency‐based medical education. Calls for 'fixed standards' with 'flexible pathways', recommended in recent reports, require competence to be well defined. Making competence explicit and measurable has, however, been difficult, in part due to a tension between the need for standardisation and the acknowledgment that medical professionals must also be valued as unique individuals. To address these conflicting demands, a multilayered conceptualisation of competence is proposed, with implications for the definition of standards and approaches to assessment. The model: Three layers are elaborated. This first is a core layer of canonical knowledge and skill, 'that, which every professional should possess', independent of the context of practice. The second layer is context‐dependent knowledge, skill, and attitude, visible through practice in health care. The third layer of personalised competence includes personal skills, interests, habits and convictions, integrated with one's personality. This layer, discussed with reference to Vygotsky's concept of Perezhivanie, cognitive load theory, self‐determination theory and Maslow's 'self‐actualisation', may be regarded as the art of medicine. We propose that fully matured professional competence requires all three layers, but that the assessment of each layer is different. Implications: The assessment of canonical knowledge and skills (Layer 1) can be approached with classical psychometric conditions, that is, similar tests, circumstances and criteria for all. Context‐dependent medical competence (Layer 2) must be assessed differently, because conditions of assessment across candidates cannot be standardised. Here, multiple sources of information must be merged and intersubjective expert agreement should ground decisions about progression and level of clinical autonomy of trainees. Competence as the art of medicine (Layer 3) cannot be standardised and should not be assessed with the purpose of permission to practice. The pursuit of personal excellence in this level, however, can be recognised and rewarded. The authors explain how assessment can evolve to support the future of medical education by suggesting canonical, contextual, and personalized layers to models of competence, thereby acknowledging tension between standards and individuality. [ABSTRACT FROM AUTHOR]

A working group at the 2012 Academic Emergency Medicine consensus conference on education research in emergency medicine (EM) convened to develop a curriculum for dedicated postgraduate fellowships in EM education scholarship. This fellowship is intended to create future education scholars, equipped with the skills to thrive in academic careers. This proceedings article reports on the consensus of a breakout session subgroup tasked with defining a common core content for education scholarship fellowships. The authors propose that the core content of an EM education scholarship fellowship can be categorized in four distinct areas: career development, theories of learning and teaching methods, education research methods, and educational program administration. This core content can be incorporated into curricula for education scholarship fellowships in EM or other fields and can also be adapted for use in general medical education fellowships.

Psychometrics has recently undergone extensive criticism within the medical education literature. The use of quantitative measurement using psychometric instruments such as response scales is thought to emphasize a narrow range of relevant learner skills and competencies. Recent reviews and commentaries suggest that a paradigm shift might be presently underway. We argue for caution, in that the psychometrics approach and the quantitative account of competencies that it reflects is based on a rich discussion regarding measurement and scaling that led to the establishment of this paradigm. Rather than reflecting a homogeneous discipline focused on core competencies devoid of consideration of context, the psychometric community has a history of discourse and debate within the field, with an acknowledgement that the techniques and instruments developed within psychometrics are heuristics that must be used pragmatically.

Internists are required to perform a number of procedures that require mastery of technical and non-technical skills, however, formal assessment of these skills is often lacking. The purpose of this study was to develop, implement, and gather validity evidence for a procedural skills objective structured clinical examination (PS-OSCE) for internal medicine (IM) residents to assess their technical and non-technical skills when performing procedures. Thirty-five first to third-year IM residents participated in a 5-station PS-OSCE, which combined partial task models, standardized patients, and allied health professionals. Formal blueprinting was performed and content experts were used to develop the cases and rating instruments. Examiners underwent a frame-of-reference training session to prepare them for their rater role. Scores were compared by levels of training, experience, and to evaluation data from a non-procedural OSCE (IM-OSCE). Reliability was calculated using Generalizability analyses. Reliabilities for the technical and non-technical scores were 0.68 and 0.76, respectively. Third-year residents scored significantly higher than first-year residents on the technical (73.5 vs. 62.2%) and non-technical (83.2 vs. 75.1%) components of the PS-OSCE (p < 0.05). Residents who had performed the procedures more frequently scored higher on three of the five stations (p < 0.05). There was a moderate disattenuated correlation (r = 0.77) between the IM-OSCE and the technical component of the PS-OSCE scores. The PS-OSCE is a feasible method for assessing multiple competencies related to performing procedures and this study provides validity evidence to support its use as an in-training examination.

Growth in the field of medical education is evidenced by the proliferation of units dedicated to advancing Medical Education Research and Innovation (MERI). While a review of the literature discovered narrative accounts of MERI unit development, we found no systematic examinations of the dimensions of and structures that facilitate the success of these units. We conducted qualitative interviews with the directors of 12 MERI units across Canada. Data were analyzed using qualitative description (Sandelowski in "Res Nurs Health" 23:334-340, 2000). Final analysis drew on Bourdieu's ("Outline of a theory of practice." Cambridge University Press, Cambridge, 1977; "Media, culture and society: a critical reader." Sage, London, 1986; "Language and symbolic power." Harvard University Press, Cambridge, 1991) concepts of "field," "habitus," and "capital", and more recent research investigating the "field" of MERI (Albert in "Acad Med" 79:948-954, 2004; Albert et al. in "Adv Health Sci Educ" 12:103-115, 2007). When asked about the metrics by which they define their success, directors cited: teaching, faculty mentoring, building collaborations, delivering conference presentations, winning grant funding, and disseminating publications. Analyzed using Bourdieu's concepts, these metrics are discussed as forms of capital that have been legitimized in the MERI field. All directors, with the exception of one, described success as being comprised of elements (capital) at both ends of the service-research spectrum (i.e., Albert's PP-PU structure). Our analysis highlights the forms of "habitus" (i.e., behaviors, attitudes, demeanors) directors use to negotiate, strategize and position the unit within their local context. These findings may assist institutions in developing a new--or reorganizing an existing--MERI unit. We posit that a better understanding of these complex social structures can help units become savvy participants in the MERI "field". With such insight, units can improve their academic output and their status in the MERI context-locally, nationally, and internationally.

Ongoing transformations in health professions education underscore the need for valid and reliable assessment. The current standard for assessment validation requires evidence from five sources: content, response process, internal structure, relations with other variables, and consequences. However, researchers remain uncertain regarding the types of data that contribute to each evidence source. We sought to enumerate the validity evidence sources and supporting data elements for assessments using technology-enhanced simulation. We conducted a systematic literature search including MEDLINE, ERIC, and Scopus through May 2011. We included original research that evaluated the validity of simulation-based assessment scores using two or more evidence sources. Working in duplicate, we abstracted information on the prevalence of each evidence source and the underlying data elements. Among 217 eligible studies only six (3%) referenced the five-source framework, and 51 (24%) made no reference to any validity framework. The most common evidence sources and data elements were: relations with other variables (94% of studies; reported most often as variation in simulator scores across training levels), internal structure (76%; supported by reliability data or item analysis), and content (63%; reported as expert panels or modification of existing instruments). Evidence of response process and consequences were each present in <10% of studies. We conclude that relations with training level appear to be overrepresented in this field, while evidence of consequences and response process are infrequently reported. Validation science will be improved as educators use established frameworks to collect and interpret evidence from the full spectrum of possible sources and elements.