Search

To further the development of a substantive theory of expert diagnostic practice, the authors explored the ways in which exceptional physicians, nominated by their peers, conceptualized their own diagnostic expertise specifically and diagnostic excellence generally.In this grounded theory, interview-based study, physicians at six North American research sites were nominated by their peers as exceptional diagnosticians and exceptional professionals and invited to participate in the study. A saturation sample included 34 participants, 20 exceptional diagnosticians, and 14 exceptional professionals. Using a constant comparative approach, the authors conducted one-on-one interviews with participants, transcribed the audiotapes of those interviews, and analyzed them for emergent themes. They developed a stable thematic structure and applied it to the entire data set.Four interconnected themes emerged that inform the community's understanding of how physicians conceptualize expert diagnostic practice: (1) possession of extensive knowledge built through purposeful, continuous engagement in clinical practice, (2) possession of the skills to effectively gather patient stories, (3) reflective integration of that knowledge and those patient stories during diagnosis, and (4) continuous learning through clinical practice.Exploring these results within the context of current discourses in medical education brings to the forefront two key features of physicians' construction of diagnostic excellence: (1) the integrated nature of the medical competencies that underpin the diagnostic process and (2) the optimally adaptive nature of the diagnostic process. These findings can inform the development of practical and effective pedagogical strategies to guide trainees, clinicians, and medical educators who strive for excellence.

Discuss ideas arising from this article at ‘discuss’.

To increase the rigor of pediatric rehabilitation research, there is a need to evaluate the degree to which an intervention is conducted as planned (i.e., fidelity). Generic fidelity measures evaluate more than one intervention and often include nonspecific attributes of the therapy process common to both interventions. The objective of this study was to describe the characteristics of generic fidelity measures and examine how these attributes fit with pediatric rehabilitation. A review of generic fidelity measures was conducted utilizing health and education databases. Five generic fidelity measures are described and examined for their applicability to pediatric rehabilitation. The measures were used in nine studies meeting the inclusion criteria, involving people ages from 11 years to65 years undergoing psychotherapy. From the 76 nonspecific items, 37 items were judged to be applicable to pediatric rehabilitation. Common characteristics of nonspecific attributes with pediatric rehabilitation are discussed, and investigator plans to conduct future testing.

Background: Although medical educators acknowledge the importance of ethics in medical training, there are few validated instruments to assess ethical decision-making. One instrument is the Ethics in Health Care Questionnaire – version 2 (EHCQ-2). The instrument consists of 12 scenarios, each posing an ethical problem in health care, and asking for a decision and rationale. The responses are subjectively scored in four domains: response, issue identification, issue sophistication, and values. Goals: This study was intended to examine the inter-rater and inter-case reliability of the AHCQ-2 and validity against a national licensing examination of the EHCQ-2 in an international sample. Methods: A total of 20 final year McMaster students and 45 final year Glasgow students participated in the study. All questionnaires were scored by multiple raters. Generalizability theory was used to examine inter-rater, inter-case and overall test reliability. Validity was assessed by comparing EHCQ-2 scores with scores on the Canadian written licensing examination, both total score and score for the ethics subsection. Results: For both samples, reliability was quite low. Except for the first task, which is multiple choice, inter-rater reliability was 0.08–0.54, and inter-case reliability was 0.14–0.61. Overall test reliability was 0.12–0.54. Correlation between EHCQ-2 task scores and the licensing examination scores ranged from 0.07 to 0.40; there was no evidence that the correlation was higher with the ethics subsection. Conclusions: The reliability and validity of the measure remains quite low, consistent with other measures of ethical decisionmaking. However, this does not limit the utility of the instrument as a tool to generate discussion on ethical issues in medicine.

Use of dual-processing has been widely touted as a strategy to reduce diagnostic error in clinical medicine. However, this strategy has not been tested among medical trainees with complex diagnostic problems. We sought to determine whether dual-processing instruction could reduce diagnostic error across a spectrum of experience with trainees undertaking cardiac physical exam. Three experiments were conducted using a similar design to teach cardiac physical exam using a cardiopulmonary simulator. One experiment was conducted in each of three groups: experienced, intermediate and novice trainees. In all three experiments, participants were randomized to receive undirected or dual-processing verbal instruction during teaching, practice and testing phases. When tested, dual-processing instruction did not change the probability assigned to the correct diagnosis in any of the three experiments. Among intermediates, there was an apparent interaction between the diagnosis tested and the effect of dual-processing instruction. Among relative novices, dual processing instruction may have dampened the harmful effect of a bias away from the correct diagnosis. Further work is needed to define the role of dual-processing instruction to reduce cognitive error. This study suggests that it cannot be blindly applied to complex diagnostic problems such as cardiac physical exam.

Medical Education 2012: 46: 326–335 Context The inadequacy of self-assessment as a mechanism to guide performance improvements has placed greater emphasis on the value of testing as a pedagogic strategy. The mechanism whereby testing influences learning is incompletely understood. This study was performed to examine which aspects of a testing experience most influence self-regulated learning behaviour among medical students. Methods Sixty-seven medical students participated in a computer-based, multiple-choice test. Initially, participants were instructed to attempt only items for which they felt confident of their response. They were then asked to indicate their best responses to deferred items. Students were then given an opportunity to review the items, with correct responses indicated. Accuracy, the attempt/defer decision and the time taken to reach this decision were recorded, along with participants’ ratings of their confidence in each response and the time spent reviewing each item on completion of the test. Results Students correctly answered a larger proportion of attempted items than deferred items (71% versus 40%; p

Background To understand the modern conceptions of elite practice informing the hidden curriculum through use of peer nominations asking clinicians to identify exceptional practitioners. Method We distributed a Web-based survey to Department of Medicine faculty at five universities in North America. Participants were asked to nominate individuals they deemed to be "outstanding practitioners" and to provide reasons. They were then asked to nominate "exceptional diagnosticians" and "exceptional professionals." Results Two hundred eighty-two physicians nominated 558 unique peers as "outstanding practitioners." Justifications included knowledge (45.1%), patient-related interpersonal skill (18.7%), teaching skill (10.8%), and research success (6.8%). More "exceptional diagnostician" nominees were nominated as "outstanding practitioners" (65.2%) relative to "exceptional professional" nominees (56.1%), although the effect size was small (phi = 0.09). Conclusions Knowledge-based competencies maintain a central role in modern conceptions of elite medical practice, although, contrary to the historical dominance of biomedical abilities, a diverse set of skills and professional aptitudes are also well represented.

CONTEXT Conceptualisations of self-assessment are changing as its role in professional development comes to be viewed more broadly as needing to be both externally and internally informed through activities that enable access to and the interpretation and integration of data from external sources. Education programmes use various activities to promote learners' reflection and self-direction, yet we know little about how effective these activities are in 'informing' learners' self-assessments. OBJECTIVES This study aimed to increase understanding of the specific ways in which undergraduate and postgraduate learners used learning and assessment activities to inform self-assessments of their clinical performance. METHODS We conducted an international qualitative study using focus groups and drawing on principles of grounded theory. We recruited volunteer participants from three undergraduate and two postgraduate programmes using structured self-assessment activities (e.g. portfolios). We asked learners to describe their perceptions of and experiences with formal and informal activities intended to inform self-assessment. We conducted analysis as a team using a constant comparative process. RESULTS Eighty-five learners (53 undergraduate, 32 postgraduate) participated in 10 focus groups. Two main findings emerged. Firstly, the perceived effectiveness of formal and informal assessment activities in informing self-assessment appeared to be both person- and context-specific. No curricular activities were considered to be generally effective or ineffective. However, the availability of high-quality performance data and standards was thought to increase the effectiveness of an activity in informing self-assessment. Secondly, the fostering and informing of self-assessment was believed to require credible and engaged supervisors. CONCLUSIONS Several contextual and personal conditions consistently influenced learners' perceptions of the extent to which assessment activities were useful in informing self-assessments of performance. Although learners are not guaranteed to be accurate in their perceptions of which factors influence their efforts to improve performance, their perceptions must be taken into account; assessment strategies that are perceived as providing untrustworthy information can be anticipated to have negligible impact.

Item does not contain fulltext PURPOSE: Informed self-assessment describes the set of processes through which individuals use external and internal data to generate an appraisal of their own abilities. The purpose of this project was to explore the tensions described by learners and professionals when informing their self-assessments of clinical performance. METHOD: This 2008 qualitative study was guided by principles of grounded theory. Eight programs in five countries across undergraduate, postgraduate, and continuing medical education were purposively sampled. Seventeen focus groups were held (134 participants). Detailed analyses were conducted iteratively to understand themes and relationships. RESULTS: Participants experienced multiple tensions in informed self-assessment. Three categories of tensions emerged: within people (e.g., wanting feedback, yet fearing disconfirming feedback), between people (e.g., providing genuine feedback yet wanting to preserve relationships), and in the learning/practice environment (e.g., engaging in authentic self-assessment activities versus "playing the evaluation game"). Tensions were ongoing, contextual, and dynamic; they prevailed across participant groups, infusing all components of informed self-assessment. They also were present in varied contexts and at all levels of learners and practicing physicians. CONCLUSIONS: Multiple tensions, requiring ongoing negotiation and renegotiation, are inherent in informed self-assessment. Tensions are both intraindividual and interindividual and they are culturally situated, reflecting both professional and institutional influences. Social learning theories (social cognitive theory) and sociocultural theories of learning (situated learning and communities of practice) may inform our understanding and interpretation of the study findings. The findings suggest that educational interventions should be directed at individual, collective, and institutional cultural levels. Implications for practice are presented. 01 september 2011

None

OBJECTIVES This study aimed to test the extent to which the use of medicalese (i.e. formal medical terminology and semantic qualifiers) alters the test performance of medical graduates; to tease apart the extent to which any observed differences are driven by language difficulties versus differences in medical knowledge; and to assess the impact of varying the language used to present clinical features on the ability of the test to consistently discriminate between candidates. METHODS Six clinical cases were manipulated in the context of pilot items on the Canadian national qualifying examination. Features indicative of two diagnoses were presented uniformly in lay terms, medical terminology and semantic qualifiers, respectively, and in mixed combinations (e.g. features of one diagnosis were presented using lay terminology and features of the other using medicalese). The rate at which the indicated diagnoses were named was considered as a function of language used, site of training, birthplace and medical knowledge (as measured by overall performance on the examination). RESULTS In the mixed conditions, Canadian medical graduates were not influenced by the language used to present the cases, whereas international medical graduates (IMGs) were more likely to favour the diagnosis associated with medical terminology relative to that associated with lay terms. This was true regardless of whether the entire sample or only North American-born candidates were considered. Within the IMG cohort, high performers were not influenced by the language manipulation, whereas low performers were. Uniform use of lay terminology resulted in the highest test reliability compared with the other experimental conditions. CONCLUSIONS The results indicate that the influence of medical terminology is driven more by substandard medical knowledge than by the language issues that challenge some candidates. Implications for both the assessment and education of medical professionals are discussed.

To determine how learners and physicians engaged in various structured interventions to inform self-assessment, how they perceived and used self-assessment in clinical learning and practice, and the components and processes comprising informed self-assessment and factors that influence these.This was a qualitative study guided by principles of grounded theory. Using purposive sampling, eight programs were selected in Canada, the United States, the United Kingdom, the Netherlands, and Belgium, representing low, medium, and high degrees of structure/rigor in self-assessment activities. In 2008, 17 focus groups were conducted with 134 participants (53 undergraduate learners, 32 postgraduate learners, 49 physicians). Focus-group transcripts were analyzed interactively and iteratively by the research team to identify themes and compare and confirm findings.Informed self-assessment appeared as a flexible, dynamic process of accessing, interpreting, and responding to varied external and internal data. It was characterized by multiple tensions arising from complex interactions among competing internal and external data and multiple influencing conditions. The complex process was evident across the continuum of medical education and practice. A conceptual model of informed self-assessment emerged.Central challenges to informing self-assessment are the dynamic interrelationships and underlying tensions among the components comprising self-assessment. Realizing this increases understanding of why self-assessment accuracy seems frequently unreliable. Findings suggest the need for attention to the varied influencing conditions and inherent tensions to progress in understanding self-assessment, how it is informed, and its role in self-directed learning and professional self-regulation. Informed self-assessment is a multidimensional, complex construct requiring further research.

Multidisciplinary research and policy work that focuses on identifying and mitigating sources of medical errors has proliferated in recent years. System-level safeguards to protect against human error now range from increasing use of information technology to avoid medication errors to having patients sign their own surgical sites as part of the consent process to avoid mistakes during operations. Despite this breadth of strategy, medical errors persist and research suggests a substantial human component. Diagnostic errors, a subset of all medical errors, have been reported to occur in 10–15% of all patient cases,1 and 74% of all diagnostic errors have been identified as having some cognitive basis.2 Understanding physicians’ cognitive processes is therefore a linchpin to error reduction, but there are multiple challenges that make it an especially difficult topic to research. Researchers have identified more than 30 cognitive biases,3 but most are believed to operate without conscious awareness, making it unrealistic to ask physicians which biases influence their judgment, how often, with which patients, and whether those biases lead them toward suboptimal decisions. Instead, labels are typically applied to diagnostic errors through post hoc explanations provided by case reviewers, a process that is itself susceptible to bias.4 The recent book How Doctors Think5 popularized the refrain that diagnosticians (and patients) must become more aware of decision-making biases to avoid falling prey to nonanalytic heuristics (cognitive shortcuts) like confirmation bias (preferentially seeking out information that confirms a hypothesis) and premature closure (concluding in favor of a particular diagnosis before sufficient information has been gathered). The latter issue has been identified as the single most prevalent cause of diagnostic error.1 Some researchers believe that making clinicians aware of the influence of nonanalytic processes (also known as “System 1” processes), such as pattern recognition, will help them overcome these misleading influences and the resulting certainty in their initial diagnostic impressions.6 Others argue that such biases are difficult to extinguish naturally because they represent fast and frugal ways of dealing with challenging problems that, in general, yield more benefit than harm.7 While this debate highlights the complexity of the issue and the amount of attention it has received, researchers still know relatively little about how the various biases identified affect physicians’ actual decision making and whether cognitive de-biasing strategies have potential to reduce errors in diagnosticians’ thinking. Therefore, implementing targeted and effective educational and policy interventions to reduce error remains an elusive goal. To partially address this knowledge gap, we conducted a study to examine diagnostic flexibility—that is, the extent to which physicians change their minds about the most likely diagnosis during a clinical encounter. By examining which characteristics of patients, diagnosticians, or physicians’ approaches to a particular case are predictive of physicians’ diagnostic flexibility, we may be able to more precisely guide continuing education efforts, facilitate self-monitoring habits, and better understand when cognitive biases yield diagnostic errors.

The drive toward competency-based education frameworks has created a tension between competing desires-for quantified, standardized measures on one hand, and for an authentic representation of what it means to be a good doctor on the other. The purpose of this study was to better understand the tensions that exist between competency frameworks and faculty's real-life experiences in evaluating residents.Interviews were conducted with 19 experienced internal medicine attendings at two Canadian universities in 2007. Attendings each discussed a specific outstanding, average, and problematic resident they had supervised. Interviews were analyzed using grounded theory.Eight major themes emerged reflecting how faculty conceptualize residents' performance: knowledge, professionalism, patient interactions, team interactions, systems, disposition, trust, and impact on staff. Attendings' impressions of residents did not seem to result from a linear sum of dimensions; rather, domains idiosyncratically took on variable degrees of importance depending on the resident. Relative deficiencies in outstanding residents could be overlooked, whereas strengths in problematic residents could be discounted. Some constructs (e.g., impact on staff) were not competencies at all; rather, they seem to act as explanations or evidence of attendings' opinions. Standardized evaluation forms might constrain authentic depictions of residents' performance.Despite concerted efforts to create standardized, objective, competency-based evaluations, the assessment of residents' clinical performance still has a strong subjective influence. Attendings' holistic impressions should not be considered invalid simply because they are subjective. Instead, assessment methods should consider novel ways of accommodating these impressions to improve evaluation.

Objectives In formulary committee deliberations, evidence for the efficacy of medications is often based on changes in the scale scores of patient-reported outcome measures. Our aim was to examine whether clinician judgement about the efficacy of medications for Alzheimer's disease, when added to scale score evidence, affects formulary committee members’ recommendations about providing these medications under public insurance. Methods The study was conducted using mixed methods. In a survey of formulary committee members in Canada, 32 participants were presented with scenarios that outlined different levels of efficacy for a medication. For each scenario, participants were asked to specify their likelihood of recommending that the medication be provided under public insurance. Of the 32 participants, 23 agreed to take part in an interview to explain the survey results. Content analysis was used to elicit recurrent themes across the interviews. Results When a medication was disease modifying, use of clinician judgement increased the mean likelihood of recommending that the medication be provided under public insurance. Despite this, some participants felt formulary committees should not use clinician judgement because of risks of subjectivity and bias. However, other participants believed the addition of clinician judgement would enhance the clinical relevance of evidence that might otherwise be based entirely on changes in scale score. Conclusions Clinician judgement about the efficacy of medications can influence formulary committee recommendations. This suggests the need for a new approach to govern the consideration of expert evidence during formulary committee deliberations.

This paper examines diagnostic error from an educational perspective. Rather than addressing the question of how educators in the health professions can help learners avoid error, however, the literature reviewed leads to the conclusion that educators should be working to induce error in learners, leading them to short term pain for long term gain. A variety of literatures are reviewed that suggest errors in performance are necessary pre-conditions for learning to occur such that an aversion to errors, while more comforting to the learner, is counter-productive. Similarly, research is reviewed that suggests strategies aimed at avoiding heuristic-driven diagnostic errors may successfully reduce those types of errors, but may do so at the expense of inducing errors of comprehensiveness. Taken together, the variety of studies contained suggest that diagnostic errors are often beneficial and that we as an educational community should strive to determine how to harness their pedagogical and diagnostic benefits rather than simply trying to eliminate mistakes entirely.

The purpose of this cross-cultural study was to compare the level of moral judgement between two groups of students over the period of their professional physical therapy educational programmes as measured by the Defining Issues Test (DIT). Students from two entry-level physical therapy programmes volunteered to participate. The DIT was completed at entry and exit of their respective programmes. DIT mean scores were compared using ancova controlling for age and grade point average. Thirty-eight female, second baccalaureate degree students of diverse religious backgrounds, living in a Western culture and 13 female, first baccalaureate degree students of Islamic religious backgrounds living in an Arabic culture participated. At both entry and exit of their programmes, students from the Western group scored significantly higher on the DIT than the Arabic group (average: 51.1 and 29.9 respectively, P

Healthcare professionals often encounter moral dilemmas in clinical practice that require increased responsibility and accountability for ethical decision-making. This paper reports the results of a 6-year longitudinal study that explored changes in moral judgement of five consecutive cohorts of occupational therapy (OT) and physical therapy (PT) students over the course of their professional training. The training programme included an ethics education component. The Defining Issues Test (DIT) developed at the University of Minnesota was used to measure moral judgement. A total of 548 students participated in the study. At entry into their professional training, the DIT scores of the OT and PT students were similar but higher overall than the norms established for college level students or for graduates from professional programmes in the DIT standardization sample. At the time of graduation, results showed no significant differences in moral judgement scores between males and females, their chosen programme of study (OT or PT), year of entry, or previous education. Comparing entry scores to exit scores from both programmes for 288 students who provided data at both times, we found that moral judgement scores increased significantly in both OT and PT students over the 2-year programme of study. No differences were found in scores across gender, programme, year of entry, or previous education. Implications are discussed for including a formal ethics education component in the curricula of all health professional training programmes. Recommendations for future research are outlined.

PurposeThe purpose of this paper is to determine the relevance and reliability of the ten‐item Consultation and Relational Empathy (CARE) Measure as a tool for measuring patients' views of anaesthetists during preoperative assessment consultations.Design/methodology/approachSelf‐completed patient questionnaire containing the ten‐item CARE Measure. Consecutive adult patients were asked to complete the ten‐item CARE questionnaire immediately after their pre‐operative assessment consultation with the anaesthetist and return it to a designated local co‐ordinator. Reliability co‐efficient of the overall measure, and relevance of each item to patients' concerns were measured.FindingsUsing the Measure, 31 consultant anaesthetists were assessed by 1,582 patients (559 male, 952 female). The total number of “not applicable” responses was 1,086, (6.8 per cent of the total number of possible “not applicable” responses). The overall number of missing values was 0.6 per cent. The measure effectively discriminated between doctors (reliability co‐efficient of the average score per doctor provided by 40 patients was above 0.8) and had high internal consistency (Cronbach's alpha, 0.93).Originality/valueThe present study presents evidence of a tool which may have utility in anaesthetics and other settings.

For patients with acute respiratory failure who have declined intubation and resuscitation or have chosen comfort measures only, noninvasive ventilation (NIV) may help them achieve important health or personal goals, or merely prolong the dying process.To determine clinicians' attitudes to and stated use of NIV for these patients.We developed an instrument to assess the attitudes of intensivists, pulmonologists, and respiratory therapists (RTs) toward the use of NIV for patients with acute respiratory failure near or at the end of life. After assessing its psychometric properties, we mailed the survey to these clinicians at 18 Canadian and two U.S. hospitals. We analyzed factors associated with stated use of NIV for do-not-resuscitate and comfort-measures-only patients.Overall, 104 of 183 (57%) physicians and 290 of 473 (61%) RTs participated. Two thirds of physicians include NIV during life support discussions with do-not-resuscitate patients at least sometimes, and 87% of RTs stated that NIV should be included in such discussions. For patients choosing comfort measures only, almost half of physicians reported including NIV as an option in their discussions at least sometimes, while fewer than half of RTs stated that these discussions should be conducted. Most (80%) physicians use NIV and most (80%) RTs are asked to initiate NIV for do-not-resuscitate patients with chronic obstructive pulmonary disease or cardiogenic pulmonary edema. Fewer clinicians report using NIV for do-not-resuscitate patients with underlying malignancy (59% of physicians, 69% of RTs) or for patients choosing comfort measures only (40% of physicians, 51% of RTs; p.001).For patients with do-not-resuscitate orders, many physicians use NIV, and many RTs are asked to initiate NIV, most often to treat chronic obstructive pulmonary disease and cardiogenic pulmonary edema. Further study is needed on the goals of NIV near the end of life, whether these goals are understood by all stakeholders, and how well they are achieved in practice.

It is generally well accepted in health professional education that self-assessment is a key step in the continuing professional development cycle. While there has been increasing discussion in the community pertaining to whether or not professionals can indeed self-assess accurately, much of this discussion has been clouded by the fact that the term self-assessment has been used in an unfortunate and confusing variety of ways. In this article we will draw distinctions between self-assessment (an ability), self-directed assessment seeking and reflection (pedagogical strategies), and self-monitoring (immediate contextually relevant responses to environmental stimuli) in an attempt to clarify the rhetoric pertaining to each activity and provide some guidance regarding the implications that can be drawn from making these distinctions. We will further explore a source of persistence in the community's efforts to improve self-assessment despite clear findings from a large body of research that we as humans do not (and, in fact, perhaps cannot) self-assess well by describing what we call a "they not we" phenomenon. Finally, we will use this phenomenon and the distinctions previously described to advocate for a variety of research projects aimed at shedding further light on the complicated relationship between self-assessment and other forms of self-regulating professional development activities.

In 2004, the Michael G. DeGroote School of Medicine at McMaster University in Hamilton, Ontario, developed the McMaster Community and Rural Education program (Mac-CARE), to coordinate core rotations for undergraduate and post-graduate medical learners in communities in Southern Ontario.The purpose of this study is to compare the academic performance of medical clerks learning at distributed sites to students who remained in Hamilton using four measures of academic performance.Progress test, OSCE, clerkship scores, and pre-clerkship tutorial-based evaluations were collected and Mac-CARE students were compared to non-Mac-CARE students on each performance measure using ANOVA.Outcomes are based on the first cohort to engage in Mac-CARE rotations. There were no statistically significant differences in academic performance between the 2 groups before the intervention rotation (pre-clerkship and clerkship evaluations, progress tests, or an inaugural OSCE). Mac-CARE students, however, scored higher on their post-clerkship OSCE than did non-Mac-CARE students.This study has shown that academic performance among students was at least comparable across all learning sites. To our knowledge, this is the first such study to be published within a Canadian context.

Context The implementation of an assessment system may be facilitated by stakeholder agreement that appropriate qualities are being tested. This study investigated the extent to which stakeholders perceived 8 assessment formats (multiple-choice questions, objective structured clinical examination, video, significant event analysis, criterion audit, multi-source feedback, case analysis and patient satisfaction questionnaire) as able to assess varying qualities of doctors training in UK general practice. Methods Educationalists, general practice trainers and registrars completed a blueprinting style of exercise to rate the extent to which each evaluation format was perceived to assess each of 8 competencies derived primarily from the General Medical Council document Good Medical Practice. Results There were high levels of agreement among stakeholders regarding the perceived qualities tested by the proposed formats (G = 0.82–0.93). Differences were found in participants’ perceptions of how well qualities were able to be assessed and in the ability of the respective formats to test each quality. Multi-source feedback (MSF) was expected to assess a wide range of qualities, whereas Probity, Health and Ability to work with colleagues were limited in terms of how well they could be tested by the proposed formats. Discussion Awareness of the perceptions of stakeholders should facilitate the development and implementation of workplace-based assessment (WPBA) systems. These data shed light on the acceptability of various formats in a way that will inform further investigation of WPBA formats’ validity and feasibility, while also providing evidence on which to base educational efforts regarding the value of each format.

OBJECTIVE Previous research has revealed a pedagogical benefit of instructing novice diagnosticians to utilise a combined approach to clinical reasoning (familiarity-driven pattern recognition combined with a careful consideration of the presenting features) when diagnosing electrocardiograms (ECGs). This paper reports 2 studies demonstrating that the combined instructions are especially valuable in helping students overcome biasing influences. METHODS Undergraduate psychology students were trained to diagnose 10 cardiac conditions via ECG presentation. Half of all participants were instructed to reason in a combined manner and half were given no explicit instruction regarding the diagnostic task. In Study 1 (n = 60), half of each group was biased towards an incorrect diagnosis through presentation of counter-indicative features. In Study 2 (n = 48), a third of the test ECGs were presented with a correct diagnostic suggestion, a third with an incorrect suggestion, and a third without a suggestion. RESULTS Overall, the instruction to utilise a combined reasoning approach resulted in greater diagnostic accuracy relative to leaving students to their own intuitions regarding how best to approach new cases. The effect was particularly pronounced when cases were made challenging by biasing participants towards an incorrect diagnosis, either through mention of a specific feature or by making an inaccurate diagnostic suggestion. DISCUSSION These studies advance a growing body of evidence suggesting that various diagnostic strategies identified in the literature on clinical reasoning are not mutually exclusive and that trainees can benefit from explicit guidance regarding the value of both analytic and non-analytic reasoning tendencies.

BACKGROUND: Because of common use of adolescent simulated patients (ASPs), clarification of the risk of negative effects associated with high-stress simulations is essential. This study evaluates the safety of suicidality simulations. METHOD: ASPs participated in a suicidality role or pediatric role. Explicit measures of their impact included the Suicidal Ideation Questionnaire (SIQ) and Reynolds Adolescent Depression Scale-2 (RADS-2), both of which were completed pre-/poststudy. Implicit behavioral measures of impact were also collected during training, including the Implicit Association Test. RESULTS: Three of the 24 participants had clinical RADS-2 and/or SIQ scores. None of these adolescents' mental status deteriorated with SP participation. ASPs in the suicidality role showed behavioral effects consistent with a negative reaction, and two reported brief depression. CONCLUSIONS: ASPs participated in a suicidality simulation without evidence of suicide contagion. However, ASPs' behavioral reactions and self-reported depression suggested a transient depressive reaction. Language: en

To understand clinicians’ perceptions regarding practice guidelines in Canadian intensive care units (ICUs) to inform guideline development and implementation strategies. We developed a self-administered survey instrument and assessed its clinical sensibility and reliability. The survey was mailed to ICU physicians and nurses in Canada to determine local ICU guideline development and use, and to compare physicians’ and nurses’ attitudes and preferences towards guidelines. The survey was completed by 51.6% (565/1095) of potential respondents. Although less than half reported a formal guideline development committee in their ICU, 81.0% reported that guidelines were developed at their institutions. Of clinicians who used guidelines in the ICU, 70.2% of nurses and 42.6% of physicians reported using them frequently or always. Professional society guidelines (with or without local modification) were reportedly used in most ICUs, but physicians were more confident than nurses of their validity (P < 0.001). Physicians considered endorsement of guidelines by a colleague more relevant for enhancing guideline use than did nurses (P < 0.001). Nurses considered low risk of the guideline and whether the guideline is consistent with their practice (P < 0.001) to be more relevant to guideline uptake than did physicians (P < 0.001). Lack of agreement with recommendations was a more important barrier to use of guidelines for physicians than for nurses (P < 0.001). Many Canadian institutions locally develop guidelines, and many ICU physicians and nurses report using them. Planning implementation strategies according to clinician preferences may increase guideline use. The nature of the differences in attitudes towards guidelines between nurses and physicians, and their impact on clinician adherence to guidelines requires further exploration.

INTRODUCTION Each of the two most commonly used five-level triage tools in North America, the Emergency Severity Index and the Canadian Triage and Acuity Scale have been used as a measure of emergency department resource utilization in addition to acuity. In both cases, it is believed that patients triaged as having a higher level of acuity require a greater number of emergency department resources. We compared the ability of each tool to predict the emergency department resources for each emergency department visit and associated hospital admission and in-hospital mortality rates. METHODS This is an observational, cohort study of a population-based random sample of patients triaged at two emergency departments over a 4-month period. Correlational analyses were performed to examine the relationship between the triage assessment and: (i) resource utilization, (ii) hospital admission, and (iii) in-hospital mortality. RESULTS From 486 patients, analyses revealed the greatest correlation was between Emergency Severity Index and diagnostic resources [-0.54 (95% confidence intervals: -0.58, -0.50)] and the poorest correlation was between Canadian Triage and Acuity Scale and mortality [-0.16 (95% confidence intervals: -0.20, -0.12)]. No statistically significant differences (P

Background Published studies of triage scale inter-rater reliability assessment have been conducted mostly using paper case scenarios. Objective To determine if this method of inter-rater reliability assessment generated significantly different measures from those generated from live triage cases. Methods This was a multicenter, prospective, observational cohort study of a population-based random sample of patients triaged at 2 emergency departments during a period of 4 months. All patients presenting to the emergency department within the study periods were simultaneously and independently triaged using a 5-level triage acuity scale by 2 to 3 research triage nurses blinded to each other's assessment and to the study objective. After 6 months, the same nurses were asked to assign triage scores to paper case scenarios of the same patients that they had each previously triaged. Results Each of the 9 research nurses triaged approximately 90 cases. The inter-rater reliabilities as measured by an intraclass correlation coefficient were 0.9 (95% CI=0.87 to 93) for the live triage assessments and 0.76 (95% CI=0.73 to 0.79) for the paper case scenarios. The mean triage score assigned to the live cases was significantly less than that assigned to the paper-based cases (3.17; 95% CI=3.08 to 3.26) (p Conclusions There is moderate to high agreement between live cases and paper case scenarios, and the inter-rater reliabilities, although significantly different, are acceptable in both cases. It is impossible to determine which triage setting provides a more accurate triage score but paper case scenarios generally receive lower triage scores than live cases.

This paper reports the results of a 2-year pilot study that involved 136 students from various health professions in 13 interprofessional education projects in north-western Ontario, Canada. The educational model was based on principles of problem-based, self-directed, small group learning and combined a clinical placement with a series of interprofessional tutorials and other shared learning experiences. Project evaluation entailed the use of both quantitative and qualitative outcome measures. Student ratings revealed a high level of learner satisfaction. There was no change in student perceptions of interprofessional collaboration between pre-test and post-test. A difference was observed between professions, with rehabilitation students having more positive perceptions than medical students. Qualitative analysis of student journals revealed four major themes: (i) new insights into interprofessional roles and the potential for collaboration; (ii) increased understanding of aboriginal culture, spirituality and health beliefs; (iii) new insights into healthcare system issues in rural and remote regions; and (iv) reflections on the benefits and challenges to interprofessional learning. Although barriers to implementing interprofessional education exist, the need to overcome them is critical in order to better prepare health professional students for collaborative practice within a changing healthcare system. Lessons learned and strategies for success are discussed.

Interprofessional teamwork should include interprofessional feedback to optimize performance and collaboration. Social identity theory predicts that hierarchy and stereotypes may limit receptiveness to interprofessional feedback, but literature on this is sparse. This study explores perceptions among health professions students regarding interprofessional peer feedback received after a team exercise.In 2012-2013, students from seven health professions schools (medicine, pharmacy, nursing, dentistry, physical therapy, dietetics, and social work) participated in a team-based interprofessional exercise early in clinical training. Afterward, they wrote anonymous feedback comments for each other. Each student subsequently completed an online survey to rate the usefulness and positivity (on five-point scales) of feedback received and guessed each comment's source. Data analysis included analysis of variance to examine interactions (on usefulness and positivity ratings) between profession of feedback recipients and providers.Of 353 study participants, 242 (68.6%) accessed the feedback and 221 (62.6%) completed the survey. Overall, students perceived the feedback as useful (means across professions = 3.84-4.27) and positive (means = 4.17-4.86). There was no main effect of profession of the feedback provider, and no interactions between profession of recipient and profession of provider regardless of whether the actual or guessed provider profession was entered into the analysis.These findings suggest that students have positive perceptions of interprofessional feedback without systematic bias against any specific group. Whether students actually use interprofessional feedback for performance improvement and remain receptive toward such feedback as they progress in their professional education deserves further study.

Background Health professions programs continue to search for meaningful and efficient ways to evaluate the quality of education they provide and support ongoing program improvement. Despite flaws inherent in self-assessment, recent research suggests that aggregated self-assessments reliably rank aspects of competence attained during preclerkship MD training. Given the novelty of those observations, the purpose of this study was to test their generalizability by evaluating an MD program as a whole. Method The Readiness for Residency Survey (RfR) was developed and aligned with the published Readiness for Clerkship Survey (RfC), but focused on the competencies expected to be achieved at graduation. The RfC and RfR were administered electronically four months after the start of clerkship and six months after the start of residency, respectively. Generalizability and decision studies examined the extent to which specific competencies were achieved relative to one another. Results The reliability of scores assigned by a single resident was G = 0.32. However, a reliability of G = 0.80 could be obtained by averaging over as few as nine residents. Whereas highly rated competencies in the RfC resided within the CanMEDS domains of professional, communicator, and collaborator, five additional medical expert competencies emerged as strengths when the program was evaluated after completion by residents. Conclusions Aggregated resident self-assessments obtained using the RfR reliably differentiate aspects of competence attained over four years of undergraduate training. The RfR and RfC together can be used as evaluation tools to identify areas of strength and weakness in an undergraduate medical education program.

Previous demonstrations of context effects in the forensic comparison sciences have shown that the number of "match" responses a person makes can be swayed by case information. Less clear is whether these effects are a result of changes in accuracy (e.g., discrimination ability), a shift in response bias (e.g., tendency to say "match" or "no match") or a mix of the 2. We present a series of experiments where we use a signal detection framework to examine the effects of case information (separately) on forensic comparison accuracy and response bias. We also explore the role of familiarity as 1 potential mechanism for case information to sway accuracy. In Experiment 1, case information about crimes perceived to be more severe swayed people to say "match" more, but had little bearing on their ability to discriminate matching and nonmatching fingerprint pairs. In Experiment 2, case information did affect accuracy when it was familiar (i.e., if a previous similar case was associated with a "match" then people were more likely to also rate the current case as a "match," even though it was not). Even when we blinded people to all extrinsic case information in Experiment 3, accuracy was significantly affected by the familiarity of the fingerprints. These results demonstrate that contextual factors can have different (and independent) influences on accuracy and response bias and that even subtle information can affect accuracy if it is sufficiently similar to the case or trace at hand.

Many assessment practices used in primary care rely upon judgements provided by individuals observing trainees or colleagues. Despite there being many reasons to view these observations as cognitively complex, the extent to which fallibility in judgement reflects mental workload has not been examined experimentally. The objective of this study was to evaluate the impact of increasing rating demands on rater-based assessments of clinical competence.Participants were randomly assigned to one of four conditions (in a 2×2 factorial design) and asked to rate three pre-recorded unscripted clinical encounters illustrating three levels of performance (high, medium, low). We looked at the effect on participants of having a larger (seven) or smaller (two) number of dimensions to rate, and/or distracting them with extraneous tasks (attending to patient status and the activity of additional individuals observable on video). Outcome measures included number of dimension-relevant behaviours identified, ability to differentiate between levels of performance, and inter-rater reliability.Using the two dimensions common to both groups, ANOVA revealed a significant effect of the number of dimensions included in the scale on the number of relevant behaviours identified: participants in the 2D group identified more features than those in the 7D group. Both groups were able to differentiate between levels of performance, but post hoc analyses revealed significance on all pairwise comparisons in the 2D group and not in the 7D group. Inter-rater reliability increased from 0.45 in the 7D group to 0.70 when participants were required to consider only two dimensions. By contrast, the distractions had little effect.The results of this study provide preliminary evidence that requiring raters to consider a greater number of dimensions can decrease (a) the number of dimension-relevant behaviours identified, (b) the capacity to differentiate between levels of performance, and (c) inter-rater reliability.

PURPOSE Laboratory studies have shown that performance assessment judgments can be biased by "contrast effects." Assessors' scores become more positive, for example, when the assessed performance is preceded by relatively weak candidates. The authors queried whether this effect occurs in real, high-stakes performance assessments despite increased formality and behavioral descriptors. METHOD Data were obtained for the 2011 United Kingdom Foundational Programme clinical assessment and the 2008 University of Alberta Multiple Mini Interview. Candidate scores were compared with scores for immediately preceding candidates and progressively distant candidates. In addition, average scores for the preceding three candidates were calculated. Relationships between these variables were examined using linear regression. RESULTS Negative relationships were observed between index scores and both immediately preceding and recent scores for all exam formats. Relationships were greater between index scores and the average of the three preceding scores. These effects persisted even when examiners had judged several performances, explaining up to 11% of observed variance on some occasions. CONCLUSIONS These findings suggest that contrast effects do influence examiner judgments in high-stakes performance-based assessments. Although the observed effect was smaller than observed in experimentally controlled laboratory studies, this is to be expected given that real-world data lessen the strength of the intervention by virtue of less distinct differences between candidates. Although it is possible that the format of circuital exams reduces examiners' susceptibility to these influences, the finding of a persistent effect after examiners had judged several candidates suggests that the potential influence on candidate scores should not be ignored.

Most medical school applicants are screened out preinterview. Some cognitive scores available preinterview and some noncognitive scores available at interview demonstrate reasonable reliability and predictive validity. A reliable preinterview noncognitive measure would relax dependence upon screening based entirely on cognitive tendencies.In 2005, applicants interviewing at McMaster University's Michael G. DeGroote School of Medicine completed an offsite, noninvigilated, Autobiographical Submission (ABS) preinterview and another onsite, invigilated, ABS at interview. Traditional and new ABS scoring methods were compared, with raters either evaluating all ABS questions for each candidate in turn (vertical scoring-traditional method) or evaluating all candidates for each question in turn (horizontal scoring-new method).The new scoring method revealed lower internal consistency and higher interrater reliability relative to the traditional method. More importantly, the new scoring method correlated better with the Multiple Mini-Interview (MMI) relative to the traditional method.The new ABS scoring method revealed greater interrater reliability and predictive capacity, thus increasing its potential as a screen for noncognitive characteristics.

There has been much debate in the medical education literature regarding the extent to which feature-driven and nonanalytic (similarity-based) reasoning strategies define expertise, but the relative value of teaching these strategies, together or in isolation, remains uncertain. The purpose of this study was to compare the diagnostic accuracy achieved upon receiving instruction to use each strategy in isolation to that of a combined approach.In 2003-04, 48 undergraduate psychology students from McMaster University in Ontario, Canada, were taught to diagnose ten cardiac disorders (including normal) via electrocardiogram (ECG) presentation. Twelve students were instructed to carefully identify all features present before assigning a diagnosis (feature first). Twelve were given the same instruction with notice that some test ECGs had been seen during training (implicit combined). Twelve were simply instructed to trust familiarity and diagnose based on this impression (similarity-based). Finally, 12 students were given feature first and similarity-based instructions in combination (explicit combined).No difference in diagnostic accuracy was observed between the groups given the feature first (42%) and first impression (41%) instructions (p.4), but the groups instructed to use both strategies (explicitly or implicitly) performed significantly better (56% and 53%, respectively; p.01).The results support an additive model of clinical reasoning in which instructions to be feature oriented and to trust similarity improve performance in novice diagnosticians.

Background: Achievement on grade point average and Medical College Admissions Test contribute as unintentional barriers to advancement of underrepresented minorities. So long as noncognitive measures mimic random number generators, they merely perpetuate such discrepancies. As reliable noncognitive measures are developed, it is crucial to ensure immunity from bias, enabling them to better dilute unintended discrimination of cognitive measures. Purpose: The Multiple Mini-Interview (MMI) is a recently developed, reliable (overall reliability =. 70), noncognitive measure used for assessment of medical school applicants. Our purpose in this study was to evaluate whether any suggestion of bias existed in application of the MMI in its assessment of aboriginal medical school applicants. Methods: In this study of the MMI (overall reliability =. 70), each of 5 self-declared aboriginal applicants and 7 general-pool applicants experienced the same 11 vetted interview stations with the same 6 aboriginal raters and 5 ...

A recent review of the physician performance literature concluded that the risk of prematurely closing one's diagnostic search increases with years of experience. To minimize confounding variables and gain insight into cognitive issues relevant to continuing education, the current study was performed to test this conclusion.Physician participants were shown a series of case histories and asked to judge the probability of a pair of diagnoses. The order in which features were presented was manipulated across participants and the probabilities compared to determine the impact of information order. Two groups of participants were recruited, 1 older than and 1 younger than 60 years.The probability assigned to a diagnosis tended to be greater when features consistent with that diagnosis preceded those consistent with an alternative than when the same features followed those consistent with the alternative. Older participants revealed a greater primacy effect than less experienced participants across 4 experimental conditions.Physicians with greater experience appear to weigh their first impressions more heavily than those with less experience. Educators should design instructional activities that account for experience-specific cognitive tendencies.

INTRODUCTION Heterogeneous results exist regarding the impact of security violations on student performances in objective structured clinical examinations (OSCEs). Three separate studies investigate whether anticipated security violations result in undesirable enhancement of MMI performance ratings. METHODS Study 1: low-stakes: MMI station stems provided to a random half of 57 medical school applicants 2 weeks in advance of participation in a research study. Study 2: high-stakes: 384 medical school applicants sat a 12-station MMI to determine admission. Each half received 1 of 2 pilot MMI station stems 2 weeks in advance. Study 3: high-stakes: 38 interviewees with dual applications to occupational therapy and physiotherapy experienced the same 7-station MMI twice on the same date. RESULTS No statistically significant differences in MMI performances were detected. CONCLUSIONS Predictable violations of MMI security do not unduly influence applicant performance ratings.

Contexte : Une des tâches principales de l’enseignant clinicien est d’aider les etudiants a faire le tri parmi les donnees du malade et a poser un diagnostic exact, tout en ayant comme objectif ultime un traitement approprie. Depuis les 30 dernieres annees, il y a eu de nombreux debats pour savoir quel modele decrit le mieux les processus mentaux utilises par les cliniciens experts pour prendre des decisions diagnostiques. But : Le but du present essai est de proceder a une revue des publications scientifiques consacrees a ce sujet afin de mieux renseigner les cliniciens qui enseignent au lit du malade ou en clinique. Les forces et faiblesses de differentes methodes d’investigation du raisonnement clinique seront evaluees en utilisant comme cadre conceptuel une des typologies fondamentales qui distingue les modeles de raisonnement analytique (raisonnement conscient, controle) par rapport aux modeles de raisonnement non-analytique (raisonnement inconscient, automatique). Discussion : Les resultats d’etudes recentes suggerent que les enseignants cliniciens devraient promouvoir l’importance des deux types de raisonnement, permettant ainsi aux etudiants de faire appel a des processus mentaux varies et appropries au contexte. Des conseils pratiques sont formules a l’intention des enseignants.

Background: In defining the characteristics of medical students that society and the medical profession find desirable, little effort has been spent assessing the relative value of the dozens of characteristics that have been identified. Furthermore, many institutions go to great lengths to ensure equal representation across stakeholder groups in an effort to maximize the heterogeneity of the pool of students accepted to study medicine; however, the extent to which different stakeholders value different characteristics has yet to be determined. Purpose: This study was an attempt to assess the relative value of the characteristics of medical students that society and the medical profession find desirable. Methods: Using documents created internationally to identify the core competencies of medical personnel, a series of 7 characteristics were generated for inclusion in a study that adopted the paired comparison technique. Of 347 surveyed, 292 respondents indicated the rank ordering they would assign to eac...