Search

Introduction: Competency‐based medical education (CBME) led to the widespread adoption of workplace‐based assessment (WBA) with the promise of achieving assessment for learning. Despite this, studies have illustrated tensions between the summative and formative role of WBA which undermine learning goals. Models of workplace‐based learning (WBL) provide insight, however, these models excluded WBA. This scoping review synthesizes the primary literature addressing the role of WBA to guide learning in postgraduate medical education, with the goal of identifying gaps to address in future studies. Methods: The search was applied to OVID Medline, Web of Science, ERIC and CINAHL databases, articles up to September 2020 were included. Titles and abstracts were screened by two reviewers, followed by a full text review. Two members independently extracted and analysed quantitative and qualitative data using a descriptive‐analytic technique rooted in Billett's four premises of WBL. Themes were synthesized and discussed until consensus. Results: All 33 papers focused on the perception of learning through WBA. The majority applied qualitative methodology (70%), and 12 studies (36%) made explicit reference to theory. Aligning with Billett's first premise, results reinforce that learning always occurs in the workplace. WBA helped guide learning goals and enhanced feedback frequency and specificity. Billett's remaining premises provided an important lens to understand how tensions that existed in WBL have been exacerbated with frequent WBA. As individuals engage in both work and WBA, they are slowly transforming the workplace. Culture and context frame individual experiences and the perceived authenticity of WBA. Finally, individuals will have different goals, and learn different things, from the same experience. Conclusion: Analysing WBA literature through the lens of WBL theory allows us to reframe previously described tensions. We propose that future studies attend to learning theory, and demonstrate alignment with philosophical position, to advance our understanding of assessment‐for‐learning in the workplace. The authors' synthesis of literature that focuses on the overlap between workplace learning and workplace assessment highlights gaps and identifies key strategies for enabling "assessment for learning" in the workplace. [ABSTRACT FROM AUTHOR]

Background: Many medical students may encounter a range of academic and personal challenges during their course of study, but very little is known about their experiences. Our aim was to review the literature to inform future scholarship and to inform policy change. Methods: A scoping review was conducted searching PubMed, MEDLINE, EMBASE, PsycInfo, British Education Index, Web of Science and ERIC for English language primary research with no date limits. This retrieved 822 papers of which eight met the requirements for inclusion in the review. Data were independently reviewed by two researchers and underwent thematic analysis by the research team. Results: Three major themes emerged. Theme 1: 'Identity preservation' addressed students' aim to preserve their sense of self in the face of academic difficulty and their tendency to seek support. This connected the apprehension many students expressed about their educational institutions to Theme 2: 'The dual role of the medical school'—medical schools are required to support struggling students but are predominantly seen as a punitive structure acting as the gatekeeper to a successful career in medicine. Students' apprehension and attempts to protect their identities within this complex landscape often resulted in 'maladaptive coping strategies' (Theme 3). Conclusion: Understanding and exploring the academic challenges faced by medical students through their own experiences highlight the need for the development of more individualised remediation strategies. Educators may need to do more to bridge the gap between students and institutions. There is a need to build trust and to work with students to enhance their sense of self and remediate approaches to engagement with learning, rather than focusing efforts on success in assessments and progression. Kirtchuk et al review qualitative studies of students' experiences with academic challenges. Their results highlight a need for institutions to build students' trust, enhance their self‐esteem, and improve engagement. [ABSTRACT FROM AUTHOR]

Background: Over the past few years, anatomy education has been revolutionized through digital media, resulting in innovative computer‐based 3D models to supplement or even replace traditional learning materials. However, the added value of these models in terms of learning performance remains unclear. Multiple mechanisms may contribute to the inconclusive findings. This study focusses on the impact of active manipulation on learning performance and the influence that posttest design features may have on the outcome measurement. Methods: Participants were randomly assigned to one of two research conditions: studying on the base of a computer‐based manipulable pelvic bone model versus online static images of the same model. Pretests focused on students' baseline anatomy knowledge and spatial ability. Three knowledge posttests were administered: a test based on a physical pelvic bone model, and two computer‐based tests based on static images and a manipulable model. Mental effort was measured with the Paas mental effort rating scale. Results: In the static images‐based posttest, significantly higher knowledge scores were attained by participants studying in the static images research condition (p = 0.043). No other significant knowledge‐related differences could be observed. In the manipulable model‐based posttest, spatial ability rather than the research condition seemed to have an influential role on the outcome scores (r = 0.18, p = 0.049). Mental effort scores reflected no difference between both research conditions. Conclusion: The research results are counter‐intuitive, especially because no significant differences were found in the physical model‐based posttest in students who studied with the manipulable model. Explaining the results builds on differences in anatomical models requiring less or more active manipulation to process spatial information. The pelvic bone manipulable model, and by extension osteology models, might be insufficiently complex to provide added value compared with static images. Moreover, the posttest modality should be chosen with care since spatial ability rather than anatomy knowledge may be measured. In recent years, digital tools have revolutionized anatomy education. This paper demonstrates that not all anatomical branches may profit from advanced 3D representations. [ABSTRACT FROM AUTHOR]

Introduction: The OSCE is a sociomaterial assemblage—a meshing together of human and material components producing multiple effects. Materials matter because they shape candidate performance, with potentially calamitous career consequences if materials influence performance unjustly. Although the OSCE literature refers to materials, few papers study the sociomateriality of OSCEs. Therefore, we explored OSCE stakeholders' talk about sociomaterial assemblages to better understand their importance for candidate performance. Methods: We conducted 15 focus groups with OSCE candidates (n = 42), examiners (n = 20) and simulated patients (n = 17) after an Australian postgraduate nursing OSCE. Sociomateriality informed our team‐based framework analysis of data. Results: Participants identified a multiplicity of OSCE materials (objects, technologies and spaces) thought to matter for candidate performance. Candidates' unfamiliarity with materials and missing or malfunctioning materials were reported to yield numerous negative impacts (eg cognitive overload, negative affect, time‐wasting), thereby adversely affecting candidate performance. Both examiners and candidates made micro‐adjustments to sociomaterial assemblages during the OSCE in order to make it work (eg candidates saying what they would do rather than doing it). Sometimes, such tinkering extended so far that sociomaterial assemblages were ruptured (eg examiners ignoring rubrics to help pass candidates), potentially influencing OSCE standardisation. Discussion: Our novel empirical study extends previous conceptual work by illustrating wide‐ranging sociomaterial assemblages influencing OSCE candidate performance. Further research is now needed employing sociomaterial approaches to further elucidate sociomaterial entanglements in diverse OSCEs. We encourage OSCE stakeholders to become more attuned to the productive nature of materials within all stages of OSCE design and implementation. The authors demonstrate the extent to which materials really matter in shaping candidate performance in OSCEs [ABSTRACT FROM AUTHOR]

The author recalls the article "Assessing Professional Competence: From Methods to Programmes" he co-wrote and published in a 2005 issue of "Medical Education" and the progress made since then. He stresses that most of the assessment methods have their limitations. He warns that attempt to combine such methods in in-training assessments could compromise their reliability. He cites his initiatives to develop the evaluation of assessment programmes in partnership with his colleagues.

An essay is presented which reflects on the report "Measuring the Process of Solving Clinical Diagnostic Problems" by Ray E. Helfer and Carl H. Slater. It examines how the study, published in the late 1960s, looks at the clinical diagnosis process and goes on to comment on how it represents that state of medical education during that period.

Context Thirty years ago, it was suggested in the Edinburgh Declaration that medical school applicants should be selected not only on academic, but also on non-academic, attributes. The main rationale behind extending medical school selection procedures with the evaluation of (non-academic) personal qualities is that this will lead to the selection of students who will perform better as a doctor than those who are selected on the basis of academic measures only. A second rationale is the expectation that this will lead to a representative health workforce as a result of reduced adverse impact. The aims of this paper are (i) to describe what can be learned about the use of selection criteria other than grades from over 15 years of Dutch experience and (ii) to summarise current knowledge on the issue of adverse impact in relation to non-grades-based selection. Methods A narrative review was undertaken of the (published) evidence that has resulted from non-grades-based school-specific selection procedures in the Netherlands and from recent explorations of the effect of the use of non-grades-based selection criteria on student diversity. Results The Dutch evidence is grouped into five key themes: the effect of participation in voluntary selection procedures, the assessment of pre-university extracurricular activities, the use of work samples, Dutch experiences with situational judgement tests and the effects of changing circumstances. This is followed by several lessons learned for medical schools that aim to increase their student diversity. Conclusion Over the last 30 years, important steps towards reliable and valid methods for measuring non-academic abilities have been taken. The current paper describes several lessons that can be learned from the steps taken in the Dutch context. The importance of sharing evidence gathered around the globe and building on this evidence to reach our goal of predicting who will be a good doctor is acknowledged. [ABSTRACT FROM AUTHOR]

Objectives: Gaining medical residency interviews has become more competitive and costly for medical students. Although limited evidence from residency programme directors indicates predictors for successfully matching into a programme, past research has not sufficiently explored the application components necessary to receive an interview offer. The present study will identify which application components are most helpful in obtaining interview offers for different medical specialties. Methods: Data were sourced from the Texas Seeking Transparency in Application to Residency (STAR), a survey of recently matched fourth‐year American medical students who self‐reported information on their residency application components and interview offers. Multi‐level logistic regression analyses were employed to predict the odds of interview offer according to applicants' academic, research and extracurricular characteristics. Sub‐analyses were conducted for each medical specialty. Results: Nearly 10 000 students reported information on over 419 010 applications submitted, which resulted in 164 696 interview offers. Across the sample, applicants had greater odds of receiving an interview offer if they had a geographic connection to the programme (odds ratio [OR] = 4.10, 95% confidence interval [CI] 4.00‐4.20), had completed an away rotation at the programme (OR = 16.00, 95% CI 14.92‐17.15), were Alpha Omega Alpha Honor Medical Society members (OR = 1.49, 95% CI 1.36‐1.64), or had been inducted into the Gold Humanism Honor Society (OR = 1.50, 95% CI 1.39‐1.62). Applicants had reduced odds of getting an interview if they had been required to remediate a course in medical school (OR = 0.73, 95% CI 0.64‐0.83) or had failed the US Medical Licensing Examination Step 1 or Step 2 examination on their first attempt (OR = 0.40, 95% CI 0.33‐0.47). Predictors of obtaining an interview varied by specialty. Conclusions: The present findings can assist senior medical students as they prepare residency applications and identify programmes to which they will apply. Knowledge of the significant factors can help applicants more efficiently use resources to maximise their number of interview offers. Completing away rotations and selecting programmes with which applicants have geographic connections may increase their odds of receiving interview offers. To shine light on the highly competitive world of residency selection, this paper explores the factors that alter the odds of candidates being offered an interview. [ABSTRACT FROM AUTHOR]

Context: Understanding self‐regulated learning (SRL) is complicated due to the different measures used to identify the key SRL processes. There is a growing trend in applying event measures of SRL (microanalysis and trace) but aptitude measures (questionnaires) continue to be widely used in medical education. A major concern is whether aptitude measures are a valid approach to capture the dimensions of SRL processes. This study examined correlations between SRL microanalysis, SRL trace and the Motivated Strategies for Learning Questionnaire (MSLQ) and how these measures were associated with biomedical science performance. Methods: An SRL microanalysis assessment interview was administered to 76 first‐year medical students individually when performing a biomedical science learning task. All written materials by students were collected for further trace analysis. Students completed an MSLQ 2 weeks before completing their biomedical science course. Correlation analyses were used to determine the correlations between the three SRL assessment measures. Bivariate and multiple analyses were conducted to compare students on different course or task performance using the three SRL assessment measures. Results: Microanalytic metacognitive monitoring (κ = 0.30, P <.001) and causal attributions (κ = 0.17, P =.009) had statistically significant correlations with use of the SRL trace strategy. MSLQ self‐efficacy correlated with microanalytic self‐efficacy (r =.39, P =.001). Bivariate tests showed that microanalytic metacognitive monitoring, causal attributions and adaptive inferences, and SRL trace strategy use had significant associations with task performance (P <.05). Microanalytic self‐efficacy, metacognitive monitoring and causal attributions, SRL trace strategy use and MSLQ self‐efficacy had significant associations with course performance (P <.05). Measures of use of the SRL trace strategy and MSLQ subscales did not show significant associations with task and course outcomes in multiple analyses (P >.05). Conclusions: Event measures, specifically SRL microanalysis, had greater associations with both task and course outcomes compared with the MSLQ measure. The SRL microanalysis is recommended for the assessment of SRL in biomedical science learning. However, to fully understand medical students' SRL a multidimensional assessment approach that combines event and aptitude measures should be used. Trying to measure engagement in self‐regulated learning? This paper stresses the value of prioritizing microanalysis given relationships with biomedical science learning outcomes. [ABSTRACT FROM AUTHOR]

Background: Deliberate reflection when practising the diagnosis of clinical cases has been shown to develop medical students' diagnostic competence. Adding guidance by cueing reflection or providing modelling of reflection increased the benefits of reflection for advanced (Years 5–6) students. The present study investigated whether we could replicate and extend these findings by comparing the effects of free, cued and modelled reflection on novice students' diagnostic competence. Methods: A total of 80 third‐year medical students participated in a two‐phase experiment. In the learning phase, students diagnosed nine clinical cases under one of three conditions: free reflection; cued reflection, and modelled reflection. Two weeks later, all students diagnosed four new examples of the diseases studied in the learning phase and four cases of non‐studied related diseases ('adjacent diseases'). The main outcome measurements were diagnostic accuracy scores (range 0–1) on studied and adjacent diseases. Results: For studied diseases, there was a significant effect of experimental condition on diagnostic accuracy (p < 0.02), with the cued‐reflection group (mean = 0.58, standard deviation [SD] = 0.23) performing significantly better than the free‐reflection group (mean = 0.41, SD = 0.20; p < 0.02). The cued‐reflection and modelled‐reflection groups (mean = 0.54, SD = 0.22) did not differ in diagnostic accuracy (p > 0.05), nor did the modelled‐reflection group perform better than the free‐reflection group (p > 0.05). For adjacent diseases, the three groups scored extremely low, without significant differences in performance (p > 0.05). Cued reflection and free reflection were rated as requiring similar effort (p > 0.05) and both were more demanding than studying examples of reflection (both p < 0.001) in the learning phase. Conclusions: Simply cueing novice students' reflection to focus it on relevant diseases was sufficient to increase diagnostic performance relative to reflection without any guidance. Cued reflection and studying examples of reflection appear to be equally useful approaches for teaching clinical diagnosis to novice students. Students found studying examples of reflection required less effort but cued reflection will certainly demand much less investment from teachers. Prompting deliberate reflection has been shown previously to facilitate improved diagnostic performance. In this paper it is reported that the cues to do so need not be elaborate to reveal their benefit. [ABSTRACT FROM AUTHOR]

Context Multiple-choice questions (MCQs) are frequently used to assess students in health science disciplines. However, few educators have formal instruction in writing MCQs and MCQ items often have item-writing flaws. The purpose of this study was to examine the impact of item-writing flaws on student achievement in high-stakes assessments in a nursing programme in an English-language university in Hong Kong. Methods From a larger sample, we selected 10 summative test papers that were administered to undergraduate nursing students in 1 nursing department. All test items were reviewed for item-writing flaws by a 4-person consensus panel. Items were classified as ‘flawed’ if they contained ≥ 1 flaw. Items not containing item-writing violations were classified as ‘standard’. For each paper, 2 separate scales were computed: a total scale which reflected the characteristics of the assessment as administered and a standard scale which reflected the characteristics of a hypothetical assessment including only unflawed items. Results The proportion of flawed items on the 10 test papers ranged from 28–75%; 47.3% of all items were flawed. Fewer examinees passed the standard scale than the total scale (748 [90.6%] versus 779 [94.3%]). Conversely, the proportion of examinees obtaining a score ≥ 80% was higher on the standard scale than the total scale (173 [20.9%] versus 120 [14.5%]). Conclusions Flawed MCQ items were common in high-stakes nursing assessments but did not disadvantage borderline students, as has been previously demonstrated. Conversely, high-achieving students were more likely than borderline students to be penalised by flawed items. [ABSTRACT FROM AUTHOR]

Objectives Given the absence of a common passing standard for students at UK medical schools, this paper compares independently set standards for common 'one from five' single-best-answer (multiple-choice) items used in graduation-level applied knowledge examinations and explores potential reasons for any differences. Methods A repeated cross-sectional study was conducted. Participating schools were sent a common set of graduation-level items (55 in 2013-2014; 60 in 2014-2015). Items were selected against a blueprint and subjected to a quality review process. Each school employed its own standard-setting process for the common items. The primary outcome was the passing standard for the common items by each medical school set using the Angoff or Ebel methods. Results Of 31 invited medical schools, 22 participated in 2013-2014 (71%) and 30 (97%) in 2014-2015. Schools used a mean of 49 and 53 common items in 2013-2014 and 2014-2015, respectively, representing around one-third of the items in the examinations in which they were embedded. Data from 19 (61%) and 26 (84%) schools, respectively, met the inclusion criteria for comparison of standards. There were statistically significant differences in the passing standards set by schools in both years (effect sizes ( f2): 0.041 in 2013-2014 and 0.218 in 2014-2015; both p < 0.001). The interquartile range of standards was 5.7 percentage points in 2013-2014 and 6.5 percentage points in 2014-2015. There was a positive correlation between the relative standards set by schools in the 2 years (Pearson's r = 0.57, n = 18, p = 0.014). Time allowed per item, method of standard setting and timing of examination in the curriculum did not have a statistically significant impact on standards. Conclusions Independently set standards for common single-best-answer items used in graduation-level examinations vary across UK medical schools. Further work to examine standard-setting processes in more detail is needed to help explain this variability and develop methods to reduce it. [ABSTRACT FROM AUTHOR]

Context Being able to accurately monitor learning activities is a key element in self-regulated learning in all settings, including medical schools. Yet students' ability to monitor their progress is often limited, leading to inefficient use of study time. Interventions that improve the accuracy of students' monitoring can optimise self-regulated learning, leading to higher achievement. This paper reviews findings from cognitive psychology and explores potential applications in medical education, as well as areas for future research. Cognitive Psychology Effective monitoring depends on students' ability to generate information ('cues') that accurately reflects their knowledge and skills. The ability of these 'cues' to predict achievement is referred to as 'cue diagnosticity'. Interventions that improve the ability of students to elicit predictive cues typically fall into two categories: (i) self-generation of cues and (ii) generation of cues that is delayed after self-study. Providing feedback and support is useful when cues are predictive but may be too complex to be readily used. Application to Medical Education Limited evidence exists about interventions to improve the accuracy of self-monitoring among medical students or trainees. Developing interventions that foster use of predictive cues can enhance the accuracy of self-monitoring, thereby improving self-study and clinical reasoning. First, insight should be gained into the characteristics of predictive cues used by medical students and trainees. Next, predictive cue prompts should be designed and tested to improve monitoring and regulation of learning. Finally, the use of predictive cues should be explored in relation to teaching and learning clinical reasoning. Conclusions Improving self-regulated learning is important to help medical students and trainees efficiently acquire knowledge and skills necessary for clinical practice. Interventions that help students generate and use predictive cues hold the promise of improved self-regulated learning and achievement. This framework is applicable to learning in several areas, including the development of clinical reasoning. [ABSTRACT FROM AUTHOR]

Context Basic science teaching in undergraduate medical education faces several challenges. One prominent discussion is focused on the relevance of biomedical knowledge to the development and integration of clinical knowledge. Although the value of basic science knowledge is generally emphasised, theoretical positions on the relative role of this knowledge and the optimal approach to its instruction differ. The present paper addresses whether and to what extent biomedical knowledge is related to the development of clinical knowledge. Methods We analysed repeated-measures data for performances on basic science and clinical knowledge assessments. A sample of 598 medical students on a traditional curriculum participated in the study. The entire study covered a developmental phase of 2 years of medical education. Structural equation modelling was used to analyse the temporal relationship between biomedical knowledge and the acquisition of clinical knowledge. Results At the point at which formal basic science education ends and clinical training begins, students show the highest levels of biomedical knowledge. The present data suggest a decline in basic science knowledge that is complemented by a growth in clinical knowledge. Statistical comparison of several structural equation models revealed that the model to best explain the data specified unidirectional relationships between earlier states of biomedical knowledge and subsequent changes in clinical knowledge. However, the parameter estimates indicate that this association is negative. Discussion Our analysis suggests a negative relationship between earlier levels of basic science knowledge and subsequent gains in clinical knowledge. We discuss the limitations of the present study, such as the educational context in which it was conducted and its non-experimental nature. Although the present results do not necessarily contradict the relevance of basic sciences, we speculate on mechanisms that might be related to our findings. We conclude that our results hint at possibly critical issues in basic science education that have been rarely addressed thus far. [ABSTRACT FROM AUTHOR]

Objectives Remediation is usually offered to medical students and doctors in training who underperform on written or clinical examinations. However, there is uncertainty and conflicting evidence about the effectiveness of remediation. The aim of this systematic review was to synthesise the available evidence to clarify how and why remediation interventions may have worked in order to progress knowledge on this topic. Methods The MEDLINE, EMBASE, CINAHL (Cumulative Index to Nursing and Allied Health Literature), ERIC (Educational Resources Information Centre), Web of Science and Scopus databases were searched for papers published from 1984 to April 2012, using the search terms 'remedial teaching', 'education', 'medical', 'undergraduate'/or 'clinical clerkship'/or 'internship and residency', 'at risk' and 'struggling'. Only studies that included an intervention, then provided retest data, and reported at least one outcome measure of satisfaction, knowledge, skills or effects on patients were eligible for inclusion. Studies of practising doctors were excluded. Data were abstracted independently in duplicate for all items. Coding differences were resolved through discussion. Results Thirty-one of 2113 studies met the review criteria. Most studies were published after 2000 ( n = 24, of which 12 were published from 2009 onwards), targeted medical students ( n = 22) and were designed to improve performance on an immediately subsequent examination ( n = 22). Control or comparison groups, conceptual frameworks, adequate sample sizes and long-term follow-up measures were rare. In studies that included long-term follow-up, improvements were not sustained. Intervention designs tended to be highly complex, but their design or reporting did not enable the identification of the active components of the remedial process. Conclusions Most remediation interventions in medical education focus on improving performance to pass a re-sit of an examination or assessment and provide no insight into what types of extra support work, or how much extra teaching is critical, in terms of developing learning. More recent studies are generally of better quality. Rigorous approaches to developing and evaluating remediation interventions are required. [ABSTRACT FROM AUTHOR]

This study aims to identify whether non-science graduates perform as well as science graduates in Basic and Clinical Sciences (B&CS) assessments during Years 1–3 of a four-year graduate-entry programme at the University of Sydney (the‘USydMP’).Students were grouped into five categories: Health Professions (HP), Biomedical Sciences (BMS), Other Biology (BIOL), Physical Sciences (PHYS) or Non-Science (NONS). We examined the performance rank of students in each of the five groups for single best answer (SBA) and modified essay (MEQ) assessments separately, and also calculated the relative risk of failure in the summative assessments in Years 2 and 3.Students with science-based prior degrees performed better in the SBA assessments. The same occurred initially in the MEQs, but the effect diminished with time. The HP students performed consistently better but converged with other groups over time, particularly in the MEQs. Relative performance by the NONS students improved with time in both assessment formats. Overall, differences between the highest and lowest groups were small and very few students failed to meet the overall standard for the summative assessments. HP and BMS students had the lowest failure rate. NONS students were more likely to fail the assessments in Year 2 and 3, but their pass rates were still high. Female students performed significantly better overall at the end of Year 2 and in Year 3. There were only minor differences between Australian resident and International students.While there are small differences in performance in B&CS early in the programme, these lessen with time. The study results will inform decisions regarding timing of summative assessments, selection policy and for providing additional support to students who need it to minimize their risk of failure. Readers should note that this paper refers to student performance in only one of the four curriculum themes, where health professional and science graduates would be expected to have a significant advantage. [ABSTRACT FROM AUTHOR]

Introduction: As interest in medicine grows, admissions committees must review an increasingly competitive pool of medical school and residency candidates. Nearly all admissions committees have moved towards a holistic review, which considers an applicant's experiences and attributes in addition to academic metrics. As such, identifying nonacademic predictors of success in medicine is necessary. Parallels between skills necessary to succeed in athletics and in medicine have been drawn, including teamwork, discipline and resiliency. This systematic review synthesises the current literature to evaluate the relationship between participation in athletics and performance in medicine. Methods: The authors searched five databases to conduct a systematic review following PRISMA guidelines. Included studies assessed medical students, residents or attending physicians in the United States or Canada and used prior athletic participation as a predictor or explanatory variable. The review examined associations between prior athletic participation and outcomes in medical school, residency and/or as an attending physician. Results: Eighteen studies evaluating medical students (78%), residents (28%) or attending physicians (6%) met inclusion criteria for this systematic review. Twelve (67%) studies specifically assessed participants based on skill level, and five (28%) studies specifically assessed participants based on type of athletic participation (team versus individual). Sixteen studies (89%) found that former athletes performed significantly better than their counterparts (p < 0.05). These studies found significant associations between prior athletic participation and better outcomes in multiple performance indicators, including exam scores, faculty ratings, surgical errors and burnout. Conclusions: Current literature, although limited, suggests that prior participation in athletics may be a predictor of success in medical school and residency. This was demonstrated through objective scoring methods, such as USMLE, and subjective outcomes, such as faculty ratings and burnout. Specifically, multiple studies indicate that former athletes demonstrated increased surgical skill proficiency and decreased burnout as medical students and residents. The authors systematically review the literatrue to demonstrate that prior participation in athletics may be a predictor of success in medical school and residency. [ABSTRACT FROM AUTHOR]

Skills examinations use a cut-off score of 50% to separate good and bad performing students. However, a fixed cut-off score is only associated with students ability to pass exams. The difficulty of the exam or variability between examiners in marking student's performance is not being taken into account. The Online Marking Tool incorporates various Global Rating Scales (GRS) marking professional competence as Pass, Borderline, Fail, Good and Excellent. Borderline Regression Analysis is used to determine a flexible cut-off score. The Online Marking Tool contains of an OSCE Management Tool to plan and execute Objective Structured Clinical Examinations (OSCE), a OSCE station bank and a Result Analysis tool including a fully-fledged Borderline Regression Analysis pack (Excel export pack). Individual (blinded) scores at item level are correlated with the GRS of Fail, Borderline, Pass, Good and Excellent or variations of this GRS (Borderline Fail and Borderline Pass; Fail, Borderline, Pass). Borderline Group Average, the average mean score of those students being marked as 'Borderline' is compared with Borderline Regression Method 1 and 2 (Borderline Fail e.g. Pass) to determine a flexible cut-off score. Twelve prestigious universities are currently using this unique online marking tool for clinical skills assessments. Over 200 OSCE were successfully administered and analysed using this software solution. Costs of the labour intense paper trail of regular OSCE are reduced by 70% and the error reduction in the paper trail is reduced by 30%. Approximately 19% more students fail after introducing Borderline Regression Analysis, together with a minimum amount of stations that need to be passed. [ABSTRACT FROM AUTHOR]

Background: Inequity in assessment can lead to differential attainment. Degree classifications, such as 'Honours', are an assessment outcome used to differentiate students after graduation. However, there are no standardised criteria used to determine what constitutes these awards. Methods: We contacted all medical schools in the UK and collected data relating to classifications awarded, criteria used and percentage of students receiving classifications across the 5‐year period prior to the 2019/2020 academic year. Results: All 42 UK medical schools responded, and 36 universities provided usable data. Of these 36 universities, 30 (83%) awarded classifications above a 'Pass'. We identified four classifications above a 'Pass', and these were 'Commendation', 'Merit', 'Distinction' and 'Honours'. Sixteen (44%) universities awarded a single additional classification, and 14 (39%) universities awarded two or more. There was considerable variation in the criteria used by each university to award classifications. For example, 30 (67%) out of 45 classifications were dependent on all examined years, 9 (20%) for a combination of years and 6 (13%) for final year alone. Twenty‐five of 30 universities that awarded classifications provided data on the percentage of students awarded a classification, and a median of 15% of students received any type of classification from their university (range 5.3% to 38%). There was a wide range in the percentage of students awarded each classification type across the universities (e.g. Honours, range = 3.1%–24%). Conclusions: We demonstrate considerable variation in the way UK medical degree classifications are awarded—regarding terminology, criteria and percentage of students awarded classifications. We highlight that classifications are another form of inequity in medical education. There is a need to fully evaluate the value of hierarchical degree awards internationally as the consequential validity of these awards is understudied. Degree classifications may be another form of inequity in medical education given variation in terminology, criteria, and percentage of students awarded. Byrne et al. argue a need to better understand the consequential validity of these awards. [ABSTRACT FROM AUTHOR]

The article describes how standard settings using audience response system (ARS) was established for medical college examination cut-off scores that distinguish between acceptable and non-acceptable performance. It states that the Angoff method of standard setting is time-consuming and laborious. Therefore, ARS with multiple-choice questions (MCQs) was used; it saved time and labour and eliminated the need for paper copies of test materials, which protected the integrity of the examination.

Introduction: Many countries are driving forward policies and practices to train medical students for later rural practice. Previous research has investigated individual (e.g., rural upbringing) and structural factors (e.g., curricular exposure) associated with rural practice intention. However, the relationship between academic performance in medical school and rural practice intention has been neglected, although optimisation theory suggests there may be a relationship. To address this gap, our aim was to identify the relationship between academic performance and rural practice intention. Methods: Data were collected via a cross‐sectional (self‐report) survey in 2021. Participants were students from 60 of the 96 rural order directed (RODs) medical programmes across China. We asked students their rural practice intention. We conducted univariate analyses to test for associations between rural practice intention and independent variables, including socio‐demographics, ROD location, grade year and academic performance measures. We used multilevel logistic regression models to test whether students' academic performance in medical school could be used to predict rural practice intention, holding the other factors constant. Results: There were 13 123 respondents, representing roughly 77.6% of the student population from the 60 schools. There was a statistically significant relationship between student (self)‐reported academic performance in medical school and rural practice intention. Higher performers had a lower likelihood (ORs: 0.65–0.78) of rural practice intention. This held across all performance measures (GPA rank, academic awards and student leadership) and for the sub‐group with rural upbringing (ORs: 0.68–0.78). Discussion: This is the first study to identify a relationship between medical school performance and rural practice intention. The findings suggest that students maximise their utility when choosing career options, with higher performers having lower rural practice intention. These data provide insight into the complexity of medical career decision making and can be used by medical school and workforce planners to inform rural training, recruitment and retention strategies. You et al. report that medical students with higher perceived academic performance have lower rural practice intention, illustrating an important factor in students' consideration of career options. [ABSTRACT FROM AUTHOR]

Objective Culturally and linguistically diverse ( CALD) students often experience difficulties with the clinical communication skills that are essential for successful interactions in the workplace. However, there is little evidence on the effectiveness of assessment and intervention strategies for this population. The two aims of this study were: to evaluate the effectiveness of assessment tools in identifying and describing the clinical communication difficulties of CALD health care students; and to determine whether communication programmes improved their clinical communication skills. Methods Systematic review based on the Cochrane protocol. Articles were identified through a search of established databases using Me SH and key search terms. Studies published in English from 1990 to March 2015 were included if they described assessment strategies or a training programme for communication skills of CALD students. Studies were excluded if they did not describe implementation of a specific assessment or intervention programme. Data were extracted independently by the first author and verified by the second author. Quality was measured by the Best Evidence Medical Education guide and the Educational Interventions Critical Appraisal Tool. The Kirkpatrick hierarchy was used to measure impact. Meta-analysis was not conducted because of the heterogeneity of programme design and outcome measures. Results One hundred and twenty-nine articles met the criteria for full text review. Eighty-six articles were excluded. Thirteen articles addressing assessment and 30 articles reporting on communication training programmes were included in this review. Assessment tools used rubrics and rating scales effectively. Intervention studies focused on speech and language skills ( n = 20), interpersonal skills ( n = 7) and faculty-level support ( n = 5). Although 17 studies reported positive findings on student satisfaction, only eight reported improved skills post-training. Conclusions The development of effective assessment and intervention programmes should have an integrated design and include specific outcome measures to increase educational impact. [ABSTRACT FROM AUTHOR]

Context Selection methods used by medical schools should reliably identify whether candidates are likely to be successful in medical training and ultimately become competent clinicians. However, there is little consensus regarding methods that reliably evaluate non-academic attributes, and longitudinal studies examining predictors of success after qualification are insufficient. This systematic review synthesises the extant research evidence on the relative strengths of various selection methods. We offer a research agenda and identify key considerations to inform policy and practice in the next 50 years. Methods A formalised literature search was conducted for studies published between 1997 and 2015. A total of 194 articles met the inclusion criteria and were appraised in relation to: (i) selection method used; (ii) research question(s) addressed, and (iii) type of study design. Results Eight selection methods were identified: (i) aptitude tests; (ii) academic records; (iii) personal statements; (iv) references; (v) situational judgement tests ( SJTs); (vi) personality and emotional intelligence assessments; (vii) interviews and multiple mini-interviews ( MMIs), and (viii) selection centres ( SCs). The evidence relating to each method was reviewed against four evaluation criteria: effectiveness (reliability and validity); procedural issues; acceptability, and cost-effectiveness. Conclusions Evidence shows clearly that academic records, MMIs, aptitude tests, SJTs and SCs are more effective selection methods and are generally fairer than traditional interviews, references and personal statements. However, achievement in different selection methods may differentially predict performance at the various stages of medical education and clinical practice. Research into selection has been over-reliant on cross-sectional study designs and has tended to focus on reliability estimates rather than validity as an indicator of quality. A comprehensive framework of outcome criteria should be developed to allow researchers to interpret empirical evidence and compare selection methods fairly. This review highlights gaps in evidence for the combination of selection tools that is most effective and the weighting to be given to each tool. [ABSTRACT FROM AUTHOR]

Background: Although used widely, there is limited evidence of the BioMedical Admissions Test's (BMAT) predictive validity and incremental validity over prior educational attainment (PEA). We investigated BMAT's predictive and incremental validity for performance in two undergraduate medical schools, Imperial College School of Medicine (ICSM), UK, and Lee Kong Chian School of Medicine (LKCMedicine), Singapore. Our secondary goal was to compare the evidence collected with published evidence relating to comparable tools. Methods: This was a retrospective cohort study of four ICSM (1188 students, entering 2010–2013) and three LKCMedicine cohorts (222 students, 2013–2015). We investigated associations between BMAT Section 1 ('Thinking Skills'), Section 2 ('Scientific Knowledge and Applications') and Section 3a ('Writing Task') scores, with written and clinical assessment performance across all programme years. Incremental validity was investigated over PEA (A‐levels) in a subset of ICSM students. Results: When BMAT sections were investigated independently, Section 2 scores predicted performance on all written assessments in both institutions with mainly small effect sizes (standardised coefficient ranges: ICSM: 0.08–0.19; LKCMedicine: 0.22–0.36). Section 1 scores predicted Years 5 and 6 written assessment performance at ICSM (0.09–0.14) but nothing at LKCMedicine. Section 3a scores only predicted Year 5 clinical assessment performance at ICSM with a coefficient <0.1. There were no positive associations with standardised coefficients >0.1 between BMAT performance and clinical assessment performance. Multivariable regressions confirmed that Section 2 scores were the most predictive. We found no clear evidence of incremental validity for any BMAT section scores over A‐level grades. Discussion: Schools who wish to assess scientific knowledge independently of A‐levels may find BMAT Section 2 useful. Comparison with previous studies indicates that, overall, BMAT seems less useful than comparable tools. Larger scale studies are needed. Broader questions regarding why institutions adopt certain admissions tests, including those with little evidence, need consideration. The authors use exploration of the BMAT to invite readers to reflect on why institutions adopt certain admissions tests even in the face of little utility evidence. [ABSTRACT FROM AUTHOR]

Introduction: Situational judgement tests (SJTs) have been widely adopted, internationally, into medical selection. It was hoped that such assessments could identify candidates likely to exhibit future professional behaviours. Understanding how performance on such tests may predict the risk of disciplinary action during medical school would provide evidence for the validity of such SJTs within student selection. It would also inform the implementation of such tests within student recruitment. Methods: This cohort study used data for 6910 medical students from 36 UK medical schools who sat the University Clinical Aptitude Test (UCAT) SJT in 2013. The relationship between SJT scores at application and the risk of subsequent disciplinary action during their studies was modelled. The incremental ability of the SJT scores to predict the risk of disciplinary action, above that already provided by UCAT cognitive test scores and secondary (high) school achievement, was also evaluated in 5535 of the students with information available on this latter metric. Results: Two hundred and ten (3.05%) of the students in the cohort experienced disciplinary action. The risk of disciplinary action reduced with increasing performance on the admissions SJT (odds ratio (OR) 0.80, 95% confidence interval (CI) 0.69 to 0.92, p = 0.002). This effect remained similar after adjusting for cognitive performance and prior academic attainment (OR 0.77, 95% CI 0.65 to 0.92, p = 0.004). The overall estimated effect‐size was small (Cohen's d = 0.08) and no evidence of 'threshold' effects were observed for the SJT scores and risk of disciplinary action. Conclusions: Performance on admissions SJTs can, at least modestly, incrementally predict the risk of subsequent disciplinary action, supporting their use in this context. However, for this SJT and outcome, there did not seem a distinct threshold score above which the risk of disciplinary action disproportionately increased. This should be considered when using the scores within medical selection. Performance on an admissions Situational Judgment Test is reported to predict risk of formal disciplinary action in medical students with implications for how such tests should be used to improve medical professionalism. [ABSTRACT FROM AUTHOR]

Objectives: The prevalence of peer‐assisted learning (PAL) featuring alongside the core medical curriculum is increasing; however, the evidence base for PAL's efficacy on academic performance is limited. This systematic review of randomised studies of PAL in medical school sets out to assess the impact of PAL on academic outcomes in medical school and evaluate whether PAL confers a benefit in specific educational contexts. Methods: A literature search was conducted across MEDLINE, Ovid Embase, Web of Science and Education Research Complete. Titles and abstracts were screened, and records were selected following strict eligibility criteria. Following full‐text assessment for eligibility, two reviewers independently extracted data from the final selection of records and a meta‐analysis was performed. Studies were classified using a modified version of Kirkpatrick's levels of learning. Student test scores were standardised by calculating the standardised mean difference (SMD). Results: Twenty‐seven randomised controlled trials were eligible for inclusion, and twenty‐one provided sufficient and complete data to enable meta‐analysis. There was a significant improvement in the academic performance of medical students who received PAL compared with those in the control group (SMD = 0.52 [95% confidence interval 0.18–0.85]; p =.003). The impact of PAL was greater amongst clinical medical students (SMD = 0.63; p =.02) than preclinical medical students (SMD = 0.39; p =.08) and when used for teaching practical skills (SMD = 0.69; p =.001) compared with theory (SMD = −0.11; p =.21). Students taught by PAL also achieved better results in assessments conducted more than four weeks after course completion (SMD = 1.20; p =.04). [Correction added on 8 November 2020 after first publication. The data values in the Results section have been corrected in this version.] Conclusions: Medical students experiencing PAL benefit in terms of academic performance, relative to those not receiving PAL. PAL is of greatest value in the clinical stages of training and for practical skills. The long‐term outcomes of PAL remain a priority for future research. Using a systematic review and meta‐analysis, the authors demonstrate that peer‐assisted learning's benefit to academic performance are sustained in the medium‐to‐long term. [ABSTRACT FROM AUTHOR]

Context Academic achievement and social class are positively related and applications to medical schools reflect a class-based bias favouring middle-class candidates. Applying a measure that is class-free could be useful as an indicator of a potential good health professional may widen the pool of applicants. In the Working in Health Access Programme (WHAP), we report on the potential usefulness of such a measure. In addition, we describe a programme for raising awareness of higher education (HE) and careers in health care. Methods Pupils attending schools with low HE participation rates sat a battery of psychometric tests assessing both cognitive and non-cognitive skills. A total of 2349 pupils sat the tests and 1000 of them took part in the subsequent activities. These pupils are being followed up and have obtained their Standard Grade (Year 11) examination results. Results Although social class influenced cognitive skills, it had no influence on non-cognitive abilities. Pupils with high levels of cognitive ability were found in all classes, including the most deprived. Both the testing and the awareness-raising programme were successful. Additional information was obtained on factors influencing the participants’ choices of career. Conclusions Psychometric assessment could act as an early indicator for identifying potential in young people. For pupils with both cognitive and non-cognitive attributes that might indicate they would be successful in veterinary or human medicine, appropriate advice regarding subject choices for school examinations and future careers could be provided. [ABSTRACT FROM AUTHOR]

Teachers and students regulate learning to varying degrees in educational programmes in higher education. We present evidence that students in a student-centred medical programme self- and co-regulate their learning in independently formed study groups. We describe the perceived benefits of study groups and the effect of study group membership on student achievement. Years 1–2 of a 4-year, graduate-entry problem-based medical programme. We surveyed 233 year 2 students about features of their study groups and their study group membership in years 1–2. We compared study group membership with students' scores on a written summative assessment held at the end of their second year. For students who joined 1 study group, the length of time their group stayed together was positively related to achievement in the written summative assessment. There were no differences in summative assessment results between students who had been in a study group and students who had not been in a study group. Effective study groups are supportive, socially cohesive groups who generate mutual trust and loyalty, and self- and co-regulate their learning by giving and receiving explanations and summaries and motivating individual study. Teachers can support the formation of study groups by using small-group teaching/learning activities, providing clear learning outcomes and assessment criteria, minimising competition for grades and allocating room space. [ABSTRACT FROM AUTHOR]

Context Ethnic minority students find that their ethnicity negatively affects the evaluation of their capacities and their feelings in medical school. This study tests whether ethnic minority and majority students differ in their 'self‐regulatory focus' in clinical training, that is, their ways to approach goals, due to differences in social learning experiences. Self‐regulatory focus consists of a promotion and prevention focus. People who are prone to stereotypes and unfair treatments are more likely to have a prevention focus and conceal certain identity aspects. The objectives of the study are to test whether ethnic minority students, as compared with ethnic majority students, are equally likely to have a promotion focus, but more likely to have a prevention focus in clinical training due to more negative social learning experiences (Hypothesis 1), and whether the relationship between student ethnicity and clinical evaluations can be explained by students' gender, social learning experiences, self‐regulatory focus and impression management (Hypothesis 2). Methods: Survey and clinical evaluation data of 312 (71.2% female) clerks were collected and grouped into 215 ethnic majority (69.4%) and 95 ethnic minority students (30.6%). Students' social learning experiences were measured as perceptions of unfair treatment, trust in supervisors and social academic fit. Self‐regulatory focus (general and work specific) and impression management were also measured. A parallel mediation model (Hypothesis 1) and hierarchical multiple regression analyses were used (Hypothesis 2). Results: Ethnic minority students had higher perceptions of unfair treatment and lower trust in their supervisors in clinical training. They were more prevention focused in clinical training, but this was not mediated by having more negative social learning experiences. Lower clinical evaluations for ethnic minority students were unexplained. Promotion focus in clinical training and trust in supervisors positively relate to clinical grades. Conclusion: Student ethnicity predicts social learning experiences, self‐regulatory focus and grades in clinical training. The hidden curriculum plausibly plays a role here. van Andel et al. demonstrate that systematic differences between ethnic minority and majority students can be predicted based on social learning experiences, self‐regulatory focus and clinical grades. [ABSTRACT FROM AUTHOR]

Processes involved in the regulation of learning have been researched for decades, because of its impact on academic and workplace performance. In fact, self‐regulated learning is the focus of countless studies in health professions education and higher education in general. While we will always need competent individuals who are able to regulate their own learning, developments in healthcare require a shift from a focus on the individual to the collective: collaboration within and between healthcare teams is at the heart of high‐quality patient care. Concepts of collaborative learning and collective competence challenge commonly held conceptualisations of regulatory learning and call for a focus on the social embeddedness of regulatory learning and processes regulating the learning of the collective. Therefore, this article questions the alignment of current conceptualisations of regulation of learning with demands for collaboration in current healthcare. We explore different conceptualisations of regulation of learning (self‐, co‐, and socially shared regulation of learning), and elaborate on how the integration of these conceptualisations adds to our understanding of regulatory learning in healthcare settings. Building on these insights, we furthermore suggest ways forward for research and educational practice. Bransen et al. explore the concept of 'self' to argue that the health professions need to shift from striving to optimize self‐regulation to understanding how to effectively regulate across self‐, co‐, and shared‐regulation. [ABSTRACT FROM AUTHOR]

Introduction: Assessment of healthcare professionals should be authentic to clinical practice. As clinicians regularly use resources in practice, similar resources should be available to those sitting assessment. There is limited information on the impacts of open‐book (resource) assessments on standard setting for use in high‐stakes assessments. This research aims to explore the effects on standard setting and student perceptions when open vs closed resources are available in high‐stakes assessment of medical students. Methods: Students sat multiple‐choice question (MCQ) examinations under both closed‐ and open‐resource conditions in a randomised crossover design. A standard setting panel set pass‐marks for both closed‐ and open‐resource conditions of delivery, and we compared these pass‐marks with each other and with actual performance. Students responded to a survey on perceptions of open‐resource assessments. Results: The pass‐mark was set higher when panellists considered open‐resource conditions compared to closed conditions (59% vs 47%), but actual student performance showed no difference in scores between the two conditions. The net effect was that the pass rate was higher for closed than open‐resource conditions (71% vs 34%). Open‐resource conditions increased the time to complete the questions. The students perceived that open resource was more authentic but was more time‐consuming and would require different preparation. Regarding the acceptability of including open resources in high‐stakes assessment, the responses of students were mixed. Discussion: Pass standards based solely on judgements by panellists experienced in closed‐resource conditions might not be applicable under open conditions. Questions vary in how much time accessing resources may take and the degree of assistance in selecting the correct answer. A programme of assessment could be constructed to include both closed‐ and open‐resource condition assessments depending on the questions' content and format. Open‐resource conditions may promote assessment preparation that focuses more on seeking and evaluating resources rather than learning facts. When student use of open‐resources is novel a disconnect emerges as standard setting panelists perceive a need to increase the pass‐mark while the availability of resources actually slows student responding, resulting in no effect on score. [ABSTRACT FROM AUTHOR]

Introduction: As educators seek to improve medical student well‐being, it is essential to understand the interplay between distress and important outcomes. Performance on Step 1 of the United States Medical Licensing Examination has played a significant role in selection for postgraduate residency positions in the United States and consequently has been a source of great stress for medical students. The purpose of this study was to examine whether student well‐being correlates with performance on a high stakes licensing examination. Methods: Between 2014 and 2016, three sequential cohorts of medical students at the University of Michigan Medical School completed the Medical Student Well‐Being Index (MSWBI) at the end of their 2nd‐year curriculum, shortly before taking Step 1. Associations between well‐being and Step 1 scores were investigated while adjusting for MCAT scores and cumulative second‐year course scores. Results: In total, 354 students were included in the analysis (68.1% of potential responders). On bivariate analysis, poor student well‐being (0 = low distress [high well‐being], 7 = high distress [poor well‐being]) was associated with lower Step 1 examination scores (slope = −2.10, P <.01), and well‐being accounted for 5% of overall Step 1 score variability (R2 =.05). However, after adjustment for MCAT scores and cumulative GPA (full model R2 =.51), the relationship between well‐being and Step 1 score was no longer significant (slope = −0.70, P‐value =.06). Conclusions: When controlling for metrics of academic performance, student well‐being prior to taking Step 1 was not associated with how well students performed on Step 1 for the study sample. Querying the relationship between well‐being and performance, the authors examined licensing exam data and found well‐being to account for only 5% of the variance, substantially less than academic metrics. [ABSTRACT FROM AUTHOR]

In this article the author discusses the education of medical students and the article "Believing is Seeing; How People’s Beliefs Influence Goals, Emotions and Behavior" by P. W. Teunissen and H. G. J. Bok. He is supportive of making changes in the medical education environment and is also supportive of several points which are raised in Teunissen and Bok's article.

Context: Medical school admissions committees use a variety of criteria to determine which candidates to admit to their programmes. Effective communication is increasingly considered a key requisite to the practice of effective medicine. Medical students with pre‐medical backgrounds in the humanities and social sciences may be more likely to acquire skills relevant to patient‐centred communication, either prior to or during medical school. Objectives: The purpose of this study was to investigate the relationship between pre‐medical backgrounds in the humanities and social sciences and outcomes in medical school, including in communication and interpersonal skills (CIS), licensure examination results and postgraduate specialty choice (primary care versus non‐primary care specialties). Methods: The American Medical College Application Service database was used to identify pre‐medical college majors, demographic characteristics, Medical College Admission Test scores and college grade point averages for medical students at a large, midwestern medical school. Data were obtained for 465 medical students across three cohorts (classes of 2014–2016). Correlation and regression analyses were used to examine relationships between pre‐medical background, performance on graduation competency examination standardised patient encounter CIS scores and on United States Medical Licensing Examination (USMLE) Step 1 and Step 2 Clinical Knowledge scores, and postgraduate specialty choice. Results: Graduating medical students with pre‐medical humanities or social sciences majors performed significantly better in terms of CIS than those with natural science majors (Cohen's d = 0.28, p = 0.011). There were no significant associations between pre‐medical majors and USMLE Step 1 and Step 2 Clinical Knowledge scores or postgraduate specialty choice. Conclusions: These results suggest that considering humanistic factors as part of admissions criteria may promote the selection and training of physicians with good communication skills. What value hath the humanities and social sciences? This study revealed that medical students with such backgrounds performed better on communication and interpersonal skills measures than peers with natural science backgrounds. [ABSTRACT FROM AUTHOR]

Context: Resources for medical education are becoming more constrained, whereas accountability in medical education is increasing. In this constrictive environment, medical schools need to consider and justify their selection procedures in terms of costs and benefits. To date, there have been no studies focusing on this aspect of selection. Objectives: We aimed to examine and compare the costs and benefits of two different approaches to admission into medical school: a tailored, multimethod selection process versus a lottery procedure. Our goal was to assess the relative effectiveness of each approach and to compare these in terms of benefits and costs from the perspective of the medical school. Methods: The study was conducted at Maastricht University Medical School, at which the selection process and a weighted lottery procedure ran in parallel for 3 years (2011–2013). The costs and benefits of the selection process were compared with those of the lottery procedure over three student cohorts throughout the Bachelor's programme. The extra costs of selection represented the monetary investment of the medical school in conducting the selection procedure; the benefits were derived from the increase in income generated by the prevention of dropout and the reductions in extra costs facilitated by decreases in the repetition of blocks and objective structured clinical examinations. Results: The tailor‐made selection procedure cost about €139 000 when extrapolated to a full cohort of students (n = 286). The lottery procedure came with negligible costs for the medical school. However, the average benefits of selection compared with the lottery system added up to almost €207 000. Conclusions: This study not only shows that conducting a cost–benefit comparison is feasible in the context of selection for medical school, but also that an 'expensive' selection process can be cost‐beneficial in comparison with an 'inexpensive' lottery system. We encourage other medical schools to examine the cost‐effectiveness of their own selection processes in relation to student outcomes in order to extend knowledge on this important topic. It's too expensive! Although feasibility is a common concern about selection practice, this study shows through cost‐benefit comparison that 'expensive' admissions processes can be cheaper than 'inexpensive' lotteries [ABSTRACT FROM AUTHOR]

Context: The flipped classroom (FC), reversing lecture and homework elements of a course, is popular in medical education. The FC uses technology‐enhanced pre‐class learning to transmit knowledge, incorporating in‐class interaction to enhance higher cognitive learning. However, the FC model is expensive and research on its effectiveness remains inconclusive. The aim of this study was to compare the efficacy of the FC model over traditional lecture‐based (LB) learning by meta‐analysis. Methods: We systematically searched MEDLINE, PubMed, ERIC, CINAHL, EMBASE, reference lists and Association for Medical Education in Europe (AMEE) conference books. Controlled trials comparing academic outcomes between the FC and LB approaches in higher education were considered eligible. The main findings were pooled using a random‐effects model when appropriate. Results: Forty‐six studies (9026 participants) were included, comprising four randomised controlled trials (RCTs), 19 quasi‐experimental studies and 23 cohort studies. Study populations were health science (n = 32) and non health science (n = 14) students. The risk of bias was high (36/37 articles). Meta‐analyses revealed that the FC had significantly better outcomes than the LB method in examination scores (post‐intervention and pre–post change) and course grades, but not in objective structured clinical examination scores. Subgroup analyses showed the advantage of the FC was not observed in RCTs, non‐USA countries, nursing and other health science disciplines and earlier publication years (2013 and 2014). Cumulative analysis and meta‐regression suggested a tendency for progressively better outcomes by year. Outcome assessments rarely focused on behaviour change. Conclusions: The FC method is associated with greater academic achievement than the LB approach for higher‐level learning outcomes, which has become more obvious in recent years. However, results should be interpreted with caution because of the high methodological diversity, statistical heterogeneity and risk of bias in the studies used. Future studies should have high methodological rigour, a standardised FC format and utilise assessment tools evaluating higher cognitive learning and behaviour change to further examine differences between FC and LB learning. [ABSTRACT FROM AUTHOR]

Context: Differential performance in postgraduate examinations between home medical graduates and those who qualified outside their country of practice is well recognised. This difference is especially marked in the practical component of the UK Membership of the Royal College of General Practitioners (MRCGP) examination. The potential causes of such disparities are not well understood. Methods: Data were available for 1874 international medical graduates who applied for general practice (GP) specialty training in the UK in 2008–2012. The primary outcome was performance in the Clinical Skills Assessment (CSA) OSCE component of the MRCGP. The main predictors were performance on a situational judgement test (SJT) and clinical problem‐solving test (CPST), a test of applied clinical knowledge, used in the selection for GP training. Data relating to demographic characteristics and English language fluency were also available. To better understand the relationship between the predictors, the selection measures and the outcome, a series of univariable and multivariable models were developed and tested, concluding with a structural equation model to explore causality. Results: The CSA rating was more strongly predicted by SJT scores (standardised beta, 0.26) than by performance on the CPST (standardised beta, 0.17). There was a relationship between English language fluency and CSA score that was mainly mediated via SJT performance. Conclusions: These findings demonstrate that performance on an SJT predicts performance in a high‐fidelity clinical simulation (the CSA) in international medical graduates. Although the constructs tested by SJTs are debated, and are likely to vary across settings, culturally appropriate knowledge of interpersonal competence is likely to be evaluated. Improving the confidence of doctors in this area through targeted educational interventions, rather than focusing on increased clinical knowledge, is likely to be more effective at reducing disparities observed in postgraduate examination performance. Thus, there are important implications for the design of specialty selection and licensing assessments globally. [ABSTRACT FROM AUTHOR]

Context: Single‐best‐answer questions (SBAQs) have been widely used to test knowledge because they are easy to mark and demonstrate high reliability. However, SBAQs have been criticised for being subject to cueing. Objectives: We used a novel assessment tool that facilitates efficient marking of open‐ended very‐short‐answer questions (VSAQs). We compared VSAQs with SBAQs with regard to reliability, discrimination and student performance, and evaluated the acceptability of VSAQs. Methods: Medical students were randomised to sit a 60‐question assessment administered in either VSAQ and then SBAQ format (Group 1, n = 155) or the reverse (Group 2, n = 144). The VSAQs were delivered on a tablet; responses were computer‐marked and subsequently reviewed by two examiners. The standard error of measurement (SEM) across the ability spectrum was estimated using item response theory. Results: The review of machine‐marked questions took an average of 1 minute, 36 seconds per question for all students. The VSAQs had high reliability (alpha: 0.91), a significantly lower SEM than the SBAQs (p < 0.001) and higher mean item–total point biserial correlations (p < 0.001). The VSAQ scores were significantly lower than the SBAQ scores (p < 0.001). The difference in scores between VSAQs and SBAQs was attenuated in Group 2. Although 80.4% of students found the VSAQs more difficult, 69.2% found them more authentic. Conclusions: The VSAQ format demonstrated high reliability and discrimination and items were perceived as more authentic. The SBAQ format was associated with significant cueing. The present results suggest the VSAQ format has a higher degree of validity. [ABSTRACT FROM AUTHOR]

Context: The journey through medical school can be challenging, especially for undergraduate medical students who must deal with a demanding curriculum, coupled with the demands of transitioning into adulthood. Despite experiencing learning challenges, most students succeed with appropriate learning support. Many medical schools offer learning support programmes, particularly in the latter years, but it has been suggested that such support could be more beneficial, especially during the initial years. Objectives: This review explores learning support intervention programmes used to address learning challenges and deficits in the first year of medical school. Additionally, we propose a potential framework for supporting learning during the first year of medical school. Methods: We searched PubMed, Web of Science, MEDLINE, CINAHL (Cumulative Index to Nursing and Allied Health Literature), ERIC (Educational Resources Information Centre), Academic Search Premier and Google Scholar using the search terms ‘learning support’, ‘learning challenge’, ‘remediation’, ‘change’, ‘medical education’ and ‘first year’. We developed and used a review matrix to record the main elements of each article. We also coded the matrix to identify emerging themes. Results: The main themes that emerged from the study were ‘intervention approaches’, ‘area of intervention’, ‘intervention strategies’, ‘intervention dose’ and ‘intervention outcomes’. Interventions: (i) used proactive‐deficit, reactive‐deficit and proactive‐developmental approaches; (ii) addressed content knowledge, academic success skills, personal and professional skills and programme‐related elements; (iii) utilised faculty staff‐facilitated, peer‐facilitated, support staff‐facilitated, experiential placement, self‐study and reduced‐load strategies; (iv) varied in length from 5 weeks to 2 years, and (v) generally showed positive results. Conclusions: This review has identified the main components of learning support interventions used for Year 1 medical students. Interventions, however, are generally not grounded on empirical assessment that elucidates the nature of the challenges faced by students. Future research should provide empirical understanding of the learning challenges to be addressed. [ABSTRACT FROM AUTHOR]

Objective Self-regulation is recognised as being a requisite skill for professional practice This study is part of a programme of research designed to explore efficient methods of feedback that improve medical students' ability to self-regulate their learning. Our aim was to clarify how students respond to different forms and content of written feedback and to explore the impact on study behaviour and knowledge acquisition. Methods Year 2 students in a 4-year graduate entry medical programme completing four formative progress tests during the academic year were randomised into three groups receiving different feedback reports. All reports included proportion correct overall and by clinical rotation. One group received feedback reports including lists of clinical presentations relating to questions answered correctly and incorrectly; another group received reports containing this same information in combination with response certitude. The final group received reports involving normative comparisons. Baseline progress test performance quartile groupings (a proxy for academic ability) were determined by results on the first progress test. A mixed-method approach with triangulation of research findings was used to interpret results. Outcomes of interest included progress test scores, summative examination results and measures derived from study diaries, questionnaires and semi-structured interviews. Results Of the three types of feedback provided in this experiment, feedback containing normative comparisons resulted in inferior test performance for students in the lowest performance quartile group. This type of feedback appeared to stimulate general rather than examination-focused study. Conclusions Medical students are often considered relatively homogenous and high achieving, yet the results of this study suggest caution when providing them with normative feedback indicating poorer performance relative to their peers. There is much need for further work to explore efficient methods of providing written feedback that improves medical students' ability to self-regulate their learning, particularly when giving feedback to those students who have the most room for improvement. [ABSTRACT FROM AUTHOR]

Context Predictive validity studies on the use of the multiple mini-interview ( MMI) have been primarily in medicine. Objectives This study sought to determine the predictive validity of the MMI for performance within a pharmacy programme and on the Pharmacy Examining Board of Canada ( PEBC) Qualifying Examination for licensure, and to compare the predictive validity of the MMI with that of pre-pharmacy grade point average ( GPA) and Pharmacy College Admission Test ( PCAT) score. Methods Admissions data for 223 graduates of the pharmacy programme at the University of Toronto were matched to programme and licensure outcome measures. Multiple linear regression assessed the predictive ability of the MMI, pre-pharmacy GPA, PCAT and covariates for performance in final-year experiential rotations, cumulative GPA ( cGPA) and PEBC- MCQ (multiple-choice question examination) and PEBC- OSCE (objective structured clinical examination) overall and subcomponent scores. Results The PCAT, pre-pharmacy GPA and age significantly predicted the PEBC- MCQ overall score. The MMI was the only significant predictor of overall score on the PEBC- OSCE (β = 0.17, p = 0.02); it also predicted communication and performance subscores. Scores on the PCAT and female gender predicted the communication subscore. Pre-pharmacy GPA, age and female gender significantly predicted cGPA. The MMI was the only significant predictor of institutional/ambulatory rotation score (β = 0.26, p = 0.00). Conclusions The MMI, designed to measure non-academic attributes including communication, motivation and problem-solving skills, was the only admissions tool with significant predictive validity for performance on the PEBC- OSCE national pharmacy certification examination and in an institutional/ambulatory rotation. These findings, from a single cohort of undergraduates, provide the first report of the predictive validity of the MMI for performance on pharmacy licensure examinations and thereby strengthen the evidence for its use in health professions selection. Prior university academic performance significantly predicted cGPA and performance on the PEBC- MCQ. Performance on the PCAT also predicted PEBC- MCQ results. [ABSTRACT FROM AUTHOR]

Context Developed by Jacobson and Truax, the reliable change index ( RCI) provides a measure of whether the change in an individual's score over time is within or beyond that which might be accounted for by measurement variability. In combination with measures of whether an individual's final score is closer to those of one population or another, this provides useful individual-level information that can be used to supplement traditional analyses. Objectives This article aims to highlight the potential of the RCI for use within medical education, particularly as a novel means of monitoring progress at the student level across successive test occasions or academic years. Methods We provide an example of how the RCI can be applied informatively to assessment evaluation, and discuss its wider usage. Conclusions The RCI approach can be used to identify and support failing students, as well as to determine best teaching and learning practices by identifying high-performing students. Furthermore, the individual-level nature of the RCI makes it well suited for educational research with small cohorts, as well as for tracking individual profiles within a larger cohort or addressing questions about individual performance that may be unanswerable at group level. [ABSTRACT FROM AUTHOR]

Context Over nearly two decades, a wealth of literature describing the various capabilities, uses and adaptations of virtual microscopy ( VM) has been published. Many studies have investigated the effects on and benefits to student learning of VM compared with optical microscopy ( OM). Objectives This study statistically aggregated the findings of multiple comparative studies through a meta-analysis in order to summarise and substantiate the pedagogical efficacy of teaching with VM. Methods Using predefined eligibility criteria, teams of paired researchers screened the titles and abstracts of VM studies retrieved from seven different databases. After two rounds of screening, numerical and thematic data were extracted from the eligible studies for analysis. A summary effect size and estimate of heterogeneity were calculated to determine the effects of VM on learner performance and the amount of variance between studies, respectively. Trends in student perceptions were also analysed and reported. Results Of the 725 records screened, 72 studies underwent full-text review. In total, 12 studies were viable for meta-analysis and additional studies were reviewed to extract themes relating to learners' perceptions of VM. The meta-analysis detected a small yet significant positive effect on learner performance (standardised mean difference 0.28, 95% confidence interval 0.09-0.47; p = 0.003), indicating that learners experience marked knowledge gains when exposed to VM over OM. Variation among studies was evident as high heterogeneity was reported. An analysis of trends in learner perceptions noted that respondents favoured VM over OM by a large margin. Conclusions Although many individual studies have reported non-significant findings in comparisons of VM and OM, the enhanced power afforded by meta-analysis revealed that the pedagogical approach of VM is modestly superior to that of OM and is preferred by learners. [ABSTRACT FROM AUTHOR]

Objective The overall aim of this review is to map the area around the topic of the relationship between physical space and learning and to then draw further potential implications from this for the specific area of health profession education. Methods The nature of the review is a scoping review following a 5-step-model by Arksey & O'Malley. The charting of the data has been conducted with the help of the networked learning landscape framework from Nordquist and Laing. Results The majority of the research studies on classroom-scale level have focused on how technology may enable active learning. There are no identified research studies on the building-scale level. Hence, the alignment of curricula and physical learning spaces has scarcely been addressed in research from other sectors. In order to 'create a field', conclusions from both case studies and research in related areas must be identified and taken into account to provide insights into health profession education. Four areas have been identified as having potential for future development in health profession education: (i) active involvement of faculty members in the early stages of physical space development; (ii) further development of the assessment strategies for evaluating how physical space impacts learning; (iii) exploration of how informal spaces are being developed in other sectors; and (iv) initiating research projects in HPE to study how informal spaces impact on students' learning. Conclusion Potentially, the results of this scoping review will result in better future research questions and better-designed studies in this new and upcoming academic field of aligning physical learning spaces and curricula in health profession education. [ABSTRACT FROM AUTHOR]

Context This study is based on the premise that the game of 'Twenty Questions' ( TQ) tests the knowledge people acquire through their lives and how well they organise and store it so that they can effectively retrieve, combine and use it to address new life challenges. Therefore, performance on TQ may predict how effectively medical school applicants will organise and store knowledge they acquire during medical training to support their work as doctors. Objectives This study was designed to determine whether TQ game performance on medical school entrance predicts performance on a clinical performance examination near graduation. Methods This prospective, longitudinal, observational study involved each medical student in one class playing a game of TQ on a non-medical topic during the first week of medical school. Near graduation, these students completed a 14-case clinical performance examination. Performance on the TQ task was compared with performance on the clinical performance examination. Results The 24 students who exhibited a logical approach to the TQ task performed better on all senior clinical performance examination measures than did the 26 students who exhibited a random approach. Approach to the task was a better predictor of senior examination diagnosis justification performance than was the Medical College Admission Test ( MCAT) Biological Science Test score and accounts for a substantial amount of score variation not attributable to a co-relationship with MCAT Biological Science Test performance. Conclusions Approach to the TQ task appears to be one reasonable indicator of how students process and store knowledge acquired in their everyday lives and may be a useful predictor of how they will process the knowledge acquired during medical training. The TQ task can be fitted into one slot of a mini medical interview. [ABSTRACT FROM AUTHOR]

Context Despite evidence for the predictive value of both pre-admission characteristics and past performance at medical school, their relative contribution to predicting medical school performance has not been thoroughly investigated. Objectives This study was designed to determine the relative importance of pre-admission characteristics and past performance in medical school in predicting student performance in pre-clinical and clinical training. Methods This longitudinal prospective study followed six cohorts of students admitted to a Dutch, 6-year, undergraduate medical course during 2002-2007 ( n = 2357). Four prediction models were developed using multivariate logistic regression analysis. Main outcome measures were 'Year 1 course completion within 1 year' (models 1a, 1b), 'Pre-clinical course completion within 4 years' (model 2) and 'Achievement of at least three of five clerkship grades of ≥ 8.0' (model 3). Pre-admission characteristics (models 1a, 1b, 2, 3) and past performance at medical school (models 1b, 2, 3) were included as predictor variables. Results In model 1a - including pre-admission characteristics only - the strongest predictor for Year 1 course completion was pre-university grade point average ( GPA). Success factors were 'selected by admission testing' and 'age > 21 years'; risk factors were 'Surinamese/Antillean background', 'foreign pre-university degree', 'doctor parent' and male gender. In model 1b, number of attempts and GPA at 4 months were the strongest predictors for Year 1 course completion, and male gender remained a risk factor. Year 1 GPA was the strongest predictor for pre-clinical course completion, whereas being male or aged 19-21 years were risk factors. Pre-clinical course GPA positively predicted clinical performance, whereas being non-Dutch or a first-generation university student were important risk factors for lower clinical grades. Nagelkerke's R2 ranged from 0.16 to 0.62. Conclusions This study not only confirms the importance of past performance as a predictor of future performance in pre-clinical training, but also reveals the importance of a student's background as a predictor in clinical training. These findings have important practical implications for selection and support during medical school. [ABSTRACT FROM AUTHOR]