Search

Introduction: Competency‐based medical education (CBME) led to the widespread adoption of workplace‐based assessment (WBA) with the promise of achieving assessment for learning. Despite this, studies have illustrated tensions between the summative and formative role of WBA which undermine learning goals. Models of workplace‐based learning (WBL) provide insight, however, these models excluded WBA. This scoping review synthesizes the primary literature addressing the role of WBA to guide learning in postgraduate medical education, with the goal of identifying gaps to address in future studies. Methods: The search was applied to OVID Medline, Web of Science, ERIC and CINAHL databases, articles up to September 2020 were included. Titles and abstracts were screened by two reviewers, followed by a full text review. Two members independently extracted and analysed quantitative and qualitative data using a descriptive‐analytic technique rooted in Billett's four premises of WBL. Themes were synthesized and discussed until consensus. Results: All 33 papers focused on the perception of learning through WBA. The majority applied qualitative methodology (70%), and 12 studies (36%) made explicit reference to theory. Aligning with Billett's first premise, results reinforce that learning always occurs in the workplace. WBA helped guide learning goals and enhanced feedback frequency and specificity. Billett's remaining premises provided an important lens to understand how tensions that existed in WBL have been exacerbated with frequent WBA. As individuals engage in both work and WBA, they are slowly transforming the workplace. Culture and context frame individual experiences and the perceived authenticity of WBA. Finally, individuals will have different goals, and learn different things, from the same experience. Conclusion: Analysing WBA literature through the lens of WBL theory allows us to reframe previously described tensions. We propose that future studies attend to learning theory, and demonstrate alignment with philosophical position, to advance our understanding of assessment‐for‐learning in the workplace. The authors' synthesis of literature that focuses on the overlap between workplace learning and workplace assessment highlights gaps and identifies key strategies for enabling "assessment for learning" in the workplace. [ABSTRACT FROM AUTHOR]

Background: Many medical students may encounter a range of academic and personal challenges during their course of study, but very little is known about their experiences. Our aim was to review the literature to inform future scholarship and to inform policy change. Methods: A scoping review was conducted searching PubMed, MEDLINE, EMBASE, PsycInfo, British Education Index, Web of Science and ERIC for English language primary research with no date limits. This retrieved 822 papers of which eight met the requirements for inclusion in the review. Data were independently reviewed by two researchers and underwent thematic analysis by the research team. Results: Three major themes emerged. Theme 1: 'Identity preservation' addressed students' aim to preserve their sense of self in the face of academic difficulty and their tendency to seek support. This connected the apprehension many students expressed about their educational institutions to Theme 2: 'The dual role of the medical school'—medical schools are required to support struggling students but are predominantly seen as a punitive structure acting as the gatekeeper to a successful career in medicine. Students' apprehension and attempts to protect their identities within this complex landscape often resulted in 'maladaptive coping strategies' (Theme 3). Conclusion: Understanding and exploring the academic challenges faced by medical students through their own experiences highlight the need for the development of more individualised remediation strategies. Educators may need to do more to bridge the gap between students and institutions. There is a need to build trust and to work with students to enhance their sense of self and remediate approaches to engagement with learning, rather than focusing efforts on success in assessments and progression. Kirtchuk et al review qualitative studies of students' experiences with academic challenges. Their results highlight a need for institutions to build students' trust, enhance their self‐esteem, and improve engagement. [ABSTRACT FROM AUTHOR]

Background: Over the past few years, anatomy education has been revolutionized through digital media, resulting in innovative computer‐based 3D models to supplement or even replace traditional learning materials. However, the added value of these models in terms of learning performance remains unclear. Multiple mechanisms may contribute to the inconclusive findings. This study focusses on the impact of active manipulation on learning performance and the influence that posttest design features may have on the outcome measurement. Methods: Participants were randomly assigned to one of two research conditions: studying on the base of a computer‐based manipulable pelvic bone model versus online static images of the same model. Pretests focused on students' baseline anatomy knowledge and spatial ability. Three knowledge posttests were administered: a test based on a physical pelvic bone model, and two computer‐based tests based on static images and a manipulable model. Mental effort was measured with the Paas mental effort rating scale. Results: In the static images‐based posttest, significantly higher knowledge scores were attained by participants studying in the static images research condition (p = 0.043). No other significant knowledge‐related differences could be observed. In the manipulable model‐based posttest, spatial ability rather than the research condition seemed to have an influential role on the outcome scores (r = 0.18, p = 0.049). Mental effort scores reflected no difference between both research conditions. Conclusion: The research results are counter‐intuitive, especially because no significant differences were found in the physical model‐based posttest in students who studied with the manipulable model. Explaining the results builds on differences in anatomical models requiring less or more active manipulation to process spatial information. The pelvic bone manipulable model, and by extension osteology models, might be insufficiently complex to provide added value compared with static images. Moreover, the posttest modality should be chosen with care since spatial ability rather than anatomy knowledge may be measured. In recent years, digital tools have revolutionized anatomy education. This paper demonstrates that not all anatomical branches may profit from advanced 3D representations. [ABSTRACT FROM AUTHOR]

Introduction: The OSCE is a sociomaterial assemblage—a meshing together of human and material components producing multiple effects. Materials matter because they shape candidate performance, with potentially calamitous career consequences if materials influence performance unjustly. Although the OSCE literature refers to materials, few papers study the sociomateriality of OSCEs. Therefore, we explored OSCE stakeholders' talk about sociomaterial assemblages to better understand their importance for candidate performance. Methods: We conducted 15 focus groups with OSCE candidates (n = 42), examiners (n = 20) and simulated patients (n = 17) after an Australian postgraduate nursing OSCE. Sociomateriality informed our team‐based framework analysis of data. Results: Participants identified a multiplicity of OSCE materials (objects, technologies and spaces) thought to matter for candidate performance. Candidates' unfamiliarity with materials and missing or malfunctioning materials were reported to yield numerous negative impacts (eg cognitive overload, negative affect, time‐wasting), thereby adversely affecting candidate performance. Both examiners and candidates made micro‐adjustments to sociomaterial assemblages during the OSCE in order to make it work (eg candidates saying what they would do rather than doing it). Sometimes, such tinkering extended so far that sociomaterial assemblages were ruptured (eg examiners ignoring rubrics to help pass candidates), potentially influencing OSCE standardisation. Discussion: Our novel empirical study extends previous conceptual work by illustrating wide‐ranging sociomaterial assemblages influencing OSCE candidate performance. Further research is now needed employing sociomaterial approaches to further elucidate sociomaterial entanglements in diverse OSCEs. We encourage OSCE stakeholders to become more attuned to the productive nature of materials within all stages of OSCE design and implementation. The authors demonstrate the extent to which materials really matter in shaping candidate performance in OSCEs [ABSTRACT FROM AUTHOR]

Background: Inequity in assessment can lead to differential attainment. Degree classifications, such as 'Honours', are an assessment outcome used to differentiate students after graduation. However, there are no standardised criteria used to determine what constitutes these awards. Methods: We contacted all medical schools in the UK and collected data relating to classifications awarded, criteria used and percentage of students receiving classifications across the 5‐year period prior to the 2019/2020 academic year. Results: All 42 UK medical schools responded, and 36 universities provided usable data. Of these 36 universities, 30 (83%) awarded classifications above a 'Pass'. We identified four classifications above a 'Pass', and these were 'Commendation', 'Merit', 'Distinction' and 'Honours'. Sixteen (44%) universities awarded a single additional classification, and 14 (39%) universities awarded two or more. There was considerable variation in the criteria used by each university to award classifications. For example, 30 (67%) out of 45 classifications were dependent on all examined years, 9 (20%) for a combination of years and 6 (13%) for final year alone. Twenty‐five of 30 universities that awarded classifications provided data on the percentage of students awarded a classification, and a median of 15% of students received any type of classification from their university (range 5.3% to 38%). There was a wide range in the percentage of students awarded each classification type across the universities (e.g. Honours, range = 3.1%–24%). Conclusions: We demonstrate considerable variation in the way UK medical degree classifications are awarded—regarding terminology, criteria and percentage of students awarded classifications. We highlight that classifications are another form of inequity in medical education. There is a need to fully evaluate the value of hierarchical degree awards internationally as the consequential validity of these awards is understudied. Degree classifications may be another form of inequity in medical education given variation in terminology, criteria, and percentage of students awarded. Byrne et al. argue a need to better understand the consequential validity of these awards. [ABSTRACT FROM AUTHOR]

Introduction: Many countries are driving forward policies and practices to train medical students for later rural practice. Previous research has investigated individual (e.g., rural upbringing) and structural factors (e.g., curricular exposure) associated with rural practice intention. However, the relationship between academic performance in medical school and rural practice intention has been neglected, although optimisation theory suggests there may be a relationship. To address this gap, our aim was to identify the relationship between academic performance and rural practice intention. Methods: Data were collected via a cross‐sectional (self‐report) survey in 2021. Participants were students from 60 of the 96 rural order directed (RODs) medical programmes across China. We asked students their rural practice intention. We conducted univariate analyses to test for associations between rural practice intention and independent variables, including socio‐demographics, ROD location, grade year and academic performance measures. We used multilevel logistic regression models to test whether students' academic performance in medical school could be used to predict rural practice intention, holding the other factors constant. Results: There were 13 123 respondents, representing roughly 77.6% of the student population from the 60 schools. There was a statistically significant relationship between student (self)‐reported academic performance in medical school and rural practice intention. Higher performers had a lower likelihood (ORs: 0.65–0.78) of rural practice intention. This held across all performance measures (GPA rank, academic awards and student leadership) and for the sub‐group with rural upbringing (ORs: 0.68–0.78). Discussion: This is the first study to identify a relationship between medical school performance and rural practice intention. The findings suggest that students maximise their utility when choosing career options, with higher performers having lower rural practice intention. These data provide insight into the complexity of medical career decision making and can be used by medical school and workforce planners to inform rural training, recruitment and retention strategies. You et al. report that medical students with higher perceived academic performance have lower rural practice intention, illustrating an important factor in students' consideration of career options. [ABSTRACT FROM AUTHOR]

Background: Although used widely, there is limited evidence of the BioMedical Admissions Test's (BMAT) predictive validity and incremental validity over prior educational attainment (PEA). We investigated BMAT's predictive and incremental validity for performance in two undergraduate medical schools, Imperial College School of Medicine (ICSM), UK, and Lee Kong Chian School of Medicine (LKCMedicine), Singapore. Our secondary goal was to compare the evidence collected with published evidence relating to comparable tools. Methods: This was a retrospective cohort study of four ICSM (1188 students, entering 2010–2013) and three LKCMedicine cohorts (222 students, 2013–2015). We investigated associations between BMAT Section 1 ('Thinking Skills'), Section 2 ('Scientific Knowledge and Applications') and Section 3a ('Writing Task') scores, with written and clinical assessment performance across all programme years. Incremental validity was investigated over PEA (A‐levels) in a subset of ICSM students. Results: When BMAT sections were investigated independently, Section 2 scores predicted performance on all written assessments in both institutions with mainly small effect sizes (standardised coefficient ranges: ICSM: 0.08–0.19; LKCMedicine: 0.22–0.36). Section 1 scores predicted Years 5 and 6 written assessment performance at ICSM (0.09–0.14) but nothing at LKCMedicine. Section 3a scores only predicted Year 5 clinical assessment performance at ICSM with a coefficient <0.1. There were no positive associations with standardised coefficients >0.1 between BMAT performance and clinical assessment performance. Multivariable regressions confirmed that Section 2 scores were the most predictive. We found no clear evidence of incremental validity for any BMAT section scores over A‐level grades. Discussion: Schools who wish to assess scientific knowledge independently of A‐levels may find BMAT Section 2 useful. Comparison with previous studies indicates that, overall, BMAT seems less useful than comparable tools. Larger scale studies are needed. Broader questions regarding why institutions adopt certain admissions tests, including those with little evidence, need consideration. The authors use exploration of the BMAT to invite readers to reflect on why institutions adopt certain admissions tests even in the face of little utility evidence. [ABSTRACT FROM AUTHOR]

Introduction: Situational judgement tests (SJTs) have been widely adopted, internationally, into medical selection. It was hoped that such assessments could identify candidates likely to exhibit future professional behaviours. Understanding how performance on such tests may predict the risk of disciplinary action during medical school would provide evidence for the validity of such SJTs within student selection. It would also inform the implementation of such tests within student recruitment. Methods: This cohort study used data for 6910 medical students from 36 UK medical schools who sat the University Clinical Aptitude Test (UCAT) SJT in 2013. The relationship between SJT scores at application and the risk of subsequent disciplinary action during their studies was modelled. The incremental ability of the SJT scores to predict the risk of disciplinary action, above that already provided by UCAT cognitive test scores and secondary (high) school achievement, was also evaluated in 5535 of the students with information available on this latter metric. Results: Two hundred and ten (3.05%) of the students in the cohort experienced disciplinary action. The risk of disciplinary action reduced with increasing performance on the admissions SJT (odds ratio (OR) 0.80, 95% confidence interval (CI) 0.69 to 0.92, p = 0.002). This effect remained similar after adjusting for cognitive performance and prior academic attainment (OR 0.77, 95% CI 0.65 to 0.92, p = 0.004). The overall estimated effect‐size was small (Cohen's d = 0.08) and no evidence of 'threshold' effects were observed for the SJT scores and risk of disciplinary action. Conclusions: Performance on admissions SJTs can, at least modestly, incrementally predict the risk of subsequent disciplinary action, supporting their use in this context. However, for this SJT and outcome, there did not seem a distinct threshold score above which the risk of disciplinary action disproportionately increased. This should be considered when using the scores within medical selection. Performance on an admissions Situational Judgment Test is reported to predict risk of formal disciplinary action in medical students with implications for how such tests should be used to improve medical professionalism. [ABSTRACT FROM AUTHOR]

Objectives: The prevalence of peer‐assisted learning (PAL) featuring alongside the core medical curriculum is increasing; however, the evidence base for PAL's efficacy on academic performance is limited. This systematic review of randomised studies of PAL in medical school sets out to assess the impact of PAL on academic outcomes in medical school and evaluate whether PAL confers a benefit in specific educational contexts. Methods: A literature search was conducted across MEDLINE, Ovid Embase, Web of Science and Education Research Complete. Titles and abstracts were screened, and records were selected following strict eligibility criteria. Following full‐text assessment for eligibility, two reviewers independently extracted data from the final selection of records and a meta‐analysis was performed. Studies were classified using a modified version of Kirkpatrick's levels of learning. Student test scores were standardised by calculating the standardised mean difference (SMD). Results: Twenty‐seven randomised controlled trials were eligible for inclusion, and twenty‐one provided sufficient and complete data to enable meta‐analysis. There was a significant improvement in the academic performance of medical students who received PAL compared with those in the control group (SMD = 0.52 [95% confidence interval 0.18–0.85]; p =.003). The impact of PAL was greater amongst clinical medical students (SMD = 0.63; p =.02) than preclinical medical students (SMD = 0.39; p =.08) and when used for teaching practical skills (SMD = 0.69; p =.001) compared with theory (SMD = −0.11; p =.21). Students taught by PAL also achieved better results in assessments conducted more than four weeks after course completion (SMD = 1.20; p =.04). [Correction added on 8 November 2020 after first publication. The data values in the Results section have been corrected in this version.] Conclusions: Medical students experiencing PAL benefit in terms of academic performance, relative to those not receiving PAL. PAL is of greatest value in the clinical stages of training and for practical skills. The long‐term outcomes of PAL remain a priority for future research. Using a systematic review and meta‐analysis, the authors demonstrate that peer‐assisted learning's benefit to academic performance are sustained in the medium‐to‐long term. [ABSTRACT FROM AUTHOR]

Context Ethnic minority students find that their ethnicity negatively affects the evaluation of their capacities and their feelings in medical school. This study tests whether ethnic minority and majority students differ in their 'self‐regulatory focus' in clinical training, that is, their ways to approach goals, due to differences in social learning experiences. Self‐regulatory focus consists of a promotion and prevention focus. People who are prone to stereotypes and unfair treatments are more likely to have a prevention focus and conceal certain identity aspects. The objectives of the study are to test whether ethnic minority students, as compared with ethnic majority students, are equally likely to have a promotion focus, but more likely to have a prevention focus in clinical training due to more negative social learning experiences (Hypothesis 1), and whether the relationship between student ethnicity and clinical evaluations can be explained by students' gender, social learning experiences, self‐regulatory focus and impression management (Hypothesis 2). Methods: Survey and clinical evaluation data of 312 (71.2% female) clerks were collected and grouped into 215 ethnic majority (69.4%) and 95 ethnic minority students (30.6%). Students' social learning experiences were measured as perceptions of unfair treatment, trust in supervisors and social academic fit. Self‐regulatory focus (general and work specific) and impression management were also measured. A parallel mediation model (Hypothesis 1) and hierarchical multiple regression analyses were used (Hypothesis 2). Results: Ethnic minority students had higher perceptions of unfair treatment and lower trust in their supervisors in clinical training. They were more prevention focused in clinical training, but this was not mediated by having more negative social learning experiences. Lower clinical evaluations for ethnic minority students were unexplained. Promotion focus in clinical training and trust in supervisors positively relate to clinical grades. Conclusion: Student ethnicity predicts social learning experiences, self‐regulatory focus and grades in clinical training. The hidden curriculum plausibly plays a role here. van Andel et al. demonstrate that systematic differences between ethnic minority and majority students can be predicted based on social learning experiences, self‐regulatory focus and clinical grades. [ABSTRACT FROM AUTHOR]

Processes involved in the regulation of learning have been researched for decades, because of its impact on academic and workplace performance. In fact, self‐regulated learning is the focus of countless studies in health professions education and higher education in general. While we will always need competent individuals who are able to regulate their own learning, developments in healthcare require a shift from a focus on the individual to the collective: collaboration within and between healthcare teams is at the heart of high‐quality patient care. Concepts of collaborative learning and collective competence challenge commonly held conceptualisations of regulatory learning and call for a focus on the social embeddedness of regulatory learning and processes regulating the learning of the collective. Therefore, this article questions the alignment of current conceptualisations of regulation of learning with demands for collaboration in current healthcare. We explore different conceptualisations of regulation of learning (self‐, co‐, and socially shared regulation of learning), and elaborate on how the integration of these conceptualisations adds to our understanding of regulatory learning in healthcare settings. Building on these insights, we furthermore suggest ways forward for research and educational practice. Bransen et al. explore the concept of 'self' to argue that the health professions need to shift from striving to optimize self‐regulation to understanding how to effectively regulate across self‐, co‐, and shared‐regulation. [ABSTRACT FROM AUTHOR]

Introduction: Assessment of healthcare professionals should be authentic to clinical practice. As clinicians regularly use resources in practice, similar resources should be available to those sitting assessment. There is limited information on the impacts of open‐book (resource) assessments on standard setting for use in high‐stakes assessments. This research aims to explore the effects on standard setting and student perceptions when open vs closed resources are available in high‐stakes assessment of medical students. Methods: Students sat multiple‐choice question (MCQ) examinations under both closed‐ and open‐resource conditions in a randomised crossover design. A standard setting panel set pass‐marks for both closed‐ and open‐resource conditions of delivery, and we compared these pass‐marks with each other and with actual performance. Students responded to a survey on perceptions of open‐resource assessments. Results: The pass‐mark was set higher when panellists considered open‐resource conditions compared to closed conditions (59% vs 47%), but actual student performance showed no difference in scores between the two conditions. The net effect was that the pass rate was higher for closed than open‐resource conditions (71% vs 34%). Open‐resource conditions increased the time to complete the questions. The students perceived that open resource was more authentic but was more time‐consuming and would require different preparation. Regarding the acceptability of including open resources in high‐stakes assessment, the responses of students were mixed. Discussion: Pass standards based solely on judgements by panellists experienced in closed‐resource conditions might not be applicable under open conditions. Questions vary in how much time accessing resources may take and the degree of assistance in selecting the correct answer. A programme of assessment could be constructed to include both closed‐ and open‐resource condition assessments depending on the questions' content and format. Open‐resource conditions may promote assessment preparation that focuses more on seeking and evaluating resources rather than learning facts. When student use of open‐resources is novel a disconnect emerges as standard setting panelists perceive a need to increase the pass‐mark while the availability of resources actually slows student responding, resulting in no effect on score. [ABSTRACT FROM AUTHOR]