Search

Purpose: Delivering fair and reliable summative assessments in medical education assumes examiner decision making is devoid of bias. We investigated whether candidate racial appearances influenced examiner ratings in undergraduate clinical exams., Methods: We used an internet-based design. Examiners watched a randomised set of six videos of three different white candidates and three different non-white (Asian, black and Chinese) candidates taking a clinical history at either fail, borderline or pass grades. We compared the median and interquartile range (IQR) of the paired difference between scores for the white and non-white candidates at each performance grade and tested for statistical significance., Results: 160 Examiners participated. At the fail grade, the black and Chinese candidates scored lower than the white candidate, with median paired differences of -2.5 and -1 respectively (both p  < 0.001). At the borderline grade, the black and Chinese candidates scored higher than the white candidate, with median paired differences of +2 and +3, respectively (both p  < 0.001). At the passing grade, the Asian candidate scored lower than the white candidate (median paired difference -1, p  < 0.001)., Conclusion: The racial appearance of candidates appeared to influence the scores awarded by examiners, but not in a uniform manner.

Background: We have previously shown that clinical examiners' scoring is not negatively impacted when a candidate has a tattoo, unnatural hair colour, or a regional accent. We investigated whether these physical attributes in exam candidates impact patient scoring., Methods: Simulated/real patients were randomly assigned to watch five videos of simulated candidate performances of a cranial nerve examination: clear fail, borderline, good, 'clear pass' without an attribute, and 'clear pass' with one of the attributes (tattoo, purple hair, accent). Participants scored domains of communication and professionalism. We compared scores for the clear pass candidates with and without attributes., Results: One hundred and eighty three patients participated. The total scores for the candidates with tattoos and purple hair were higher than the candidate with no physical attribute ( p  < 0.001). For the candidate with a Liverpool English accent no difference was identified ( p  = 0.120)., Conclusions: The presence of certain physical attributes (tattoos or purple hair) was associated with higher scores given by patients to candidates in a simulated physical examination station.

Background: Assessments of physician competence in the work-place are common and often contribute to high-stakes assessments. Previous research suggests that assessors' judgements can be influenced by candidates' physical attributes. We investigated whether simulated candidates' scores were influenced by assessor bias based on tattoos, hair colour, and a regional accent., Methods: We used an experimental, video-based, single-blinded, randomised, internet-based design. We created videos of simulated medical intern performances of a clinical examination at four different standards of competence. Four videos were also created of simulated candidates performing at a 'clear pass' standard, with either no stereotypical attribute (CPX), purple hair (CPH), tattoos (CPT) or a Liverpool English accent (CPA). Assessors were randomly assigned to watch five videos including the "clear pass" candidate without an attribute and one of the "clear pass" candidates with an attribute and asked to give an overall global grade for each candidate. We compared the global grades for the clear pass candidates with and without attributes., Results: Ninety-eight assessors were included in the analysis. The total scores for the candidates with stereotyped attributes were not significantly lower than the candidate with no attribute. Assessors showed moderate levels of agreement between the global grades awarded for all the candidates. The global grades awarded to candidate with a stereotypical attribute were not significantly lower than for those without., Conclusions: The presence of tattoos, purple hair, or a regional accent did not systematically negatively influence the grade or score awarded by assessors to candidates in observed clinical examination scenarios.

Purpose: The forthcoming UK Medical Licensing Assessment will require all medical schools in the UK to ensure that their students pass an appropriately designed Clinical and Professional Skills Assessment (CPSA) prior to graduation and registration with a licence to practice medicine. The requirements for the CPSA will be set by the General Medical Council, but individual medical schools will be responsible for implementing their own assessments. It is therefore important that assessors from different medical schools across the UK agree on what standard of performance constitutes a fail, pass or good grade., Methods: We used an experimental video-based, single-blinded, randomised, internet-based design. We created videos of simulated student performances of a clinical examination at four scripted standards: clear fail (CF), borderline (BD), clear pass (CPX) and good (GD). Assessors from ten regions across the UK were randomly assigned to watch five videos in 12 different combinations and asked to give competence domain scores and an overall global grade for each simulated candidate. The inter-rater agreement as measured by the intraclass correlation coefficient (ICC) based on a two-way random-effects model for absolute agreement was calculated for the total domain scores., Results: 120 assessors enrolled in the study, with 98 eligible for analysis. The ICC was 0.93 (95% CI 0.81-0.99). The mean percentage agreement with the scripted global grade was 74.4% (range 40.8-96.9%)., Conclusions: The inter-rater agreement amongst assessors across the UK when rating simulated candidates performing at scripted levels is excellent. The level of agreement for the overall global performance level for simulated candidates is also high. These findings suggest that assessors from across the UK viewing the same simulated performances show high levels of agreement of the standards expected of students at a 'clear fail,' 'borderline,' 'clear pass' and 'good' level.