Variance due to the examination conditions and factors associated with success in objective structured clinical examinations (OSCEs): first experiences at Paris-Saclay medical school

Abstract Background We aimed to measure the variance due to examination conditions during the first sessions of objective structured clinical examinations (OSCEs) performed at a French medical school and identify factors associated with student success. Methods We conducted a retrospective, observational study using data from the first three OSCEs sessions performed at Paris-Saclay medical school in 2021 and 2022. For all sessions (each organized in 5 parallel circuits), we tested a circuit effect using a linear mixed-effects model adjusted for sex and the average academic level of students (according to written tests). Then, we studied the factors associated with student success at one station using a multivariate linear mixed-effects model, including the characteristics of students, assessors, and standardized patients. Results The study included three OSCEs sessions, with 122, 175, and 197 students and a mean (± SD) session score of 13.7(± 1.5)/20, 12.7(± 1.7)/20 and 12.7(± 1.9)/20, respectively. The percentage of variance due to the circuit was 6.5%, 18.2% (statistically significant), and 3.8%, respectively. For all sessions, the student’s average level and station scenario were significantly associated with the score obtained in a station. Still, specific characteristics of assessors or standardized patients were only associated with the student’s score in April 2021 (first session). Conclusion The percentage of the variance of students’ performance due to the examination conditions was significant in one out of three of the first OSCE sessions performed at Paris-Saclay medical school. This result seems more related to individual behaviors rather than specific characteristics of assessors or standardized patients, highlighting the need to continue training teaching teams. National clinical trial number Not applicable.