Search

Background The year 2013 marks a watershed in the history of medical education in Taiwan. Following Taiwan’s Taskforce of Medical School Curriculum Reform recommendations, the medical school curriculum was reduced from 7 to 6 years. This study aimed to analyze the impact of medical school curriculum reform on medical students’ performance in objective structured clinical examinations (OSCEs). Methods We retrospectively analyzed the OSCE records at Taipei Veterans General Hospital (Taipei VGH), one of Taiwan’s largest tertiary medical centers, between November 2016 and July 2020. The eligibility criteria were medical students receiving a full one-year clinical sub-internship training at Taipei VGH and in their last year of medical school. All medical students received a mock OSCE-1 at the beginning of their sub-internship, a mock OSCE-2 after six months of training, and a national OSCE at the end of their sub-internship. The parameters for performance in OSCEs included “percentage of scores above the qualification standard” and “percentage of qualified stations.” Results Between November 2016 and July 2020, 361 undergraduates underwent clinical sub-internship training at Taipei VGH. Among them, 218 were taught under the 7-year curriculum, and 143 were instructed under the 6-year curriculum. Based on baseline-adjusted ANCOVA results, medical students under the 7-year curriculum had a higher percentage of scores above the qualification standard than those under the 6-year curriculum at the mock OSCE-1 (7-year curriculum vs. 6-year curriculum: 33.8% [95% CI 32.0–35.7] vs. 28.2% [95% CI 25.9–30.4], p p = 0.001). Moreover, medical students in the 7-year curriculum had a higher percentage of qualified stations in mock OSCE-1 (7-year curriculum vs. 6-year curriculum: 89.4% [95% CI 87.4–91.4] vs. 84.0% [95% CI 81.5–86.4], p = 0.001) and mock OSCE-2 (7-year curriculum vs. 6-year curriculum: 91.9% [95% CI 90.1–93.8] vs. 86.1% [95% CI 83.8–88.3], p = 0.001). After clinical sub-internship training, there were no differences in the percentage of scores above the qualification standard (7-year curriculum vs. 6-year curriculum: 33.5% [95% CI 32.2–34.9] vs. 34.6 [95% CI 32.9–36.3], p = 0.328) and percentage of qualified stations (7-year curriculum vs. 6-year curriculum: 89.4% [95% CI 88.1–90.7] vs. 90.2% [95% CI 88.6–91.8], p = 0.492). Conclusions At the beginning of the sub-internship, medical students under the 7-year curriculum had better OSCE performance than those under the 6-year curriculum. After the clinical sub-internship training in Taipei VGH, there was no difference in the national OSCE score between the 6- and 7-year curricula. Our study suggests that clinical sub-internship is crucial for the development of clinical skills and performance in the national OSCE.

Background Feedback is an essential element of learning. Despite this, students complain about receiving too little feedback in medical examinations, e.g., in an objective structured clinical examination (OSCE). This study aims to implement a written structured feedback tool for use in OSCEs and to analyse the attitudes of students and examiners towards this kind of feedback. Methods The participants were OSCE examiners and third-year medical students. This prospective study was conducted using a multistage design. In the first step, an unstructured interrogation of the examiners formed the basis for developing a feedback tool, which was evaluated and then adopted in the next steps. Results In total, 351 students and 51 examiners participated in this study. A baseline was created for each category of OSCE station and was supplemented with station-specific items. Each of these items was rated on a three-point scale. In addition to the preformulated answer options, each domain had space for individual comments. A total of 87.5% of the students and 91.6% of the examiners agreed or rather agreed that written feedback should continue to be used in upcoming OSCEs. Conclusion The implementation of structured, written feedback in a curricular, summative examination is possible, and examiners and students would like the feedback to be constant.

Background The Covid pandemic and associated lockdown forced medical schools globally not only to deliver emergency remote teaching, but to consider alternative methods of high stakes assessment. Here we outline our approach to the resit virtual OSCE (“VOSCE”) for final year medical students that we undertook during “lockdown” in the current pandemic. Methods The original ‘pre Covid’ examination blueprint was reviewed and modified for the virtual environment in both format and content. In anticipation of the new format delivery, a number of pre-training sessions took place for all parties, and standardised templates were developed. Results A total of 9 students undertook the VOSCE, which took the form of a two-part exam (a communication and clinical examination component, and a practical procedures component). The VOSCE was completed by all students, examiners, simulated patients and invigilators on an online digital platform with no issues with regards to technical problems. Conclusions A total of 6 students passed the VOSCE and as such progressed to graduation. The limitation of assessing some particular types of skills across the remote format (such as practical procedures) was recognised. The training and the templates developed were helpful in case the VOSCE format needs to be adopted in future at short notice and/or expanded in future.

BackgroundMedical school admission procedures have the common goal to select applicants with the greatest potential of becoming successful physicians. Hamburg Medical Faculty selects medical students by grade point average (GPA) and employs a two-step selection process of a natural sciences test (HAM-Nat), in some cases followed by multiple mini-interviews (HAM-Int). Multiple mini-interviews can predict non-cognitive outcomes, while GPA has predictive validity for cognitive outcomes. The aim of our study was to explore communication skills and clinical knowledge of advanced medical students according to their respective admission procedure.MethodsIn July 2019, 146 students grouped according to their admission procedure into GPA-only (19.2 %), HAM-Nat (33.6 %), HAM-Int (30.8 %), and Waiting List (16.4 %) participated in four OSCE stations which equally assessed students’ communication skills (OSCE part 1) and clinical knowledge (OSCE part 2) in simulated patient encounters, rated by physicians with checklists. Additionally, psychosocial assessors ranked communication skills with a global rating scale (GR). The students also participated in a multiple choice (MC) exam testing clinical knowledge. Kruskal-Wallis analyses of variance of test performance and Spearman correlation of instruments were calculated.ResultsStudents from the Waiting List group performed significantly worse on the MC exam compared to GPA-only and HAM-Int (adjustedp = .029 and 0.018, respectively). No significant differences were found between the admission groups with respect to communication skills. Global Rating and OSCE part 1 (communication) correlated significantly (ρ = 0.228,p = .006) as did OSCE part 2 (clinical knowledge) and MC exam (ρ = 0.242,p = .003), indicating criterion validity. Constructs did not overlap, indicating divergent validity.ConclusionsAdvanced medical students selected for undergraduate studies by multiple mini-interviews assessing psychosocial skills showed similar communication skills compared to students admitted to medical school by other entryways. It is unclear whether these similarities are due to an effective undergraduate longitudinal communication curriculum. Assessing baseline communication skills of all medical students at entry-level may aid with this question.

Background: Communication skills are essential in a patient-centred health service and therefore in medical teaching. Although significant differences in communication behaviour of male and female students are known, gender differences in the performance of students are still under-reported. The aim of this study was to analyse gender differences in communication skills of medical students in the context of an OSCE exam (OSCE = Objective Structured Clinical Examination).Methods: In a longitudinal trend study based on seven semester-cohorts, it was analysed if there are gender differences in medical students' communication skills. The students (self-perception) and standardized patients (SP) (external perception) were asked to rate the communication skills using uniform questionnaires. Statistical analysis was performed by using frequency analyses and t-tests in SPSS 21.Results: Across all ratings in the self-and the external perception, there was a significant gender difference in favour of female students performing better in the dimensions of empathy, structure, verbal expression and non-verbal expression. The results of male students deteriorated across all dimensions in the external perception between 2011 and 2014.Discussion & conclusion: It is important to consider if gender-specific teaching should be developed, considering the reported differences between female and male students. peer-reviewed

Background Peer-assisted learning is well established in medical education; however, peer tutors rarely act as assessors for the OSCE. In the compulsory, near-peer teaching programme covering basic medical skills at the University of Heidelberg, peer tutors serve as assessors on a formative OSCE. This study aimed to investigate the feasibility and acceptance of peer assessors and to survey the perceived advantages and disadvantages of their use. Methods In 2016 and 2017 all OSCE peer assessors (third to sixth-year medical students) and all of the peer-assessed students in 2017 (second-year-medical students) were invited to participate in a survey. Both groups were asked to complete a tablet-based questionnaire immediately after the OSCE. Peer assessors were asked to rate eight statements and the peer-assessed students to rate seven statements on a five-point Likert scale. Both were asked to comment on the advantages and disadvantages of peer-assessors. Results Overall, 74 of 76 peer assessors and 307 of 308 peer-assessed students participated in the study. 94% (67/74) of peer assessors and 90% (276/307) of the peer-assessed group thought that it is important to have peer tutors as assessors. Of the peer assessors, 92% (68/74) felt confident in giving structured feedback during the OSCE and 66% (49/74) felt they had improved their teaching skills. Of the peer-assessed students, 99% (306/307) were satisfied with their peers as OSCE assessors and 96% (292/307) considered the peer feedback during the OSCE as helpful. The participants mentioned structural benefits, such as lower costs, and suggested the quality of the OSCE was higher due to the use of peer assessors. The use of peer assessors was found to be beneficial for the learners in the form of high-quality feedback and an overall reduction in stress. Furthermore, the use of peer assessors was found to be beneficial for the peer assessors (improved teaching and clinical skills). Conclusion From a learner’s perspective, the use of peer assessors for a formative OSCE that is part of a near-peer teaching program aimed at junior medical students is favourable for all. A broad implementation of peer assessment in the formative OSCE should be encouraged to investigate effects on quality and stress-reduction.

Background: Objectively structured clinical examinations (OSCEs) are a stressful experience for many health care students and professionals in training. Mock OSCEs have been shown to be beneficial for student OSCE preparation. However, due to their expense and administrative burden students may only get a few opportunities to partake in these. To address this gap in student preparation a series of regularly run totally peer led multi-role practice OSCEs (PrOSCEs) was developed. Methods: Fifteen PrOSCEs were run over five-months. A total of 32 second year medical students took part, all of whom were enrolled on the graduate-entry programme at the University of Southampton. In each PrOSCE, 18 participants rotated through the roles of 'student', 'examiner' and 'patient' in six simulated stations designed by their peers. Peer feedback was provided after each station. At the end of the series of PrOSCEs students were asked to fill in an anonymous online feedback survey to assess the usefulness of the PrOSCEs in exam preparation. Results: Twenty-two students responded to the survey. 100% of respondents deemed routine participation either 'very useful' or 'useful' in preparing for their exam. PrOSCEs were found to improve confidence (mean = 7.9/10, 95% CI 7.4-8.3), expected performance (mean = 7.5/10, 95% CI 6.8-8.2) and help guide revision (mean = 8.3/10, 95% CI 7.6-9.0). Self-perceived teaching performance and confidence in providing feedback was also positively associated with participation. The most beneficial roles were 'student' and 'station creator'. Free-text feedback suggests that the informal setting and regular practice were particularly beneficial. Conclusion: The peer-led nature of the PrOSCEs allows for a low cost, low administrative burden and easy to replicate adjunct or alternative to large scale mock OSCEs. In addition the multi-role aspect of this approach could enhance exam preparation and may also improve aptitude as a clinical teacher. Further studies are required to understand if repeated practice has beneficial implications on OSCE performance.

Introduction Exchange transfusion is the treatment of choice for patients with severe hyperbilirubinemia who do not respond to phototherapy. This procedure is highly complex and requires substantial expertise to perform, however it´s not done frequently enough to guarantee adequate training. Traditional learning scenarios do not have a space reserved for teaching this procedure or an instrument that fully and objectively evaluates the skills that a professional must acquire. Objectives The purpose was to construct and evaluate the INEXTUS instrument´s validity evidence relevant to internal structure, in a simulated scenario through the performance of an objective structured clinical exam (OSCE). Materials and methods The Delphi consensus methodology was utilized to design the instrument; six experts participated through three rounds using the Google Forms platform. The categories and items previously obtained were subjected to validation by nine experts through a dichotomous survey. Prior to data collection, the evaluators were trained through a pilot test with 10 medical students. Subsequently, all residents of a paediatric programme were evaluated through the OSCE methodology in a simulated scenario, with 6 stations, of a clinical case of a new-born with an explicit need for exchange transfusion. During their participation in the scenario, the residents were first evaluated with the instrument developed. Additionally, audio and video filming of all students who participated was performed with the aim of conducting a second evaluation two weeks after the first four evaluators participated. Results The final INEXTUS instrument consists of 46 subitems grouped into 23 items divided into 6 categories, demonstrating an inter-rater intraclass correlation coefficient of 0.96 (95% CI 0.94, 0.98 p-value p value p value Conclusions The INEXTUS instrument evaluates exchange transfusion skills in medical personnel in training in simulated scenarios using the OSCE methodology; it has high validity and reliability and is a high-impact educational tool.

Background Obesity is a major public health problem, yet residents undergo little formal training and assessment in obesity-related care. Given the recent growth of telehealth, physicians must further learn to apply these skills using a virtual platform. Therefore, we aimed to develop an objective structured clinical examination (OSCE) with reliable checklists to assess resident ability to take a patient-centered obesity-focused history that was feasible over telehealth based on published obesity competencies for medical education. Methods We developed a 15-minute telehealth OSCE to simulate an obesity-related encounter for residents modified from a script used to assess medical student obesity competencies. We designed three checklists to assess resident skills in history taking, communication and professionalism during the obesity-related encounter. Resident performance was assessed as the percentage of obesity-related history taking questions asked during the encounter and as the mean communication and professionalism scores on a scale of 1 through 5 with 1 representing unacceptable/offensive behavior and 5 representing excellent skills. Encounters and assessments were completed by two commissioned actors (standardized patients) and 26 internal medicine residents over a secure online platform. We assessed the reliability of each checklist by calculating the percent agreement between standardized patients and the kappa (κ) statistic on each checklist overall and by each checklist item. Results Overall agreement between standardized patients on the history taking, communication and professionalism checklists were 83.2% (κ = 0.63), 99.5% (κ = 0.72) and 97.8% (κ =0.44), respectively. On average, residents asked 64.8% of questions on the history taking checklist and scored 3.8 and 3.9 out of 5 on the communication and professionalism checklists, respectively. Conclusions Results from this pilot study suggest that our telehealth obesity OSCE and checklists are moderately reliable for assessing key obesity competencies among residents on a virtual platform. Integrating obesity OSCEs and other educational interventions into residency curricula are needed to improve resident ability to take an obesity-focused history.

Background The outbreak of coronavirus disease 2019 (COVID-19) and its quick progression to a global pandemic has urged medical schools to shift from didactic to distance learning and assessment approaches. The quality of clinical training and assessment have been jeopardized due to the regulatory restrictions and potential hazards to human lives. The aim of this paper is to evaluate the utility and efficacy of an electronic Objective Structured Clinical Examination (e-OSCE), which attempted to transform the format of a face-to-face OSCE to an e-OSCE. Methods We conducted three end of clerkship e-OSCEs for final year medical students in Surgery, Medicine and Family Medicine using the teleconferencing application of Microsoft Teams (MST). The e-OSCE blueprint included the assessment of all clinical skills except physical examination and procedural skills. Examiners supervised e-OSCE from the college campus, while all students were remotely assessed through the MST channels. During the exam, the students stayed in their specified MST channel and examiners rotated across all students. The utility and efficacy of e-OSCE was evaluated using a self-administered questionnaire for students, examiners and e-OSCE team. Results The data analysis showed that 93.4% students and 84.3% examiners agreed with the quality and process of e-OSCE. Similarly, 83.6% students and 98% examiners agreed with the seamless organization of e-OSCE. As many as 45.9% students and 74.5% examiners agreed that e-OSCE was close to real life practice. Approximately one fifth of students and one third of examiners preferred e-OSCE over the face-to-face OSCE. The analysis of qualitative data generated the themes of e-OSCE structure and technology. While majority of participants were satisfied with e-OSCE, students were concerned about examiners’ training and e-OSCE contents. Examiners and e-OSCE team recognized the paper-less, tech-savy, fast and reliable format of e-OSCE. Conclusion During and beyond COVID- 19 era, e-OSCE is a strong substitute to standard OSCE for assessing clinical competence except for physical examination and procedural skills. The planning and implementation of e-OSCE reflects an ingenuity in the assessment of clinical competencies of medical students.

Background The aim of this cross-sectional survey was to apply the Objective Structured Clinical Examination (OSCE) to evaluate the cardiopulmonary resuscitation (CPR) and endotracheal intubation skills of resident dentists for stage assessment in standardized training. Methods A total of 146 third-year resident dentists were recruited and randomly assigned to perform either CPR or endotracheal intubation. Their performance was scored by experienced anesthesiologists with standardized scoring criteria. Participants were also asked to rated their self-assessed competence, willingness, and perceptions on training status using Likert-type scales in a questionnaire. Student’s ttest was applied to compare scores for CPR and endotracheal intubation performed by resident dentists with different characteristics. The results of the questionnaires were analyzed and visualized by the R package ‘Likert’. Significance was set at the P Results The mean OSCE score for endotracheal intubation (59.1 ± 12.5) was lower than that of CPR (72.4 ± 8.8). Participants with Master’s degrees scored higher than those with Bachelor’s degrees and Doctor of Philosophy (PhD) degrees in the assessment of CPR and endotracheal intubation. Low scores of self-assessed competence and willingness were observed, especially for endotracheal intubation. Resident dentists showed poor satisfaction on training volume and frequency of CPR and endotracheal intubation. Conclusion Resident dentists showed poor performance on CPR and endotracheal intubation assessed by the OSCE. Relatively low self-assessed competence and willingness were reported in endotracheal intubation. The medical emergency curriculum for resident dentists should be more consistent and standardized to help resident dentists enhance the proficiency of life support skills.

Background The usage of smartphones in the daily clinical routine is an essential aspect however it seems that they also present an important distractor that needs to be evaluated. The aim of this prospective study was the evaluation of the influence of phone calls as distractors on the performance levels of medical students during an objective structured clinical examination (OSCE), simulating the normal clinical practice. Methods As the goal of an OSCE presents the examination of clinical skills of medical students in a realistic setting, more than 100 students recruited from the university hospital of Cologne participated in either OSCE I or II. During the OSCE I intravenous cannulation was simulated while OSCE II simulated an acute abdominal pain station. Participants had to perform each of these stations under two circumstances: a normal simulated OSCE and an OSCE station with phone call distraction. Their performance during both simulations was then evaluated. Results In OSCE I students achieved significantly more points in the intravenous cannulation station if they were not distracted by phone calls (M=6.44 vs M=5.95). In OSCE II students achieved significantly more points in the acute abdominal pain station if they were not distracted by phone calls (M=7.59 vs M=6.84). While comparing only those students that completed both stations in OSCE I/II participating students achieved significantly more points in both OSCE I and II if they were not distracted by phone calls. Conclusion The presented data shows that phone call distraction decreases the performance level of medical students during an OSCE station. Therefore, it is an indicator that distraction especially for younger doctors should be held to a minimum. On a second note distraction should be integrated in the medical education system as it plays an important role in clinical routine.

Context The Japan Residency Matching Program (JRMP) launched in 2003 and is now a significant event for graduating medical students and postgraduate residency hospitals. The environment surrounding JRMP changed due to Japanese health policy, resulting in an increase in the number of unsuccessfully-matched students in the JRMP. Beyond policy issues, we suspected there were also common characteristics among the students who do not get a match with residency hospitals. Methods In total 237 out of 321 students at The University of Tokyo Faculty of Medicine graduates from 2018 to 2020 participated in the study. The students answered to the questionnaire and gave written consent for using their personal information including the JRMP placement, scores of the pre-clinical clerkship (CC) Objective Structured Clinical Examinations (OSCE), the Computer-Based Test (CBT), the National Board Examination (NBE), and domestic scores for this study. The collected data were statistically analyzed. Results The JRMP placements were correlated with some of the pre-CC OSCE factors/stations and/or total scores/global scores. Above all, the result of neurological examination station had most significant correlation between the JRMP placements. On the other hand, the CBT result had no correlation with the JRMP results. The CBT results had significant correlation between the NBE results. Conclusions Our data suggest that the pre-clinical clerkship OSCE score and the CBT score, both undertaken before the clinical clerkship, predict important outcomes including the JRMP and the NBE. These results also suggest that the educational resources should be intensively put on those who did not make good scores in the pre-clinical clerkship OSCE and the CBT to avoid the failure in the JRMP and the NBE.

Background Interpersonal and communication skills (ICS) are important core competencies in medical education and certification. In this study, we identified self- and simulated patient (SP)-reported ratings of US first-year medical students’ ICS and the influence of age and gender on performance appraisal during the Objective-Structured Clinical Examination (OSCE). Methods OSCE participants, including 172 first-year medical students and 15 SPs were asked to evaluate the students’ ICS using the American Board of Internal Medicine–Patient-Satisfaction Questionnaire (ABIM–PSQ), electronically and via paper, respectively. Self- and SP-reported ratings of students’ ICS were presented as the median on a 5-point Likert-scale and as three categories defined as “good,” “very good,” and “inadequate.” Results SPs assessed all 172 students in the OSCE, while 43.6% of students assessed their own performance. The majority of students and SPs evaluated the students’ ICS as very good. 23.3% of SPs and 5.3% of students rated the medical students’ ability to encourage patient question-asking and answer questions as inadequate (P Conclusions In the present study, self- and SP-reported ratings of first-year medical students’ ICS were mainly “very good” with no influence of students’ age or gender. Older age and female gender among the SPs were associated with a reduction in SP-reported ratings of students’ ICS.

Introduction This study aimed to explore the decision-making processes of raters during objective structured clinical examinations (OSCEs), in particular to explore the tacit assumptions and beliefs of raters as well as rater idiosyncrasies. Methods Thinking aloud protocol interviews were used to gather data on the thoughts of examiners during their decision-making, while watching trigger OSCE videos and rating candidates. A purposeful recruiting strategy was taken, with a view to interviewing both examiners with many years of experience (greater than six years) and those with less experience examining at final medical examination level. Results Thirty-one interviews were conducted in three centres in three different countries. Three themes were identified during data analysis, entitled ‘OSCEs are inauthentic’, ‘looking for glimpses of truth’ and ‘evolution with experience’. Conclusion Raters perceive that the shortcomings of OSCEs can have unwanted effects on student behaviour. Some examiners, more likely the more experienced group, may deviate from an organisations directions due to perceived shortcomings of the assessment. No method of assessment is without flaw, and it is important to be aware of the limitations and shortcomings of assessment methods on student performance and examiner perception. Further study of assessor and student perception of OSCE performance would be helpful.

BackgroundThe relationship between confidence and competence in clinical skills development is complex but important. This study aims to determine undergraduate paediatric student confidence in performing three common paediatric clinical skills framed as Objective Structured Clinical Examination (OSCE) scenarios and to compare this with subsequent assessed performance. The study also aims to explore possible barriers to successful paediatric skills completion.MethodsA mixed-methods study was conducted on medical students. Cross-sectional questionnaire data relating to confidence in performing a number of paediatric skills were compared with assessed paediatric skills competency. Focus groups were carried out to identify themes in paediatric skills completion to triangulate this data.ResultsEighty-five medical students participated in the study. Students had high levels of pre-test confidence in their ability to perform paediatrics skills. However agreement between pre-test confidence and subsequent task performance was poor and students had significantly greater belief in their skills ability than was subsequently demonstrated. Focus groups identified paediatric skills complexity, conflicting teaching and having limited supervised skills opportunities and as being possible contributory factors to this discrepancy.ConclusionsStudent paediatric skills confidence is not matched by performance. The reasons for this are diverse but mostly modifiable. A major factor is the lack of supervised skills experience with appropriate feedback to support students in learning to calibrate their confidence against their competence. A number of recommendations are made including the introduction of formative assessment opportunities.

Background: As the communication competencies of physicians are crucial for providing optimal patient care, their assessment in the context of the high-stakes Objective Structured Clinical Examination (OSCE) is of paramount importance. Despite abundant literature on the topic, evidence-based recommendations for the assessment of communication competencies in high stakes OSCEs are scarce. As part of a national project to improve communication-competencies assessments in the Swiss licensing exam, we held a symposium with national and international experts to derive corresponding guidelines. Methods: Experts were invited on account of their recognized expertise either in teaching or assessing communication competencies, or in conducting national high-stakes OSCEs. They were asked to propose concrete solutions related to four potential areas for improvement: the station design, the rating tool, the raters’ training, and the role of standardized patients. Data gene.rated in the symposium was available for analysis and consisted of video recordings of plenary sessions, of the written summaries of group work, and the cards with participants’ personal take-home messages. Data were analyzed using a thematic analysis approach. Results: Nine major suggestions for improving communication-competencies assessments emerged from the analysis and were classified into four categories, namely, the roles of the OSCE scenarios, rating tool, raters’ training, and simulated patients. Conclusion: In the absence of established evidence-based guidelines, an experts’ symposium facilitated the identification of nine practical suggestions for improving the assessment of communication competencies in the context of high-stakes OSCEs. Further research is needed to test effectiveness of the suggestions and how they contribute to improvements in the quality of high-stakes communication-competencies assessment. © The Author(s). 2020 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Background The Objective Structured Clinical Examination (OSCE) is now a standard assessment format and while examiner training is seen as essential to assure quality, there appear to be no widely accepted measures of examiner performance. Methods The objective of this study was to determine whether the routine training provided to examiners improved their accuracy and reduced their mental workload. Accuracy was defined as the difference between the rating of each examiner and that of an expert group expressed as the mean error per item. At the same time the mental workload of each examiner was measured using a previously validated secondary task methodology. Results Training was not associated with an improvement in accuracy (p = 0.547) and that there was no detectable effect on mental workload. However, accuracy was improved after exposure to the same scenario (p

Background\ud Personal qualities have been shown to affect students’ exam results. We studied the effect of experience, and level, of public performance in music, drama, dance, sport, and debate at the time of admission to medical school as a predictor of student achievement in their first objective structured clinical examination (OSCE).\ud \ud Methods\ud A single medical school cohort (n = 265) sitting their first clinical exam in 2011 as third year students were studied. Pre-admission statements made at the time of application were coded for their stated achievements in the level of public performance; participation in each activity was scored 0–3, where 0 was no record, 1 = leisure time activity, 2 = activity at school or local level, 3 = activity at district, regional or national level. These scores were correlated to OSCE results by linear regression and t-test. Comparison was made between the highest scoring students in each area, and students scoring zero by t-test.\ud \ud Results\ud There was a bell shaped distribution in public performance score in this cohort. There was no significant linear regression relationship between OSCE results and overall performance score, or between any subgroups. There was a significant difference between students with high scores in theatre, debate and vocal music areas, grouped together as verbal performance, and students scoring zero in these areas. (p

Background The Objective Structured Clinical Examination (OSCE) has been widely applied as a high-stakes examination for assessing physicians’ clinical competency. In 1992, OSCE was first introduced in Taiwan, and the authorities announced that passing the OSCE would be a prerequisite for step-2 medical licensure examination in 2013. This study aimed to investigate the impacts of the announced national OSCE policy on implementation of OSCE at the institutional level. Further, the readiness and the recognition of barriers toward a high-stakes examination were explored. Methods In 2007 and 2010, the year before and after the announcement of high-stakes OSCE policy in 2008, respectively, questionnaires on the status of OSCE implementation were distributed to all hospitals with active OSCE programs in Taiwan. Information on OSCE facilities, equipment, station length, number of administrations per year, and the recognition of barriers to the success of implementing an OSCE were collected. The missing data were completed by telephone interviews. The OSCE format, administration, and facilities before and after the announcement of the nationwide OSCE policy were compared. Results The data were collected from 17 hospitals in 2007 and 21 in 2010. Comparing the OSCE formats between 2007 and 2010, the number of stations increased and the station length decreased. The designated space and the equipment for OSCE were also found to have been improved. As for the awareness of OSCE implementation barriers, the hospital representatives concerned mostly about the availability and quality of standardized patients in 2007, as well as space and facilities in 2010. Conclusions The results of this study underscored an overall increase in the number of OSCE hospitals and changes in facilities and formats. While recruitment and training of standardized patients were the major concerns before the official disclosure of the policy, space and facilities became the focus of attention after the announcement. The study results highlighted the influence of government policy on different aspects of OSCE implementation in Taiwanese training institutes that showed high level of support as reflected in the improved hardware and the change in OSCE format to serve the summative purpose.

Background To prevent the problems of traditional clinical evaluation, the “Objective Structured Clinical Examination (OSCE)” was presented by Harden as a more valid and reliable assessment instrument. However, an essential condition to guarantee a high-quality and effective OSCE is the assurance of evidence to support the validity of its scores. This study examines the psychometric properties of OSCE scores, with an emphasis on consequential and internal structure validity evidence. Methods Fifty-three first year medical students took part in a summative OSCE at the Lebanese American University-School of Medicine. Evidence to support consequential validity was gathered by using criterion-based standard setting methods. Internal structure validity evidence was gathered by examining various psychometric measures both at the station level and across the complete OSCE. Results Compared to our actual method of computing results, the introduction of standard setting resulted in lower students’ average grades and a higher cut score. Across stations, Cronbach’s alpha was moderately low. Conclusion Gathering consequential and internal structure validity evidence by multiple metrics provides support for or against the quality of an OSCE. It is critical that this analysis be performed routinely on local iterations of given tests, and the results used to enhance the quality of assessment. Electronic supplementary material The online version of this article (10.1186/s12909-018-1421-x) contains supplementary material, which is available to authorized users.

Background Traditional Objective Structured Clinical Examinations (OSCEs) are psychometrically sound but have the limitation of fragmenting complex clinical cases into brief stations. We describe a pilot study of a modified OSCE that attempts to balance a typical OSCE format with a semblance of a continuous, complex, patient case. Methods Two OSCE scenarios were developed. Each scenario involved a single standardized patient and was subdivided into three sequential 10 minute sections that assessed separate content areas and competencies. Twenty Canadian PGY-4 internal medicine trainees were assessed by trained examiner pairs during each OSCE scenario. Paired examiners rated participant performance independent of each other, on each section of each scenario using a validated global rating scale. Inter-rater reliabilities and Pearson correlations between ratings of the 3 sections of each scenario were calculated. A generalizability study was conducted. Participant and examiner satisfaction was surveyed. Results There was no main effect of section or scenario. Inter-rater reliability was acceptable. The g-coefficient was 0.68; four scenarios would achieve 0.80. Moderate correlations between sections of a scenario suggest a possible halo effect. The majority of examiners and participants felt that the modified OSCE provided a sense of patient continuity. Conclusions The modified OSCE provides another approach to the assessment of clinical performance. It attempts to balance the advantages of a traditional OSCE with a sense of patient continuity.

Background The General Medical Council states that teaching doctors and students is important for the care of patients. Our aim was to deliver a structured teaching program to final year medical students, evaluate the efficacy of teaching given by junior doctors and review the pertinent literature. Methods We developed a revision package for final year medical students sitting the Objective Structured Clinical Examination (OSCE). The package was created and delivered exclusively by recent medical graduates and consisted of lectures and small group seminars covering the core areas of medicine and surgery, with a focus on specific OSCE station examples. Students were asked to complete a feedback questionnaire during and immediately after the program. Results One hundred and eighteen completed feedback questionnaires were analysed. All participants stated that the content covered was relevant to their revision. 73.2% stated that junior doctors delivered teaching that is comparable to that of consultant - led teaching. 97.9% stated the revision course had a positive influence on their learning. Conclusions Our study showed that recent medical graduates are able to create and deliver a structured, formal revision program and provide a unique perspective to exam preparation that was very well received by our student cohort. The role of junior doctors teaching medical students in a formal structured environment is very valuable and should be encouraged.

Background To reduce inter-rater variability in evaluations and the demand on physician time, standardized patients (SP) are being used as examiners in OSCEs. There is concern that SP have insufficient training to provide valid evaluation of student competence and/or provide feedback on clinical skills. It is also unknown if SP ratings predict student competence in other areas. The objectives of this study were: to examine student attitudes towards SP examiners; to compare SP and physician evaluations of competence; and to compare predictive validity of these scores, using performance on the multiple choice questions examination (MCQE) as the outcome variable. Methods This was a cross-sectional study of third-year medical students undergoing an OSCE during the Internal Medicine clerkship rotation. Fifty-two students rotated through 8 stations (6 physician, 2 SP examiners). Statistical tests used were Pearson's correlation coefficient, two-sample t-test, effect size calculation, and multiple linear regression. Results Most students reported that SP stations were less stressful, that SP were as good as physicians in giving feedback, and that SP were sufficiently trained to judge clinical skills. SP scored students higher than physicians (mean 90.4% +/- 8.9 vs. 82.2% +/- 3.7, d = 1.5, p < 0.001) and there was a weak correlation between the SP and physician scores (coefficient 0.4, p = 0.003). Physician scores were predictive of summative MCQE scores (regression coefficient = 0.88 [0.15, 1.61], P = 0.019) but there was no relationship between SP scores and summative MCQE scores (regression coefficient = -0.23, P = 0.133). Conclusion These results suggest that SP examiners are acceptable to medical students, SP rate students higher than physicians and, unlike physician scores, SP scores are not related to other measures of competence.

Background Little is known about using the Objective Structured Clinical Examination (OSCE) in physical diagnosis courses. The purpose of this study was to describe student performance on an OSCE in a physical diagnosis course. Methods Cross-sectional study at Harvard Medical School, 1997–1999, for 489 second-year students. Results Average total OSCE score was 57% (range 39–75%). Among clinical skills, students scored highest on patient interaction (72%), followed by examination technique (65%), abnormality identification (62%), history-taking (60%), patient presentation (60%), physical examination knowledge (47%), and differential diagnosis (40%) (p < .0001). Among 16 OSCE stations, scores ranged from 70% for arthritis to 29% for calf pain (p < .0001). Teaching sites accounted for larger adjusted differences in station scores, up to 28%, than in skill scores (9%) (p < .0001). Conclusions Students scored higher on interpersonal and technical skills than on interpretive or integrative skills. Station scores identified specific content that needs improved teaching.

Background Continuous development and use of new technologies and methodologies are key features in improving the learning, performance, and skills of medical students and students of all health care professions. Although significant improvements in teaching methodologies have been made in all areas of medicine and health care, studies reveal that students in many areas of health care taking an objective structured clinical examination (OSCE) express difficulties. Thus, this study was planned as a feasibility study to assess the educational effectiveness of an integrated objective structured clinical examination (iOSCE) using both standardized patients and virtual patients. Methods Thirty (30) medical students in their first year of internship at Taipei Medical University volunteered to be part of a feasibility study for demonstrating the concept of iOSCE. They divided themselves into five groups of six students each and were requested to evaluate two cases: 1) a patient with abdominal pain and 2) a patient with headache using a combination of a standardized patient and a virtual patient. For each of the two cases, five stations were designed in which students were given ten minutes per station leading to a final diagnosis and concluded with a debriefing. The five stations were: • Station 1) Interacting with the standardized patient. • Station 2) Writing the patient note and developing a differential diagnosis. • Station 3) Selecting appropriate laboratory and imaging studies. • Station 4) Making a final diagnosis and stating the evidence for it. • Station 5) Having the debriefing. Each group of 6 students was assigned 2 hours per day for each case. All participants completed a survey regarding the usefulness and efficiency of the iOSCE. Results All medical students (30/30; 100%) found the iOSCE program to be very satisfactory, and all expressed that they would like to have further iOSCE experiences if given the opportunity. In terms of ease and helpfulness, the students rated the program an average of 4.4 for the 1st case (abdominal pain) and 4.5 for the 2nd case (headache) on a scale of 1–5, with 5 being the highest and 1 being the lowest score. Conclusions The participants felt that the iOSCE program can offer certain advantages over the traditional OSCE with the SP alone. They cited that the iOSCE provided improved clarity of what was being assessed as well as providing an opportunity to improve their diagnostic reasoning.

Background A Clinical Log was introduced as part of a medical student learning portfolio, aiming to develop a habit of critical reflection while learning was taking place, and provide feedback to students and the institution on learning progress. It was designed as a longitudinal self-directed structured record of student learning events, with reflection on these for personal and professional development, and actions planned or taken for learning. As incentive was needed to encourage student engagement, an innovative Clinical Log station was introduced in the OSCE, an assessment format with established acceptance at the School. This study questions: How does an OSCE Clinical Log station influence Log use by students? Methods The Log station was introduced into the formative, and subsequent summative, OSCEs with careful attention to student and assessor training, marking rubrics and the standard setting procedure. The scoring process sought evidence of educational use of the log, and an ability to present and reflect on key learning issues in a concise and coherent manner. Results Analysis of the first cohort’s Log use over the four-year course (quantified as number of patient visits entered by all students) revealed limited initial use. Usage was stimulated after introduction of the Log station early in third year, with some improvement during the subsequent year-long integrated community-based clerkship. Student reflection, quantified by the mean number of characters in the ‘reflection’ fields per entry, peaked just prior to the final OSCE (mid-Year 4). Following this, very few students continued to enter and reflect on clinical experience using the Log. Conclusion While the current study suggested that we can’t assume students will self-reflect unless such an activity is included in an assessment, ongoing work has focused on building learner and faculty confidence in the value of self-reflection as part of being a competent physician.

BackgroundThe American Association of Medical Colleges has defined peripheral intravenous cannulation as one of the eight practical skills that a medical student should possess upon graduation. Since following a standard hygiene protocol can reduce the rate of complications such as bloodstream infections, the medical student’s compliance to hygienic standards is highly relevant.MethodsThis unicentric longitudinal cohort study included 177 medical students undergoing OSCE 1 in the winter semesters 2016/2017 and 2017/2018 as well as OSCE 2 during the winter semesters 2018/2019 and 2019/2020 at the University of Cologne. Their performance in peripheral intravenous cannulation was rated by trained student supervisors using a scaled 13-item questionnaire and compared between OSCE 1 and OSCE 2.ResultsOverall, a decline in the correct placement of peripheral intravenous catheters was observed among advanced medical students during OSCE 2 (mean total score: 6.27 ± 1.84) in comparison to their results in OSCE 1 (mean total score: 7.67 ± 1.7). During OSCE 2, the students were more negligent in regard to hygienic behavior, such as disinfection of the puncture site as well as hand disinfection before and after venipuncture. Their patients were also less likely to be informed about the procedure as compared to OSCE 1.ConclusionsAn unsatisfying performance in regard to peripheral intravenous cannulation was observed in medical students with hygiene compliance deteriorating between the third and fifth year of their study. Thus, we promote an extension of practical hygiene and stress management training in medical school to reduce complications associated with intravenous catheters, such as bloodstream infections.

Background Ultrasound education is propagated already during medical school due to its diagnostic importance. Courses are usually supervised by experienced faculty staff (FS) with patient bedside examinations or students among each other but often overbooked due to limited FS availability. To overcome this barrier, use of teaching videos may be advantageous. Likewise, peer teaching concepts solely with trained student tutors have shown to be feasible and effective. The aim was to evaluate 1) objective learning outcomes of a combined video-based, student-tutor (ViST) as compared to a FS-led course without media support, 2) acceptance and subjective learning success of the videos. Methods Two ultrasound teaching videos for basic and advanced abdominal ultrasound (AU) and transthoracic echocardiography (TTE) were produced and six students trained as tutors. Fourth-year medical students (N = 96) were randomized to either the ViST- or FS course (6 students per tutor). Learning objectives were defined equally for both courses. Acquired practical basic and advanced ultrasound skills were tested in an objective structured clinical examination (OSCE) using modified validated scoring sheets with a maximum total score of 40 points. Acceptance and subjective learning success of both videos were evaluated by questionnaires based on Kirkpatrick’s evaluation model with scale-rated closed and open questions. Results 79 of 96 medical students completed the OSCE and 77 could be finally analyzed. There was no significant difference in the mean total point score of 31.3 in the ViST (N = 42) and 32.7 in the FS course (N = 35, P = 0.31) or in any of the examined basic or advanced ultrasound skill subtasks. Of the 42 ViST participants, 29 completed the AU and 27 the TTE video questionnaire. Acceptance and subjective learning success of both videos was rated positively in 14–52% and 48–88% of the rated responses to each category, respectively. Attendance of either the student or faculty tutor was deemed necessary in addition to the videos. Conclusions A ViST versus FS teaching concept was able to effectively teach undergraduate students in AU and TTE, albeit acceptance of the teaching videos alone was limited. However, the ViST concept has the potential to increase course availability and FS resource allocation. Supplementary Information The online version contains supplementary material available at 10.1186/s12909-020-02431-8.

Background Excessive stress and anxiety can impair learning. The objective structured clinical examination (OSCE) is a valuable tool to assess and promote the acquisition of clinical skills. However, significant OSCE-related stress and anxiety are frequently reported. The aim of this study was to investigate the relationships between physiological stress, self-reported levels of anxiety due to an OSCE, self-efficacy, and the meanings that physical therapy students attribute to their experience with the exam. Design Concurrent mixed methods study. Methods A total of 32 students took part in this study. All were enrolled in the third semester of a 10-semester Physical Therapy Bachelor Program. Salivary cortisol levels, self-reported anxiety (State-Trait Anxiety Inventory, STAI) were measured before the OSCE. Exam scores and self-efficacy ratings were also recorded. Correlations between variables were tested with the Pearson correlation, with ɑ at 0.05. Semi-structured interviews were used to explore the personal perspectives of students. Thematic analysis was used to investigate emergent themes. Results Trait anxiety scores were significantly higher than normative values (p 49) state anxiety (37.5%) and trait anxiety (65.6%). Salivary cortisol was not associated anxiety (p > 0.05). Neither stress nor anxiety correlated with OSCE scores. A moderate and significant direct correlation was found for self-efficacy scores and OSCE scores (r = 0.475, p = 0.007). Students reported that confidence had a calming effect and led to better self-perceived performance. They also reported that the OSCE can provide meaningful learning experiences despite being stressful. Conclusions A high proportion of our students reported a stable/lingering negative affect. However, neither stress nor anxiety related to OSCE scores. Students’ confidence in their capabilities was correlated with their performance. Their subjective reports suggest that self-confidence may have protected them from the negative effects of stress and anxiety on academic performance.

Background Objective structured clinical examinations (OSCEs) are important aspects of assessment in medical education. There is anecdotal evidence suggesting that students with non-native English accents (NNEA) may be subjected to unconscious bias. It is imperative to minimise the examiners’ bias so that the difference in the scores reflects students’ clinical competence. Research shows NNEAs can cause stereotyping, often leading to the speaker being negatively judged. However, no medical education study has looked at the influence of NNEAs in assessment. Methods This is a randomized, single-blinded controlled trial. Four videos of one mock OSCE station were produced. A professional actor played a medical student. Two near identical scripts were prepared. Two videos showed the actor speaking with an Indian accent and two videos showed the actor speaking without the accent in either script. Forty-two OSCE examiners in the United Kingdom (UK) were recruited and randomly assigned to two groups. They watched two videos online, each with either script, each with a different script. One video with a NNEA and one video was without. Checklist item scores were analysed with descriptive statistics and non-parametric independent samples median test. Global scores were analysed with descriptive statistics and Mann-Whitney test. Results Thirty-two examiners completed the study. The average scores for the checklist items (41.6 points) did not change when the accent variable was changed. Independent samples median test showed no statistically significant relationship between the accent and the scores (p = 0.787). For the global scores received by the videos with the NNEA, there were one less ‘Good’ grade and one more ‘Fail’ grade compared to those without the NNEA. Mann-Whitney test on global score showed lower scores for videos with NNEA (p = 0.661). Conclusions Examiners were not biased either positively or negatively towards NNEAs when providing checklist or global scores. Further study is required to validate the findings of this study. More discussion is warranted to consider how the accent should be considered in current medical education assessment. Registration Trial registration completed trial, ID: ISRCTN17360102, Retrospectively registered on 15/04/2020.

Background Tobacco use is one of the major public health threats globally. Community pharmacists are uniquely positioned to offer tobacco cessation services owing to their easy accessibility by the public. To prepare Qatar community pharmacists to develop the competencies and skills required to offer smoking cessation services, an intensive tobacco control education program was designed and implemented. The study aimed to assess the impact of the tobacco education program on the pharmacists’ skills and competence. Methods A random sample of community pharmacists in Qatar was chosen for participation in the program. Consenting participants were randomly assigned to either intervention or control groups. The intervention group received an intensive education program on treatment of tobacco-use disorder, while a short didactic session on a non-tobacco-related topic was delivered to the control group. The pharmacists’ tobacco cessation skills and competencies were assessed using an Objective Structured Clinical Examination (OSCE). Results A total of 54 and 32 community pharmacists in the intervention group and the control group, respectively, completed the OSCE. The intensive tobacco education group achieved significantly higher total scores than the control group in all the OSCE cases. Specifically, the mean total scores for the intervention group were 15.2, 15.3, 14.2, 14.6, 16.3, and 15.2 compared to 8.8, 6.2, 7.7, 9.2, 8.3, and 11.3 for the control group (p Conclusion The study demonstrated that an intensive tobacco cessation education program can improve pharmacists’ tobacco cessation skills and increase their tobacco cessation counseling abilities. Trial registration Clinical Trials NCT03518476 (https://clinicaltrials.gov/ct2/show/NCT03518476) Registration date: May 8, 2018.

Background Objective Structured Clinical Examinations (OSCEs) are an essential part of the assessment process for medical students. They have traditionally been face-to-face assessments, however, due to the COVID-19 pandemic, medical schools have been forced to attempt to carry them out remotely. OSCEs are difficult to carry out online due to rotation management aspects which make it difficult to synchronize movement of students from one station to another. Methods The authors have developed a dynamic OSCE time management website which aids in solving the movement synchronization issue. This secure website enables participants to view the list of stations they are allowed to enter, and the start and end time of each station. OSCE administrators can control time of entry and monitor progress of the OSCE remotely. Results The authors have used the system to conduct several exams successfully, showing the feasibility and cost effectiveness of this method, as well as user acceptance and satisfaction. In contrast to traditional OSCEs, students are set up in individual virtual rooms for the whole exam while examiners, simulated patients and proctors rotate between them. Conclusions This online OSCE implementation shows feasibility, cost effectiveness and acceptance of this method. The authors found that student outcomes are comparable to traditional OSCEs conducted in the past. There was no significant difference in student marks in one exam compared to last year, while marks were slightly higher in two exams, potentially due to lack of physical exam stations. An unresolved drawback is the inability to assess physical exam stations online, although having students verbally describe what they would do in physical exam situations may be a partial solution.

Background A common feature of performance assessments is the use of human assessors to render judgements on student performance. From a measurement perspective, variability among assessors when assessing students may be viewed as a concern because it negatively impacts score reliability and validity. However, from a contextual perspective, variability among assessors is considered both meaningful and expected. A qualitative examination of assessor cognition when assessing student performance can assist in exploring what components are amenable to improvement through enhanced rater training, and the extent of variability when viewing assessors as contributing their individual expertise. Therefore, the purpose of this study was to explore assessor cognition as a source of score variability in a performance assessment of practice-based competencies. Method A mixed-method sequential explanatory study design was used where findings from the qualitative strand assisted in the interpretation of results from the quantitative strand. Scores from one objective structured clinical examination (OSCE) were obtained for 95 occupational therapy students. Two Generalizability studies were conducted to examine the relative contribution of assessors as a source of score variability and to estimate the reliability of domain and holistic scores. Think-aloud interviews were conducted with eight participants assessing a subset of student performances from the OSCE in which they participated. Findings from the analysis of think-aloud data and consideration of assessors’ background characteristics were used to assist in the interpretation of variance component estimates involving assessors, and score reliability. Results Results from two generalizability analyses indicated the highest-order interaction-error term involving assessors accounted for the second-highest proportion of variance, after student variation. Score reliability was higher in the holistic vs. analytic scoring framework. Verbal analysis of assessors' think-aloud interviews provided evidential support for the quantitative results. Conclusions This study provides insight into the nature and extent of assessor variability during a performance assessment of practice-based competencies. Study findings are interpretable from the measurement and contextual perspectives on assessor cognition. An integrated understanding is important to elucidate the meaning underlying the numerical score because the defensibility of inferences made about students’ proficiencies rely on score quality, which in turn relies on expert judgements.

Background The Faculty of Medical Sciences, University of the West Indies first implemented the Objective Structured Clinical Examination (OSCE) in the final MB Examination in Medicine and Therapeutics during the 2000–2001 academic year. Simultaneously, the Child Health Department initiated faculty and student training, and instituted the OSCE as an assessment instrument during the Child Health (Paediatric) clerkship in year 5. The study set out to explore student acceptance of the OSCE as part of an evaluation of the Child Health clerkship. Methods A self-administered questionnaire was completed by successive groups of students immediately after the OSCE at the end of each clerkship rotation. Main outcome measures were student perception of examination attributes, which included the quality of instructions and organisation, the quality of performance, authenticity and transparency of the process, and usefulness of the OSCE as an assessment instrument compared to other formats. Results There was overwhelming acceptance of the OSCE in Child Health with respect to the comprehensiveness (90%), transparency (87%), fairness (70%) and authenticity of the required tasks (58–78%). However, students felt that it was a strong anxiety-producing experience. And concerns were expressed regarding the ambiguity of some questions and inadequacy of time for expected tasks. Conclusion Student feedback was invaluable in influencing faculty teaching, curriculum direction and appreciation of student opinion. Further psychometric evaluation will strengthen the development of the OSCE.

Background The goal of the Objective Structured Clinical Examination (OSCE) in Competency-based Medical Education (CBME) is to establish a minimal level of competence. The purpose of this study was to 1) to determine the credibility and acceptability of the modified Angoff method of standard setting in the setting of CBME, using the Borderline Group (BG) method and the Borderline Regression (BLR) method as a reference standard; 2) to determine if it is feasible to set different standards for junior and senior residents, and 3) to determine the desired characteristics of the judges applying the modified Angoff method. Methods The results of a previous OSCE study (21 junior residents, 18 senior residents, and six fellows) were used. Three groups of judges performed the modified Angoff method for both junior and senior residents: 1) sports medicine surgeons, 2) non-sports medicine orthopedic surgeons, and 3) sports fellows. Judges defined a borderline resident as a resident performing at a level between competent and a novice at each station. For each checklist item, the judges answered yes or no for “will the borderline/advanced beginner examinee respond correctly to this item?” The pass mark was calculated by averaging the scores. This pass mark was compared to that created using both the BG and the BLR methods. Results A paired t-test showed that all examiner groups expected senior residents to get significantly higher percentage of checklist items correct compared to junior residents (all stations p

Background In addition to vocational education and experience, the pharmaceutical profession’s proper pursuit requires acquiring and continuously improving professional competencies. In recent years, the need has been increasingly highlighted for developing a medical education system based on helping students develop their competencies. It is necessary to adapt the tools and methods for assessing competencies during formal education. It will enable students to know the directions of further personal or professional development. Objective The study aimed to compare pharmacy students’ self-assessment outcomes before and after the Objective Structured Practical Examination (OSPE), which finished the Pharmaceutical Care course (PCc). The study’s purpose was also to compare the outcomes of the self-assessment of competencies between the students of two academic years for whom classes on the PCc were provided by different methods. Methods The study was conducted over two academic years (2018/2019 and 2019/2020) among 5th-year students enrolled at the Faculty of Pharmacy of the Jagiellonian University Medical College (JUMC) at the end of the Pharmaceutical Care course. Different teaching methods were used in the delivery of the course in these academic years. The students self-assessed their competencies using a questionnaire consisting of a list of personal and patient care competencies. The students completed the questionnaire before and after the OSPE, which followed the completion of the PCc. Results Students’ professional competencies as self-assessed after the exam were higher than those assessed before the exam. Differences were observed in both personal and patient care competencies. Students taking the course in the 2019/2020 academic year set their pre-OSPE competencies higher than students taking the PCc in 2018/2019. Conclusion The self-assessment scores increased for most competencies included in the study following the OSPE. This may suggest that taking part in the exam, involvement in patient’s case simulations, and self-assessment of performance at individual stages of the exam contributed to increased subjective assessment of professional competencies.

Background: Particularly at the beginning of their studies, international medical students face a number of language-related, social and intercultural challenges. Thus, they perform poorer than their local counterparts in written and oral examinations as well as in Objective Structured Clinical Examinations (OSCEs) in the fields of internal medicine and surgery. It is still unknown how international students perform in an OSCE in the field of psychosocial medicine compared to their local fellow students. Methods: All students (N = 1033) taking the OSCE in the field of psychosocial medicine and an accompanying written examination in their eighth or ninth semester between 2012 and 2015 were included in the analysis. The OSCE consisted of four different stations, in which students had to perform and manage a patient encounter with simulated patients suffering from 1) post-traumatic stress disorder, 2) schizophrenia, 3) borderline personality disorder and 4) either suicidal tendency or dementia. Students were evaluated by trained lecturers using global checklists assessing specific professional domains, namely building a relationship with the patient, conversational skills, anamnesis, as well as psychopathological findings and decision-making. Results: International medical students scored significantly poorer than their local peers (p

Background There is an ongoing debate about the impact of studying medicine in rural vs. metropolitan campuses on student assessment outcomes. The UNSW Medicine Rural Clinical School has five main campuses; Albury-Wodonga, Coffs Harbour, Griffith, Port Macquarie and Wagga Wagga. Historical data of student assessment outcomes at these campuses raised concerns regarding potential biases in assessment undertaken, as well as the availability and quality of learning resources. The current study aims to identify the extent to which the location of examination (rural versus metropolitan) has an impact on student marks in OSCEs. Methods Assessment data was employed for this study from 275 medical students who sat their final examinations in Years 3 and 6 of the undergraduate Medicine program at UNSW in 2018. The data consists of matched student assessment results from the Year 3 (Y3) MCQ examination and OSCE, and from the Year 6 (Y6) MCQ, OSCE and management viva examinations. The analysis used Univariate Analysis of Variance and linear regression models to identify the impact of site of learning and site of examination on assessment outcomes. Results The results demonstrate that neither site of learning nor site of examination had any significant impact on OSCE or Management Viva assessment outcomes while potential confounders are controlled. Conclusion It is suggested that some of the supposed disadvantages inherent at rural campuses are effectively mitigated by perceived advantages; more intensive interaction with patients, the general and medical communities at those sites, as well as effective e-learning resources and moderation of assessment grades.

Background As good communication skills are crucial for doctor-patient interactions, it is recommended to incorporate them in medical school programs from the very beginning. On this basis medical schools in Germany introduced the OSCE (objective structured clinical examination) to examine and by this foster learning of communication skills as assessment drives learning. The aim of the study was to examine the development of the communication skills of medical students during an OSCE to investigate how communication competence has developed between different student cohorts. Methods This study is a longitudinal trend study based on seven semester-cohorts, examining the communication skills of medical students in the OSCE both from the perspective of students and from the viewpoint of standardized patients (SP). Altogether, 1027 students from seven semester cohorts were asked to rate their own communication skills (self-perception) before the OSCE exam started. Here, sub-analyses were performed to outline a potential influence of previous history-taking group participation. The SP evaluated the students’ communication skills in external perception during the OSCE exam at each station with history-taking or physical examinations. The communication skills in both groups were ascertained in the dimensions of empathy, content structure, verbal expression, and non-verbal expression. Results Only in the dimension of non-verbal expression could a statistically significant change be found in students’ self-perception over the years. Notably, the rating of communication skills as self-rated by the students has risen constantly, whereas they deteriorated from the perspective of standardized patients (SP). It has also been found that previous history-taking courses have a positive influence on the structural dimension of communication skills in particular. Conclusions The results of this study support conclusions of other studies which also suggest differences between self- and external perception of medical students’ communication skills. Nevertheless, students showed good overall communication skills in the four dimensions of empathy, content structure, verbal expression, and non-verbal expression, as demonstrated in a longitudinal trend study over seven semesters. However, we noted that externally rated empathy levels declined over the semester cohorts, suggesting the need for new priorities to be set in student teaching.

Background Moulages can greatly extend the possibilities of simulation in teaching and assessment. Since moulages that fit an educator’s exact needs are often unavailable commercially, this paper explains how 2-dimensional transfer tattoos can be independently developed, produced, and evaluated. Methods From representative photographs of the specific skin condition an analogue copy of the pathological finding is drawn. Once validated by the medical expert, it can be digitized by scanning and processed using graphics software. The final digital image file is printed onto transfer paper. Once applied and fixed onto the intended wearer, usually a simulated patient, its authenticity can be confirmed, and further transfer tattoos can be produced. Results Using this moulage technique we produced 10 different 2-dimensional transfer tattoos to date, including hematoma, Janeway lesions and splinter nails. These moulages are used in clinical skills training, formative and high-stakes summative assessment in undergraduate medical and nursing programs. Conclusions By sharing our development process for 2-dimensional transfer tattoos, health profession educators can produce their own that best fit their local educational needs. Due to their high authenticity and standardization, 2-dimensional transfer tattoos are ideal for use in high-stakes assessment.

Background Particularly at the beginning of their studies, international medical students face a number of language-related, social and intercultural challenges. Thus, they perform poorer than their local counterparts in written and oral examinations as well as in Objective Structured Clinical Examinations (OSCEs) in the fields of internal medicine and surgery. It is still unknown how international students perform in an OSCE in the field of psychosocial medicine compared to their local fellow students. Methods All students (N = 1033) taking the OSCE in the field of psychosocial medicine and an accompanying written examination in their eighth or ninth semester between 2012 and 2015 were included in the analysis. The OSCE consisted of four different stations, in which students had to perform and manage a patient encounter with simulated patients suffering from 1) post-traumatic stress disorder, 2) schizophrenia, 3) borderline personality disorder and 4) either suicidal tendency or dementia. Students were evaluated by trained lecturers using global checklists assessing specific professional domains, namely building a relationship with the patient, conversational skills, anamnesis, as well as psychopathological findings and decision-making. Results International medical students scored significantly poorer than their local peers (p

Background Objective Structured Clinical Examination (OSCE) is a core component of undergraduate medical student assessment. With increased emphasis on integrated programmes, more courses include OSCEs in the traditionally ‘preclinical’ years. The acceptability and impact of such assessment methods at this stage of training is unknown. Methods In 2013 and 2014 University College London Medical School piloted a formative, integrated OSCE in years 1 and 2 of the undergraduate medical degree programme. This study with a sequential exploratory design aimed to explore the acceptability and impact of such an OSCE in the early years of medical school. 1280 students completed the OSCE and were invited to complete a questionnaire exploring their views of the OSCE (response rate 96.6 %). Four focus groups, each with five or six participants (22 in total) were subsequently held to further explore themes. Data was independently transcribed and coded using thematic analysis. Results Students were strongly in favour of the introduction of an OSCE in addition to existing assessments. Six overarching themes were identified: application of knowledge and skills; OSCE as an experience; OSCE as a process; a learning curve; becoming a doctor; and creating an effective OSCE. Conclusions Results strongly support the role of OSCE early in the medical course with many benefits reported. An OSCE at this stage aligns with the vision of integrated medical education which includes early patient contact and introduction of clinical and professional skills. It also fosters feelings of responsibility and professional identity amongst students. Electronic supplementary material The online version of this article (doi:10.1186/s12909-016-0542-3) contains supplementary material, which is available to authorized users.

Background The intraosseous (IO) access is an alternative route for vascular access when peripheral intravascular catheterization cannot be obtained. In Denmark the IO access is reported as infrequently trained and used. The aim of this pilot study was to investigate if medical students can obtain competencies in IO access when taught by a modified Walker and Peyton’s four-step approach. Methods Nineteen students attended a human cadaver course in emergency procedures. A lecture was followed by a workshop. Fifteen students were presented with a case where IO access was indicated and their performance was evaluated by an objective structured clinical examination (OSCE) and rated using a weighted checklist. To evaluate the validity of the checklist, three raters rated performance and Cohen’s kappa was performed to assess inter-rater reliability (IRR). To examine the strength of the overall IRR, Randolph’s free-marginal multi rater kappa was used. Results A maximum score of 15 points was obtained by nine (60%) of the participants and two participants (13%) scored 13 points with all three raters. Only one participant failed more than one item on the checklist. The expert rater rated lower with a mean score of 14.2 versus the non-expert raters with mean 14.6 and 14.3. The overall IRR calculated with Randolph’s free-marginal multi rater kappa was 0.71. Conclusion The essentials of the IO access procedure can be taught to medical students using a modified version of the Walker and Peyton’s four-step approach and the checklist used was found reliable.

Background During their pre-clinical years, medical students are given the opportunity to practice clinical skills with simulated patients. During these formative objective structured clinical encounters (OSCEs), tutors from various backgrounds give feedback on students’ history taking, physical exam, and communication skills. The aim of the study was to evaluate whether the content and process of feedback varied according to the tutors’ profile. Methods During 2013, all 2nd and 3rd year medical students and tutors involved in three formative OSCEs were asked to fill in questionnaires, and their feedback sessions were audiotaped. Tutors were divided into two groups: 1) generalists: primary care, general internist and educationalist physicians 2) specialists involved in the OSCE related to their field of expertise. Outcome measures included the students’ perceptions of feedback quality and utility and objective assessment of feedback quality. Results Participants included 251 medical students and 38 tutors (22 generalists and 16 specialists). Students self-reported that feedback was useful to improve history taking, physical exam and communication skills. Objective assessment showed that feedback content essentially focused on history taking and physical exam skills, and that elaboration on clinical reasoning or communication/professionalism issues was uncommon. Multivariate analyses showed that generalist tutors used more learner-centered feedback skills than specialist tutors (stimulating student’s self-assessment (p

Background Many medical schools state that empathy is important and have curricular learning outcomes covering its teaching. It is thought to be useful in team-working, good bedside manner, patient perspective taking, and improved patient care. Given this, one might expect it to be measured in assessment processes. Despite this, there is relatively little literature exploring how measures of empathy in final clinical examinations in medical school map onto other examination scores. Little is known about simulated patient (actors) rating of empathy in examinations in terms of inter-rater reliability compared with clinical assessors or correlation with overall examination results. Methods Examiners in final year clinical assessments in one UK medical school rated 133 students on five constructs in Objective Structured Long Examination Record (OSLER) with real patients: gathering information, physical examination, problem solving, managing the diagnosis, and relationship with the patient. Scores were based on a standardized well-established penalty point system. In separate Objective Structured Clinical Examination (OSCE) stations, different examiners used the same penalty point system to score performance in both interactional and procedural stations. In the four interaction-based OSCE stations, examiners and simulated patient actors also independently rated empathy of the students. Results The OSLER score, based on penalty points, had a correlation of −0.38 with independent ratings of empathy from the interactional OSCE stations. The intra-class correlation (a measure of inter-rater reliability) between the observing clinical tutor and ratings from simulated patients was 0.645 with very similar means. There was a significant difference between the empathy scores of the 94 students passing the first part of the sequential examination, based on combined OSCE and OSLER scores (which did not include the empathy scores), and 39 students with sufficient penalty points to trigger attendance for the second part (Cohen’s d = 0.81). Conclusions These findings suggest that empathy ratings are related to clinical performance as measured by independent examiners. Simulated patient actors are able to give clinically meaningful assessment scores. This gives preliminary evidence that such empathy ratings could be useful for formative learning, and bolsters the call for more research to test whether they are robust enough to be used summatively.

Background Patient-physician communication should be based on plain and simple language. Despite communication skill trainings in undergraduate medical curricula medical students and physicians are often still not aware of using medical jargon when communicating with patients. The aim of this study was to compare linguistic communication skills of undergraduate medical students who voluntarily translate medical documents into plain language with students who do not participate in this voluntary task. Methods Fifty-nine undergraduate medical students participated in this study. Twenty-nine participants were actively involved in voluntarily translating medical documents for real patients into plain language on the online-platform https://washabich.de (WHI group) and 30 participants were not (non-WHI group). The assessment resembled a virtual consultation hour, where participants were connected via skype to six simulated patients (SPs). The SPs assessed participants’ communication skills. All conversations were transcribed and assessed for communication skills and medical correctness by a blinded expert. All participants completed a self-assessment questionnaire on their communication skills. Results Across all raters, the WHI group was assessed significantly (p = .007) better than the non-WHI group regarding the use of plain language. The blinded expert assessed the WHI group significantly (p = .018) better regarding the use of stylistic devices of communication. The SPs would choose participants from the WHI group significantly (p = .041) more frequently as their personal physician. No significant differences between the two groups were observed with respect to the medical correctness of the consultations. Conclusion Written translation of medical documents is associated with significantly more frequent use of plain language in simulated physician-patient encounters. Similar extracurricular exercises might be a useful tool for medical students to enhance their communication skills with respect to using plain language in physician-patient communication. Electronic supplementary material The online version of this article (doi:10.1186/s12909-016-0594-4) contains supplementary material, which is available to authorized users.

Background Teacher training may improve teaching effectiveness, but it might also have paradoxical effects. Research on expertise development suggests that the integration of new strategies may result in a temporary deterioration of performance until higher levels of competence are reached. In this study, the impact of a clinical teacher training on teaching effectiveness was assessed in an intensive course in emergency medicine. As primary study outcome students’ practical skills at the end of their course were chosen. Methods The authors matched 18 clinical teachers according to clinical experience and teaching experience and then randomly assigned them to a two-day-teacher training, or no training. After 14 days, both groups taught within a 12-hour intensive course in emergency medicine for undergraduate students. The course followed a clearly defined curriculum. After the course students were assessed by structured clinical examination (SCE) and MCQ. The teaching quality was rated by students using a questionnaire. Results Data for 96 students with trained teachers, and 97 students with untrained teachers were included. Students taught by untrained teachers performed better in the SCE domains ‘alarm call’ (p