Search

Introduction: Whilst rarely researched, the authenticity with which Objective Structured Clinical Exams (OSCEs) simulate practice is arguably critical to making valid judgements about candidates' preparedness to progress in their training. We studied how and why an OSCE gave rise to different experiences of authenticity for different participants under different circumstances. Methods: We used Realist evaluation, collecting data through interviews/focus groups from participants across four UK medical schools who participated in an OSCE which aimed to enhance authenticity. Results: Several features of OSCE stations (realistic, complex, complete cases, sufficient time, autonomy, props, guidelines, limited examiner interaction etc) combined to enable students to project into their future roles, judge and integrate information, consider their actions and act naturally. When this occurred, their performances felt like an authentic representation of their clinical practice. This didn't work all the time: focusing on unavoidable differences with practice, incongruous features, anxiety and preoccupation with examiners' expectations sometimes disrupted immersion, producing inauthenticity. Conclusions: The perception of authenticity in OSCEs appears to originate from an interaction of station design with individual preferences and contextual expectations. Whilst tentatively suggesting ways to promote authenticity, more understanding is needed of candidates' interaction with simulation and scenario immersion in summative assessment. [ABSTRACT FROM AUTHOR]

During the examination of professionals, the simulated patient (SP) can be the right person to assess the professionalism of examinees. This research analysed the efficiency of SP assessments of students' professionalism. Students' professionalism was assessed during an objective structured clinical examination (OSCE) by two independent assessors (the examiner and the SP). Professionalism was assessed using the previously validated Medical Students' Professionalism Assessment Scale. The scores provided by two assessors were subject to descriptive analysis, with mean, standard deviation, median, interquartile range, and minimum and maximum values derived. Since the rating was based on the ordinal scale, the agreement between the two assessors was analysed using quadratic weighted kappa statistics. Both SPs and examiners provided similar mean scores in their assessments. The highest difference was observed in relation to one statement on "student causing pain to the SP", in which the SP assessment mean score was 0.10, while the examiner assessment mean score was 0.48. In terms of agreement, the item "the student gave clear instructions before examining me" achieved moderate concordance (agreement). Although most of the items achieved fair concordance between the assessors, we recommend periodic training of SPs to improve their abilities as assessors for rating the professionalism shown by medical students. [ABSTRACT FROM AUTHOR]

To offset grade inflation, many clerkships combine faculty evaluations with objective assessments including the Medical Examiners Subject Examination (NBME-SE) or Objective Structured Clinical Examination (OSCE), however, standardized methods are not established. Following a curriculum transition removing faculty clinical evaluations from summative grading, final clerkship designations of fail (F), pass (P), and pass-with-distinction (PD) were determined by combined NBME-SE and OSCE performance, with overall PD for the clerkship requiring meeting this threshold in both. At the time, 90% of students achieved PD on the Internal Medicine (IM) OSCE resulting in overall clerkship grades primarily determined by the NBME-SE. The clerkship sought to enhance the OSCE to provide a more thorough objective clinical skills assessment, offset grade inflation, and reduce the NBME-SE primary determination of the final clerkship grade. The single-station 43-point OSCE was enhanced to a three-station 75-point OSCE using the Reporter-Interpreter-Manager-Educator (RIME) framework to align patient encounters with targeted assessments of progressive skills and competencies related to the clerkship rotation. Student performances were evaluated pre- and post-OSCE enhancement. Student surveys provided feedback about the clinical realism of the OSCE and the difficulty. Pre-intervention OSCE scores were more tightly clustered (SD = 5.65%) around a high average performance with scores being highly negatively skewed. Post-intervention OSCE scores were more dispersed (SD = 6.88%) around a lower average with scores being far less skewed resulting in an approximately normal distribution. This lowered the total number of students achieving PD on the OSCE and PD in the clerkship, thus reducing the relative weight of the NMBE-SE in the overall clerkship grade. Student response was positive, indicating the examination was fair and reflective of their clinical experiences. Through structured development, OSCE assessment can provide a realistic and objective measurement of clinical performance as part of the summative evaluation of students. [ABSTRACT FROM AUTHOR]

Aim: Objective structured clinical examinations have long been used in dietetics education. This observational study aims to describe the development, deployment, feasibility and validity of assessment using an oral interview in place of traditional objective structured clinical examinations, and to determine the ability of this assessment to identify students who are either not ready for placement or may require early support and/or remediation. Methods: Student assessment data were collected over a two‐and‐a‐half‐year period and used to test the predictive ability of an oral interview to determine dietetic placement outcomes and highlight a need for early remediation. Descriptive statistics as well as a between‐group one‐way ANOVA was used to describe results. Results: A total of 169 students participated in the oral interview and subsequent medical nutrition therapy placement over the study period. Significant differences in oral interview score were seen between students who passed placement and students who passed with remediation or those who failed. Oral interview performance was able to predict placement outcome, yet required less resources than traditional objective structured clinical examinations. Conclusion: An oral interview may provide the same utility as the objective structured clinical examination in dietetics education. [ABSTRACT FROM AUTHOR]

Background: Medical schools use different evaluation methods after students undergo a period of instruction for certification of acquisition of requisite skills and competencies required for registration. Despite a global trend towards adoption of OSCE format to test competencies, its adoption in medical schools in Nigeria has been relatively slow and local experiences with its use are limited. We describe the development and administration of OSCE for the formative assessment of undergraduate medical students of Kaduna state university at the end of paediatrics clinical rotation. Methodology: OSCE was developed and conducted to assess clinical skills of 20 undergraduate medical students at the end of clinical posting in paediatrics. Students rotated through a series of clinical encounters arranged in ten stations in a circuit, each for a short duration of five minutes. Clinical tasks were carefully chosen to reflect learning objectives of the posting using standardized patients while scoring was done using task-specific and validated checklists by the same assessors. Results: A range of clinical competencies in different clinical scenarios, including history taking, physical examination and basic clinical skills was examined using this format. The conduct was resource intensive with much time spent during the selection of the competencies and appropriate standardized patients to be tested, standardization of checklists, briefing assessors and choosing appropriate venue as well as preparation of stations. Unfamiliarity with OSCE by both some students and assessors was a major challenge. Conclusions: OSCE is a feasible way of assessing a wide range of clinical competencies of medical students during paediatric rotation in our setting.

AbstractIntroductionMethodsResultsDiscussionEnsuring equivalence in high-stakes performance exams is important for patient safety and candidate fairness. We compared inter-school examiner differences within a shared OSCE and resulting impact on students’ pass/fail categorisation.The same 6 station formative OSCE ran asynchronously in 4 medical schools, with 2 parallel circuits/school. We compared examiners’ judgements using Video-based Examiner Score Comparison and Adjustment (VESCA): examiners scored station-specific comparator videos in addition to ‘live’ student performances, enabling 1/controlled score comparisons by a/examiner-cohorts and b/schools and 2/data linkage to adjust for the influence of examiner-cohorts. We calculated score impact and change in pass/fail categorisation by school.On controlled video-based comparisons, inter-school variations in examiners’ scoring (16.3%) were nearly double within-school variations (8.8%). Students’ scores received a median adjustment of 5.26% (IQR 2.87–7.17%). The impact of adjusting for examiner differences on students’ pass/fail categorisation varied by school, with adjustment reducing failure rate from 39.13% to 8.70% (school 2) whilst increasing failure from 0.00% to 21.74% (school 4).Whilst the formative context may partly account for differences, these findings query whether variations may exist between medical schools in examiners’ judgements. This may benefit from systematic appraisal to safeguard equivalence. VESCA provided a viable method for comparisons. [ABSTRACT FROM AUTHOR]

Purpose: The OSCE is regarded as the gold standard of competence assessment in many healthcare programs, however, there are numerous internal and external sources of variation contributing to checklist marks. There is concern amongst organisers that candidates may be unfairly disadvantaged if they follow an "excellent" preceding candidate. This study assessed if average checklist scores differed depending on who a candidate follows accounted for different sources of variation. Design/methodology/approach: This study examined assessment data from final year MBChB OSCEs at the University of Aberdeen and categorised candidates into three levels dependent on examiner awarded global scores of preceding candidates for each station. Data were modelled using a linear mixed model incorporating fixed and random effects. Findings: A total of 349 candidates sat the OSCEs. The predicted mean (95% CI) score for students following an "excellent" candidate was 21.6 (20.6, 22.6), followed "others" was 21.5 (20.5, 22.4) and followed an "unsatisfactory" student was 22.2 (21.1, 23.3). When accounted for individual, examiner and station levels variabilities, students following an "excellent" candidate did not have different mean scores compared to those who followed "other" (p = 0.829) or "unsatisfactory" candidates (p = 0.162), however, students who followed an "unsatisfactory" student scored slightly higher on average compared to those who followed "other" (p = 0.038). Originality/value: There was weak evidence that candidate's checklist variations could be attributed to who they followed, particularly those following unsatisfactory students; the difference in predicted mean scores may be of little practical relevance. Further studies with multiple centres may be warranted assuring perceived fairness of the OSCE to candidates and educators. [ABSTRACT FROM AUTHOR]

Introduction Feedback is an essential component of medical education, enhancing the quality of students' knowledge and skills. However, providing effective feedback, particularly in clinical skills assessments like Objective Structured Clinical Examinations [OSCEs], often poses challenges. This study aimed to evaluate the content of OSCE feedback given to undergraduate medical students over five years. Methods A retrospective analysis of 1034 anonymised medical students' OSCE performance was conducted, focusing on written feedback. The written feedback data were randomly selected from OSCE sessions, collected from university assessment records and anonymised for ethical considerations. R software was used to identify the most frequently repeated words in the examiners’ feedback text, and word cloud charts were created to visualise the responses. Results Word clouds generated from the top 200 most frequently used terms provided visual insights into common descriptive words in feedback comments. The most frequently repeated word over five years was "good," indicative of potentially non-specific feedback. Discussion The high frequency of non-specific terms like "good" suggests a need for more specific, constructive feedback. However, such generic terms can offer some positive reinforcement, more than they may be needed to foster significant improvement. As previously proposed in the literature, adopting structured feedback forms may facilitate the delivery of more specific, actionable feedback. Conclusion This study emphasises the importance of providing specific, actionable feedback in medical education to facilitate meaningful student development. As medical education continues to evolve, refining feedback processes is crucial for effectively guiding students' growth and skill enhancement. Using structured feedback forms can be a beneficial strategy for improving feedback quality.

Introduction: The Objective Structured Clinical Examination (OSCE) is a valid, reliable and reproducible assessment method traditionally carried out as a live examination but recently also provided online. The aim was to compare any differences in the perceptions of dental students participating in online and live OSCE using mixed methods. Materials and Methods: All Finnish fourth‐year undergraduate dental students (n = 172) attended the exam in April 2021. Due to the COVID‐19 pandemic, the official administrative restrictions in teaching in universities still existed in April 2021. By the time of the national OSCE, the pandemic situation varied in different parts of the country. Therefore, two of the universities conducted a live OSCE and two an online version of the OSCE. Data were collected after the OSCE using a voluntary anonymous electronic questionnaire with multiple‐choice and open‐ended questions (response rate 58%). Differences between the OSCE versions were analysed using the Mann–Whitney U test and open answers with qualitative content analysis. Results: The students considered both types of OSCE good in general. The main differences were found concerned adequate time allocation and overall technical implementation, in favour of the live OSCE. While a qualitative analysis revealed exam anxiety as the most often mentioned negative issue, overall, comments were positive. Conclusion: Variation in the assessments between different question entities seemed to be wider than between the implemented OSCE versions. Time management in the OSCE should be further developed by managing the assignment of tasks. [ABSTRACT FROM AUTHOR]

Abstract Purpose Ensuring equivalence of examiners’ judgements within distributed objective structured clinical exams (OSCEs) is key to both fairness and validity but is hampered by lack of cross-over in the performances which different groups of examiners observe. This study develops a novel method called Video-based Examiner Score Comparison and Adjustment (VESCA) using it to compare examiners scoring from different OSCE sites for the first time. Materials/ methods Within a summative 16 station OSCE, volunteer students were videoed on each station and all examiners invited to score station-specific comparator videos in addition to usual student scoring. Linkage provided through the video-scores enabled use of Many Facet Rasch Modelling (MFRM) to compare 1/ examiner-cohort and 2/ site effects on students’ scores. Results Examiner-cohorts varied by 6.9% in the overall score allocated to students of the same ability. Whilst only a tiny difference was apparent between sites, examiner-cohort variability was greater in one site than the other. Adjusting student scores produced a median change in rank position of 6 places (0.48 deciles), however 26.9% of students changed their rank position by at least 1 decile. By contrast, only 1 student’s pass/fail classification was altered by score adjustment. Conclusions Whilst comparatively limited examiner participation rates may limit interpretation of score adjustment in this instance, this study demonstrates the feasibility of using VESCA for quality assurance purposes in large scale distributed OSCEs.

The stark racial health disparities associated with the COVID-19 pandemic coupled with the apparent rise of white supremacy in the United States (U.S.) supports the necessity of anti-racist social work education and practice. Anti-racist practice is particularly salient given the significant numbers of Black, Indigenous, People of Color (BIPOC) served by social workers across the country. This paper highlights ways in which racism continues to permeate the country and the implications for social work practice and education, and how assessment of anti-racist practice has historically and continues to be absent in social work education. After reviewing the coming changes to the Council on Social Work Education's (CSWE) Educational and Policy Accreditation Standards (EPAS) associated with centering anti-racism in social work education, the paper then offers a rationale for utilizing simulation in student assessment of anti-racist practice behaviors. In order to ensure that social work students are meeting the mandate associated with practicing through an anti-racist lens and are ready to effectively collaborate with BIPOC communities, social work students must be adequately assessed while engaged in their educational training. [ABSTRACT FROM AUTHOR]

Background: Objective Structured Clinical Examinations (OSCEs) are widely used in health programs to assess clinical skills. We present results of a qualitative study investigating occupational therapy students' perceptions of OSCEs' impact on their learning and readiness for clinical practice. Method: Six second and six third year students in the University of Alberta's Master of Science in Occupational Therapy program were interviewed in separate focus groups. Independent reviewers applied thematic analysis to the focus group transcripts to identify, analyze, and report themes in the data. Results: Five themes were constructed from the data: from learning to action, transition to practice, stress, representativeness, and suggestions for improvement. Both cohorts perceived OSCEs as intensely stressful but ultimately beneficial to their learning, though third-years more readily identified stress as a catalyst for personal and professional growth. Further, both cohorts noted that OSCEs motivated them to practice clinical skills and constituted important stepping stones toward authentic practice, but the thirdyear students more frequently drew connections between the skills tested in their OSCEs and their confidence in working as occupational therapists. Conclusion: OSCEs play an important role in forming students' identities as clinicians in the making, supporting their continued use for formative assessment in MScOT programs. [ABSTRACT FROM AUTHOR]

The Objective Structured Clinical Examination (OSCE) is a performance-based assessment intended to assess medical students' clinical competency in a simulated, standardized environment. Because it measures the student's ability to use clinical knowledge, diagnostic skill, and decision-making, the OSCE is thought to be more objective than traditional tests. OSCE exams have been increasingly employed in dentistry schools, particularly in the last decade, and it is crucial to investigate instructors' and dental students’ experiences with this evaluation approach.

Introduction: Objective structured clinical examination (OSCE) results could be affected by low homogeneity of examiners, non-retrospectiveness of test results, and examiner-cohort effect. In China, many students participate in medical qualification examinations, and this issue is particularly significant. This study aimed to develop a video recording, video-based rating method and compare the reliability of video and on-site ratings to enhance the quality assurance of OSCEs. Methods: The subjects of this study were clinical students one year after graduation participating in the clinical skills portion of the National Medical Licensing Examination. The participants were from four cities in Jiangsu province. Participants were randomly allocated to on-site and video rating groups to evaluate the rating methods consistency. We verified the reliability of recording equipment and evaluability of video recording. Moreover, we compared the consistency and equivalence of the two rating methods and analyzed the impact of video recording on scores. Results: The reliability of recording equipment and evaluability of video recording were high. Evaluation consistency between experts and examiners was acceptable, and there was no difference in evaluation results (P = 0.61). There was good consistency between video and on-site rating; however, a difference between the two rating methods was detected. The scores of video-based rating group students were lower than those of all students (P < 0.00). Conclusions: Video-based rating could be reliable and offer advantages over on-site rating. The video recording, video-based rating method could provide greater content validity based on its traceability and the ability to view details. Video recording, video-based rating offers a promising mthod for improving the effectiveness and fairness of OSCEs. [ABSTRACT FROM AUTHOR]

Purpose: Ensuring equivalence of examiners' judgements within distributed objective structured clinical exams (OSCEs) is key to both fairness and validity but is hampered by lack of cross-over in the performances which different groups of examiners observe. This study develops a novel method called Video-based Examiner Score Comparison and Adjustment (VESCA) using it to compare examiners scoring from different OSCE sites for the first time. Materials/ methods: Within a summative 16 station OSCE, volunteer students were videoed on each station and all examiners invited to score station-specific comparator videos in addition to usual student scoring. Linkage provided through the video-scores enabled use of Many Facet Rasch Modelling (MFRM) to compare 1/ examiner-cohort and 2/ site effects on students' scores. Results: Examiner-cohorts varied by 6.9% in the overall score allocated to students of the same ability. Whilst only a tiny difference was apparent between sites, examiner-cohort variability was greater in one site than the other. Adjusting student scores produced a median change in rank position of 6 places (0.48 deciles), however 26.9% of students changed their rank position by at least 1 decile. By contrast, only 1 student's pass/fail classification was altered by score adjustment. Conclusions: Whilst comparatively limited examiner participation rates may limit interpretation of score adjustment in this instance, this study demonstrates the feasibility of using VESCA for quality assurance purposes in large scale distributed OSCEs. [ABSTRACT FROM AUTHOR]

The Ottawa Conference on the Assessment of Competence in Medicine and the Healthcare Professions was first convened in 1985 in Ottawa. Since then, what has become known as the Ottawa conference has been held in various locations around the world every 2 years. It has become an important conference for the community of assessment – including researchers, educators, administrators and leaders – to share contemporary knowledge and develop international standards for assessment in medical and health professions education. The Ottawa 2022 conference was held in Lyon, France, in conjunction with the AMEE 2022 conference. A diverse group of international assessment experts were invited to present a symposium at the AMEE conference to summarise key concepts from the Ottawa conference. This paper was developed from that symposium. This paper summarises key themes and issues that emerged from the Ottawa 2022 conference. It highlights the importance of the consensus statements and discusses challenges for assessment such as issues of equity, diversity, and inclusion, shifts in emphasis to systems of assessment, implications of 'big data' and analytics, and challenges to ensure published research and practice are based on contemporary theories and concepts. [ABSTRACT FROM AUTHOR]

The organization of medical knowledge is reflected in language and can be studied from the viewpoints of semantics and prototype theory. The purpose of this study is to analyze student verbalizations during an Objective Structured Clinical Examination (OSCE) and correlate them with test scores and final medical degree (MD) scores. We hypothesize that students whose verbalizations are semantically richer and closer to the disease prototype will show better academic performance. We conducted a single-center study during a year 6 (Y6) high-stakes OSCE where one probing intervention was included at the end of the exam to capture students' reasoning about one of the clinical cases. Verbalizations were transcribed and coded. An assessment panel categorized verbalizations regarding their semantic value (Weak, Good, Strong). Semantic categories and prototypical elements were compared with OSCE, case-based exam and global MD scores. Students with Semantic 'Strong' verbalizations displayed higher OSCE, case-based exam and MD scores, while the use of prototypical elements was associated with higher OSCE and MD scores. Semantic competence and verbalizations matching the disease prototype may identify students with better organization of medical knowledge. This work provides empirical groundwork for future research on language analysis to support assessment decisions. [ABSTRACT FROM AUTHOR]

Abstract Objective Mini clinical evaluation exercise (mini-CEX) and objective structured clinical examination (OSCE) are widely acknowledged as effective measures of resident standardization training (RST) in European and American countries. However, in China primary mini-CEX and OSCE forms are mainly limited in undergraduate clinical examination. Little knowledge is available regarding the validity and right way of mini-CEX /OSCE evaluation system in advanced dental clinical education so far. This study aimed to explore whether combination of mini-CEX and OSCE represents a global-dimension assessment for postgraduate clinical competence in RST. Methods Postgraduates who received RST from June 2017 to June 2019 were selected and evaluated by modified mini-CEX/OSCE scales. Each student received evaluations at least twice in the initial and final stages of training (tested every 4 months). A questionnaire was conducted to investigate the satisfaction with the arrangement of RST. Results Mini-CEX/OSCE test results indicated that postgraduates have significantly improved their comprehensive competence in RST projects in the department of prosthodontics (P

Abstract Background A large portion of prescribing errors can be attributed to deficiencies in medication knowledge. These errors are preventable and most often occur at the time of prescription. Antimicrobials are the drug class most common incorrectly prescribed. Objective To characterize the relationship between clinical competence and antibiotic prescription errors. We also investigated the frequency and severity of antibiotic prescription errors to identify items and attributes of clinical competence which are correlated with the antibiotic prescription error ratio. Method A cross-sectional study was applied to assess clinical competence of junior medical residents in two reference academic hospitals and a regional hospital in Mexico City. It was conducted during February 2019. We used an infectious disease Objective Structured Clinical Examination (OSCE) to assess clinical competence and a measure of frequency, and severity of antibiotic prescription errors. Results The number of eligible participants was ~ 255 (hospital meeting attendance), and the number of residents in this study were 51 (~ 20%), 31 were female (60.8%). The mean OSCE score was 0.692 ± 0.073. The inter-item (Cronbach’s alpha = 0.927) and inter-station internal consistency was adequate (Cronbach’s alpha = 0.774). The G coefficient in generalizability theory analysis was 0.84. The antibiotic prescription error ratio was 45.1% ± 7%. The most frequent category of severity of antibiotic prescription errors was category E (errors that may contribute to or result in temporary harm to the patient and require intervention), 235 (65.2%). We observed a negative and significant correlation between clinical competence and antibiotic prescription errors (r = -0.33, p

Nowadays, Medical education in the Covid-19 pandemic poses a new challenge to explore approaches for delivering quality distance education, especially in the clinical competence assessments. Assessments of clinical competence approaches to conducting an exam, implementation of a new innovative strategy to the assessment procedure, minimize costs, and efficiency in using existing digital resources, have been revised. The study aimed to compare paper and electronic checklists used in Objective Structured Clinical Exams and assessors' self-perception of electronic checklists and technologies. Material and methods: The authors compared three types of checklists (paper, bubble, and electronic checklists) according to the following criteria: ease of use, registration, and authentication of examinees and examiners, processing of exam results, and cost benefits. Results: In the course of analysis, three versions of checklists were compared, thus identifying the advantages and disadvantages of each one, including the assessors' attitude to new technologies. Despite the high cost of the tablets compared to paper checklists, the use of tablets at OSCE had many advantages, and assessors mentioned that number of errors significantly dropped. Conclusion: Thus, digitalization of the assessment procedure helped streamline the exam process, securing evaluation data, and speeding up information processing, which significantly minimized the cost of human resources.

Examiners' judgements play a critical role in competency-based assessments such as objective structured clinical examinations (OSCEs). The standardised nature of OSCEs and their alignment with regulatory accountability assure their wide use as high-stakes assessment in medical education. Research into examiner behaviours has predominantly explored the desirable psychometric characteristics of OSCEs, or investigated examiners' judgements from a cognitive rather than a sociocultural perspective. This study applies cultural historical activity theory (CHAT) to address this gap in exploring examiners' judgements in a high-stakes OSCE. Based on the idea that OSCE examiners' judgements are socially constructed and mediated by their clinical roles, the objective was to explore the sociocultural factors that influenced examiners' judgements of student competence and use the findings to inform examiner training to enhance assessment practice. Seventeen semi-structured interviews were conducted with examiners who assessed medical student competence in progressing to the next stage of training in a large-scale OSCE at one Australian university. The initial thematic analysis provided a basis for applying CHAT iteratively to explore the sociocultural factors and, specifically, the contradictions created by interactions between different elements such as examiners and rules, thus highlighting the factors influencing examiners' judgements. The findings indicated four key factors that influenced examiners' judgements: examiners' contrasting beliefs about the purpose of the OSCE; their varying perceptions of the marking criteria; divergent expectations of student competence; and idiosyncratic judgement practices. These factors were interrelated with the activity systems of the medical school's assessment practices and the examiners' clinical work contexts. Contradictions were identified through the guiding principles of multi-voicedness and historicity. The exploration of the sociocultural factors that may influence the consistency of examiners' judgements was facilitated by applying CHAT as an analytical framework. Reflecting upon these factors at organisational and system levels generated insights for creating fit-for-purpose examiner training to enhance assessment practice. [ABSTRACT FROM AUTHOR]

Simulation-based examination formats improve the possibility to assess medical students' competences during their performance. Additionally, videotaping of simulations allows for remote rating, providing advantages for raters, students, and exam organizers. We describe a simulationbased OSCE prototype with remote rating of students' competences, developed to replace a conventional OSCE at HamburgMedical Faculty. The assessment consists of two phases: a consultation phase with four simulated patient encounters and a case presentation phase where four students present two cases each. All encounters from the consultation and the presentation phase are to be videotaped and remotely rated by clinical raters. Advanced medical students (year 4) are to be assessed regarding their clinical knowledge as well as physician-patientcommunication, clinical reasoning competence, and patient management competence. We provide detailed schedules for the simulationbased OSCE procedure and a roster for organization. When piloting the assessment, we encountered two major obstacles with respect to legal obligations regarding examination time and videotaping which allowed us to provide tips on how to successfully implement this assessment prototype. Remote rating will, when successfully implemented, help students to concentrate on their consultation or presentation tasks, reduce raters' time constraints and also allow for randomized rating. Using established instruments for competence-rating rather than OSCE checklists provides an additional feature for this simulation-based OSCE prototype. Legal issues can be avoided by using the prototype for formative assessment but should be addressed in advance when it is planned to be used as summative assessment. [ABSTRACT FROM AUTHOR]

Objective: Mini clinical evaluation exercise (mini-CEX) and objective structured clinical examination (OSCE) are widely acknowledged as effective measures of resident standardization training (RST) in European and American countries. However, in China primary mini-CEX and OSCE forms are mainly limited in undergraduate clinical examination. Little knowledge is available regarding the validity and right way of mini-CEX /OSCE evaluation system in advanced dental clinical education so far. This study aimed to explore whether combination of mini-CEX and OSCE represents a global-dimension assessment for postgraduate clinical competence in RST. Methods: Postgraduates who received RST from June 2017 to June 2019 were selected and evaluated by modified mini-CEX/OSCE scales. Each student received evaluations at least twice in the initial and final stages of training (tested every 4 months). A questionnaire was conducted to investigate the satisfaction with the arrangement of RST. Results: Mini-CEX/OSCE test results indicated that postgraduates have significantly improved their comprehensive competence in RST projects in the department of prosthodontics (P < 0.05). Compared to other master of Stomatology students, postgraduates taking up prosthodontics master's degree have made more progresses through a training period of up to 1 year and four sessions of face-to-face feedback tutoring (P < 0.05). Survey results revealed high level of satisfaction on clinical practice evaluation. Conclusion: Modified mini-CEX/OSCE combined evaluation system is an effective and reliable assessment tool for clinical comprehensive ability in the RST of professional graduates and can fully highlight their respective advantages on the improvement of students' clinical competency, especially after several rounds of assessments. [ABSTRACT FROM AUTHOR]

Objective Structured Clinical Examination (OSCE) is the standard to assess and train clinical skills in healthcare students. However, conducting an OSCE demands expensive resources such as time, qualified personnel, and adequate facilities. The aim to provide more cost-effective examinations has promoted the emergence of electronic versions of OSCEs (eOSCEs). However, existing eOSCEs available in the market do not adequately adapt to diverse local contexts. For a Spanish-speaking country such as Chile, implementing foreign eOSCEs requires to adjust the examination to unfamiliar terminology, archetypes, and procedures, thus hindering the quality of the assessment. This article reports on the design of Ch-eOSCE, an eOSCE tailored for the Chilean local context and culture. Ch-eOSCE is comprised of a mobile application and a back-end system. A prototype of the application was developed and tested with three healthcare experts to gather their perception of usability and coherence, yielding an overall evaluation of 4.5 in a scale range of 5. The preliminary results reveal that Ch-eOSCE has the potential to become a viable solution for a context-specific eOSCE for the Chilean healthcare context. [ABSTRACT FROM AUTHOR]

Objective structured clinical examinations (OSCEs) are widely used, including in licensure examinations. OSCEs assess diverse skills in a structured manner, but can be stressful for students and have a significant organisational burden for faculty. Case-based video OSCE preparation resources were developed for students preparing for a licensure OSCE. The study aimed to examine student engagement, compare scores awarded by students to performances at specific competence standards with faculty scores, and examine usability, usefulness, and acceptability of the video cases. Final year pharmacy students (n = 149) enrolled in an integrated master of pharmacy programme in Ireland were invited to participate. Six sets of recorded OSCE-based video cases were developed, comprising multiple recordings of the same case scenario, with each pitched at a different level of performance. Students watched and scored the video cases. Usability, usefulness, and acceptability were evaluated via questionnaire. One or more video cases were accessed by 70.5% of students. Score ratings awarded by the students, when compared to faculty ratings, showed an overall trend towards inter-rater agreement between students and faculty. Students felt positively in terms of the usability, usefulness, and acceptability of the videos. Video cases designed to support OSCE preparation for a high-stakes national pharmacy licensure examination were widely used by students and were perceived to be usable, useful, and acceptable. Such video cases may be a feasible alternative to additional mock OSCEs to support student preparation. [ABSTRACT FROM AUTHOR]

Background: Medical schools seek the best curricular designs for the transition to postgraduate education, such as the Dutch elective-based final, 'transitional' year. Most Dutch graduates work a mean of three years as a physician-not-in-training (PNIT) before entering residency training. To ease the transition to selected specialties and to decrease the duration of the PNIT period, UMC Utrecht introduced an optional, thematic variant of the usual transitional year, that enables the development of theme-specific competencies, in addition to physicians' general competencies. Methods: We introduced an optional transitional year for interested students around the theme of acute care, called the Acute Care Transitional Year (ACTY). This study aimed to evaluate the ACTY by judging whether graduates meet postgraduate acute care expectations, indicating enhanced learning and preparation for practice. In a comprehensive assessment of acute care knowledge, clinical reasoning, skills, and performance in simulations, we collected data from ACTY students, non-ACTY students interested in acute care, and PNITs with approximately six months of acute care experience. Results: ACTY graduates outperformed non-ACTY graduates on skills and simulations, and had higher odds of coming up to the expectations faculty have of a PNIT, as determined by global ratings. PNITs did better on simulations than ACTY graduates. Discussion: ACTY graduates show better resemblance to PNITs than non-ACTY graduates, suggesting better preparation for postgraduate acute care challenges. Conclusion: Transitional years, offering multidisciplinary perspectives on a certain theme, can enhance learning and preparedness for entering residency. [ABSTRACT FROM AUTHOR]

John Blamoun,1– 3 Ahmad Hakemi,4 Teresa Armstead3 1Central Michigan University, College of Medicine, Mount Pleasant, MI, 48859, USA; 2MidMichigan Health, Midland, MI, 48670, USA; 3Central Michigan University, College of Health Professions, Physician Assistant Program, Mount Pleasant, MI, 48859, USA; 4Central Michigan University, College of Health Professions, Internal Medicine, Mount Pleasant, MI, 48859, USACorrespondence: Ahmad HakemiCentral Michigan University, College of Health Professions, Internal Medicine, 1280 East Campus Drive, Mailstop #2086, Mount Pleasant, MI, 48859, USATel +1 989-774-1273Email ahmad.hakemi@cmich.eduAbstract: The most important core competencies for medical learners to master are reviewing history, performing physical examination, communication skills and clinical reasoning. The Objective Structured Clinical Examination (OSCE) provides a consistent, reliable, and valid assessment of these integrated skills and is considered to be the gold standard. OSCEs are advantageous because they provide opportunities in evaluating skills that written tests cannot do (stage 3 of Miller’s Pyramid of Learning). In this article, we have provided tips and helpful pointers to medical students and residents, based on available literature and authors’ expertise in managing formative, summative, and virtual OSCE experiences. In virtual OSCEs, in-person learning objectives need to be modified to the virtual milieu and new competencies such as “webside manner” need to be introduced. Harmonizing the process and content of the OSCEs create operational challenges, thus learning the various moving parts of the OSCEs such as psychometrics, tasks of the standardized patients and checklists will ease optimal performance.Keywords: OSCE, checklist, virtual OSCE, assessment, reliability, validity

Examiners' professional judgements of student performance are pivotal to making high-stakes decisions to ensure graduating medical students are competent to practise. Clinicians play a key role in assessment in medical education. They are qualified in their clinical area but may require support to further develop their understanding of assessment practices. However, there are limited studies on providing examiners with structured feedback on their assessment practices for professional development purposes. This study adopts an interpretive paradigm to develop an understanding of clinical examiners' interpretations of receiving structured feedback and its impacts on enhancing their assessment literacy and practice. Data were collected from 29 interviews with clinical examiners who assessed the final-year medical objective structured clinical examinations (OSCEs) at one university. Inductive thematic analysis of these data revealed that the examiners considered the feedback to be useful with practical functions in facilitating communication, comparisons and self-reflection. However, the examiners' level of confidence in the appropriateness of their assessment practices and difficulties in interpreting feedback could be barriers to adopting better practices. Feedback for examiners needs to be practical, targeted, and relevant to support them making accountable and defensible judgements of student performance. [ABSTRACT FROM AUTHOR]

Aim: Assessments of practical clinical competencies pose a challenge during the COVID-19 pandemic. Reports about OSCE stations admin- istered online show that, despite technical feasibility and acceptance, there is a lingering desire for in-person assessments. Barriers and challenges must therefore also be identified in regard to the future in- tegration of digital competencies into the curriculum. Based on a study investigating the feasibility and acceptance of an online OSCE anamnesis station and the descriptions given by students, simulated patients and examiners of the challenges and limitations, we make re- commendations for necessary future adaptations to anamnesis training and testing in the context of telemedicine. Method: We surveyed students after completion of an OSCE anamnesis station, adapted to the telemedical setting, that was administered as an alternative assessment to 149 students via Zoom ®. Using semistructured interviews, we analyzed the resulting challenges and limitations as seen by all of the participants. Results: We confirm the existence of good technical and organizational feasibility, positive learning experiences through feedback, the acquis- ition of clinical competencies, and a high acceptance of this format as an alternative assessment during the pandemic. Using the semi-struc- tured interviews, it was also possible to analyze additional categories that identify necessary adaptations of this type of format. Conclusion: Adaptation of the content-based training for all of the participants and a targeted revision of the checklists, e.g., regarding com- munication techniques in a telemedicine setting, is required due to the effects of the online format on communication and interactions between students and simulated patients. [ABSTRACT FROM AUTHOR]

Graduate entry (GE) pharmacy students are trained in a shorter timeframe than undergraduate entry (UE) students. This study compares the academic performance of GE and UE pharmacy students at the course exit point. A retrospective analysis of final exam grades in written and objective structured clinical examination (OSCE) was performed between GE and UE students from three graduating cohorts. Final written examination contained clinical case study questions, whereas OSCE involved role play with simulated patients or doctors. Statistical analyses were performed by t -test and one-way analysis of variance at.05 significance level and Pearson's correlation coefficient. No significant difference in academic performance was seen between GE and UE groups at course exit (P >.05). There was a trend for GE students performing marginally better in OSCE than UE students. Females showed better performances in verbal communication than males. GE males showed significantly lower empathy scores than all other groups. No significant difference was seen in problem-solving scores amongst all groups. Both UE and GE groups scored significantly better in written examinations compared with OSCE. Graduate entry pharmacy students from accelerated learning pathway and UE students performed similarly at the course exit point, providing empirical support for non-traditional graduate entry pathway as a viable option. [ABSTRACT FROM AUTHOR]

Abstract Background The Covid pandemic and associated lockdown forced medical schools globally not only to deliver emergency remote teaching, but to consider alternative methods of high stakes assessment. Here we outline our approach to the resit virtual OSCE (“VOSCE”) for final year medical students that we undertook during “lockdown” in the current pandemic. Methods The original ‘pre Covid’ examination blueprint was reviewed and modified for the virtual environment in both format and content. In anticipation of the new format delivery, a number of pre-training sessions took place for all parties, and standardised templates were developed. Results A total of 9 students undertook the VOSCE, which took the form of a two-part exam (a communication and clinical examination component, and a practical procedures component). The VOSCE was completed by all students, examiners, simulated patients and invigilators on an online digital platform with no issues with regards to technical problems. Conclusions A total of 6 students passed the VOSCE and as such progressed to graduation. The limitation of assessing some particular types of skills across the remote format (such as practical procedures) was recognised. The training and the templates developed were helpful in case the VOSCE format needs to be adopted in future at short notice and/or expanded in future.

Abstract Background The Objective Structured Clinical Exam (OSCE) is a useful means of generating meaningful feedback. OSCE feedback may be in various forms (written, face to face and audio or video recordings). Studies on OSCE feedback are uncommon, especially involving Asian medical students. Methods We compared two methods of OSCE feedback delivered to fourth year medical students in Malaysia: (i) Face to face (FTF) immediate feedback (semester one) (ii) Individualised enhanced written (EW) feedback containing detailed scores in each domain, examiners’ free text comments and the marking rubric (semester two). Both methods were evaluated by students and staff examiners, and students’ responses were compared against their OSCE performance. Results Of the 116 students who sat for both formative OSCEs, 82.8% (n=96) and 86.2% (n=100) responded to the first and second survey respectively. Most students were comfortable to receive feedback (91.3% in FTF, 96% in EW) with EW feedback associated with higher comfort levels (p=0.022). Distress affected a small number with no differences between either method (13.5% in FTF, 10% in EW, p=0.316). Most students perceived both types of feedback improved their performance (89.6% in FTF, 95% in EW); this perception was significantly stronger for EW feedback (p=0.008). Students who preferred EW feedback had lower OSCE scores compared to those preferring FTF feedback (mean scores ± SD: 43.8 ± 5.3 in EW, 47.2 ± 6.5 in FTF, p=0.049). Students ranked the “marking rubric” to be the most valuable aspect of the EW feedback. Tutors felt both methods of feedback were equally beneficial. Few examiners felt they needed training (21.4% in FTF, 15% in EW) but students perceived this need for tutors’ training differently (53.1% in FTF, 46% in EW) Conclusion Whilst both methods of OSCE feedback were highly valued, students preferred to receive EW feedback and felt it was more beneficial. Learning cultures of Malaysian students may have influenced this view. Information provided in EW feedback should be tailored accordingly to provide meaningful feedback in OSCE exams.

Background: A large portion of prescribing errors can be attributed to deficiencies in medication knowledge. These errors are preventable and most often occur at the time of prescription. Antimicrobials are the drug class most common incorrectly prescribed. Objective: To characterize the relationship between clinical competence and antibiotic prescription errors. We also investigated the frequency and severity of antibiotic prescription errors to identify items and attributes of clinical competence which are correlated with the antibiotic prescription error ratio. Method: A cross-sectional study was applied to assess clinical competence of junior medical residents in two reference academic hospitals and a regional hospital in Mexico City. It was conducted during February 2019. We used an infectious disease Objective Structured Clinical Examination (OSCE) to assess clinical competence and a measure of frequency, and severity of antibiotic prescription errors. Results: The number of eligible participants was ~ 255 (hospital meeting attendance), and the number of residents in this study were 51 (~ 20%), 31 were female (60.8%). The mean OSCE score was 0.692 ± 0.073. The inter-item (Cronbach's alpha = 0.927) and inter-station internal consistency was adequate (Cronbach's alpha = 0.774). The G coefficient in generalizability theory analysis was 0.84. The antibiotic prescription error ratio was 45.1% ± 7%. The most frequent category of severity of antibiotic prescription errors was category E (errors that may contribute to or result in temporary harm to the patient and require intervention), 235 (65.2%). We observed a negative and significant correlation between clinical competence and antibiotic prescription errors (r = -0.33, p < 0.05, CI95% -0.57 to -0.07), which remained significant after controlling for the effect of gender and time since graduation from medical school (r = -0.39, p < 0.01, CI95% -0.625 to -0.118). Using exploratory factor analysis we identified two factors, which explained 69% of the variance in clinical competence, factor 1 evaluated socio-clinical skills and factor 2 evaluated diagnostic-therapeutic skills. Factor 2 was correlated with antibiotic prescription error ratio (r = -0.536, p < 0.001). Conclusions: We observed a negative correlation between clinical competence and antibiotic prescription error ratio in graduated physicians who have been accepted in a medical specialty. The therapeutic plan, which is a component of the clinical competence score, and the prescription skills had a negative correlation with antibiotic prescription errors. The most frequent errors in antibiotic prescriptions would require a second intervention. [ABSTRACT FROM AUTHOR]

Purpose: In the context of health professions education, the objective structured clinical examination (OSCE) has been implemented globally for assessment of clinical competence. Concerns have been raised about the significant influence of construct irrelevant variance arising from examiner variability on the robustness of decisions made in high-stakes OSCEs. An opportunity to explore an initiative to reduce examiner effects was provided by a secondary analysis of data from a large-scale summative OSCE of the final-year students (n > 350) enrolled in a graduate-entry four-year Bachelor of Medicine/Bachelor of Surgery (MBBS) program at one Australian research-intensive university. The aim of this study was to investigate the impact of providing examiners with structured feedback on their stringency and leniency on assessing the final-year students’ clinical competence in the pre-feedback (P1) OSCE and post-feedback (P2) OSCE. Method: This study adopted a quasi-experimental design to analyse the scores from 141 examiners before feedback was provided for the P1 OSCE, and 111 examiners after feedback was provided for the P2 OSCE. This novel approach used generalisability theory to quantify and compare the examiner stringency and leniency variance (Vj) contributing to the examiners’ scores before and after feedback was provided. Statistical analyses conducted were controlled for differences in the examiners and OSCE stations. Results: Comparing the scores of the 51 examiners who assessed students in both P1 and P2 OSCEs, the Vj reduced by 35.65% and its contribution to the overall variation in their scores decreased by 7.43%. The results were more noticeable in the 26 examiners who assessed students in both OSCEs and in at least one station common across both OSCEs. The Vj reduced by 40.56% and its contribution to the overall variation in their scores was also decreased by 7.72%. Conclusion: The findings might be suggested that providing examiners with structured feedback could reduce the examiner stringency and leniency variation contributing to their scores in the subsequent OSCE, whilst noting limitations with the quasi-experimental design. More definitive research is required prior to making recommendations for practice.

The Membership of the Royal College of Obstetricians and Gynaecologists (MRCOG) and the European Fellowship in Obstetrics and Gynaecology (EBCOG) exams are both well-renowned specialty qualifications that assess the competency of obstetricians and gynaecologists. In this article, an exam candidate shares his perspective on the changes made during the COVID-19 pandemic. Despite changing to an online format to allow candidates to take the exam remotely, the MRCOG Part 3 exam maintained its main exam structures: (1) simulated patient task to evaluate the candidates' interactions with well-trained patients in a tele-interview: (2) structure discussion with the clinical examiners based on some certain topics. In contrast, the EBCOG has created a brand new structure to suit the online model to assess the candidates' core clinical skills in broader aspects. Although it is unclear whether online exam will exist in future, this has been a unique experience for candidates during pandemic. [ABSTRACT FROM AUTHOR]

Observed structured clinical examinations (OSCEs) are a common method of assessment within higher education to prepare for the advanced nurse practitioner (ANP) role. This article reviews a wide range of literature relating to OSCE assessment in the healthcare arena, from an ANP, interprofessional and advanced clinical practice perspective. Theories underpinning OSCE and advanced nursing roles are explored, with relevant supporting literature indicating how established OSCEs can become integrated with other methods to improve outcomes for this level of practice. Alternative assessments are explored with specific reference to the context of the education delivered, and the suitability for higher education today. [ABSTRACT FROM AUTHOR]

Context: The Objective Structured Clinical Examination (OSCE) will be used in the final National Ranking Examination (NRE) at the end of year 6 of medical studies in France. Aim: To report the 13-year experience of OSCEs in a French faculty of medicine and to assess the key features of its development and establishment. Methods: The organization and administration of the OSCEs are described. The mean scores and the mean clinical skills scores obtained, their fidelity (alpha coefficient) and the distribution of scores within each cohort (Kurtosis and Skewness coefficients) were analyzed. OSCE and NRE scores were compared for the 2018 and 2019 cohorts. Results: An OSCE (mean 7.4 stations) was administered consecutively between 2008 and 2020 to year groups of approximately 200 students (range: 145–236). The mean duration of the circuit was 68 minutes (range: 48–97). Mean fidelity coefficients ranged from 0.52 [IC5%: 0.41–0.58] to 0.73 [IC5%: 0,67–0,77] for station scores and from 0.65 [IC5%: 0.57–0.70] to 0.82 [IC5%: 0.78–0.85] for clinical skills scores. Mean Kurtosis and Skewness coefficients respectively ranged from 2.36 ± 0.5 to 5.56 ± 1.69 and to −0.10 ± 0.11 to −0.96 ± 0.22. Performance on OSCE was not correlated with NRE (Person and Spearman Coefficients). Conclusion: Implementation of the OSCE within our faculty as a validating second cycle end-of-cycle exam has proven to be sustainable. Despite statistical limits, it allows the classification of students and identification of students with difficulties in clinical practice. The key points are institutional support, designing of stations, relevant training for observers. Clinical skills training must be part of a more global reflection on the role of simulation in undergraduate medical studies in France. [ABSTRACT FROM AUTHOR]

Background & Objectives: The novel coronavirus (SAR-CoV-2) pandemic has revolutionized medical education worldwide. Most medical schools have adopted the online teaching and assessments. Students attending modified clerkships and assessments under the stress of the pandemic, perform and score differently when compared to normal clerkships. We aimed to identify the impact of COVID-19 on final year MBBS students' EOC (End of Clerkship) examination by comparing them with their scores prior to the COVID and with scores of the previous final year. Methods: This cross-sectional study was conducted at Shifa College of Medicine. Final year MBBS students' scores of years 2019 and 2020 were included. Students' EOC MCQ and OSCE scores were compared in pre-COVID and COVID affected rotations of the same year and with the previous year (2019). Data were analyzed in SPSS version 21, means scores were calculated, and one-way ANOVA was applied. Pearson correlation was calculated for correlation assessment of MCQ and OSCE scores. Results: There were 118 students. The mean EOC, OSCE, and MCQ scores in rotations one to four were 72.8±6.4, 73.3±8.1, 71.6±7.4, 72.7±6.7 and 44.4±8, 47.2±8.4, 46.1±8.2, 48.8±8.1, respectively. Oneway ANOVA results before and after COVID lockdown were statistically insignificant (p=0.3) for OSCE and significant for MCQ in the final year class of 2020 (p=0.001). The Pearson correlation assessment between MCQ and OSCE scores (n=416) had a significant positive correlation (r=0.42, p=0.000). The overall comparison between scores of the final year class of 2019 and 2020 showed significant improvement in Surgery and Obstetrics/Gynae scores in 2020. Conclusion: During the COVID pandemic, the final year students' performance in EOC MCQ and OSCE over all remained unaffected. [ABSTRACT FROM AUTHOR]

Aim: The aims were to describe the development of a modified national online OSCE during COVID‐19 and assess related student feedback. Material and methods: The modified online OSCE comprising of eight question entities was organised simultaneously in all four dental institutes of Finland using the Moodle virtual learning environment. All fourth‐year students (n = 179) attended the examination online at home. Student feedback was collected via an anonymous questionnaire with multiple‐choice questions and open‐ended questions concerning attitudes towards the modified online OSCE, as well as content and usability of the question entities in the examination. Means and standard deviations were calculated for multiple‐choice questions. Content analysis was used for open‐ended questions. Results: Of 179 students, 119 (66%) consented to the study. Students experienced they had received adequate information (mean 3.8; SD 1.2), had a positive attitude before the examination (4.0; 1.0) and found the practice test useful (3.7; 1.1) (range 1–5). Technical implementation (2.7; 0.7) and the difficulty of the questions (2.9; 0.6) (range 1–4) were found to be good. The teaching students received during their studies was sufficient (3.2; 0.5) (range 1–4). Content (mean 3.2; 0.4) and usability (2.9; 0.4) of the question entities were good (range 1–4). The themes arising from open‐ended questions were importance and practicality of the topic (in questions) in relation to the work of a dentist and gratitude for the rapid conversion of the OSCE into an online examination despite COVID‐19. The themes arising from negative experiences included difficulties in completing the examination within the time allocated, and dissatisfaction with the model answers provided after the examination. Conclusion: The positive student feedback towards the modified online OSCE encourages including an online examination to complement the traditional OSCE. [ABSTRACT FROM AUTHOR]

In-person Objective Structured Clinical Examinations (OSCEs) are used as assessments in most pharmacy programs, however, reports of virtual OSCEs and skills assessments are limited. With a pivot to virtual OSCEs necessitated by the COVID-19 pandemic, it is important to consider core competencies assessed as well as logistics, technology, and other factors. The virtual redesign of a Capstone OSCE for third professional year pharmacy students at the beginning of the COVID-19 pandemic. Core components of pivoting an in-person Capstone OSCE to virtual included an initial assessment of what was planned, what competencies were being assessed, and how this could be accomplished in the virtual realm. Key stakeholders such as students, testing and evaluation services, and faculty were included in the planning process. In addition to changes in OSCE delivery, grading processes and remediation were also modified. This Capstone OSCE merged virtual education and skills-based assessments together out of necessity and provides an example of flexibility and nimbleness in times of great change. Although further assessment is needed, processes used and lessons learned are helpful for future, intentional planning of virtual or partially virtual skills assessments. [ABSTRACT FROM AUTHOR]

Abstract Background There is an ongoing debate about the impact of studying medicine in rural vs. metropolitan campuses on student assessment outcomes. The UNSW Medicine Rural Clinical School has five main campuses; Albury-Wodonga, Coffs Harbour, Griffith, Port Macquarie and Wagga Wagga. Historical data of student assessment outcomes at these campuses raised concerns regarding potential biases in assessment undertaken, as well as the availability and quality of learning resources. The current study aims to identify the extent to which the location of examination (rural versus metropolitan) has an impact on student marks in OSCEs. Methods Assessment data was employed for this study from 275 medical students who sat their final examinations in Years 3 and 6 of the undergraduate Medicine program at UNSW in 2018. The data consists of matched student assessment results from the Year 3 (Y3) MCQ examination and OSCE, and from the Year 6 (Y6) MCQ, OSCE and management viva examinations. The analysis used Univariate Analysis of Variance and linear regression models to identify the impact of site of learning and site of examination on assessment outcomes. Results The results demonstrate that neither site of learning nor site of examination had any significant impact on OSCE or Management Viva assessment outcomes while potential confounders are controlled. Conclusion It is suggested that some of the supposed disadvantages inherent at rural campuses are effectively mitigated by perceived advantages; more intensive interaction with patients, the general and medical communities at those sites, as well as effective e-learning resources and moderation of assessment grades.

The Royal College of Paediatrics and Child Health (RCPCH) developed a new end-of-training assessment held for the first time in 2012, known as START, the Specialty Trainee Assessment of Readiness for Tenure as a consultant. It is a novel, formative, multi-scenario, OSCE-style, out-of-workplace assessment using unseen scenarios with generic, external assessors undertaken in the trainees’ penultimate training year. This paper describes the introduction and structure of this formative assessment. While many other colleges have summative exit exams the inception of this assessment was designed to be formative, providing feedback on consultant-readiness skills, and not a high-stakes hurdle towards the end of training, It was developed from the College’s examinations question-setting group and following two pilots in 2009 and 2010, the assessment evolved and the first live diet was held in November 2012.

Background: The Covid pandemic and associated lockdown forced medical schools globally not only to deliver emergency remote teaching, but to consider alternative methods of high stakes assessment. Here we outline our approach to the resit virtual OSCE ("VOSCE") for final year medical students that we undertook during "lockdown" in the current pandemic. Methods: The original 'pre Covid' examination blueprint was reviewed and modified for the virtual environment in both format and content. In anticipation of the new format delivery, a number of pre-training sessions took place for all parties, and standardised templates were developed. Results: A total of 9 students undertook the VOSCE, which took the form of a two-part exam (a communication and clinical examination component, and a practical procedures component). The VOSCE was completed by all students, examiners, simulated patients and invigilators on an online digital platform with no issues with regards to technical problems. Conclusions: A total of 6 students passed the VOSCE and as such progressed to graduation. The limitation of assessing some particular types of skills across the remote format (such as practical procedures) was recognised. The training and the templates developed were helpful in case the VOSCE format needs to be adopted in future at short notice and/or expanded in future. [ABSTRACT FROM AUTHOR]

Many institutions require candidates to achieve a minimum number of OSCE stations passed (MNSP) in addition to the aggregate pass mark. The stated rationale is usually that this conjunctive standard prevents excessive degrees of compensation across an assessment. However, there is a lack of consideration and discussion of this practice in the medical education literature. We consider the motivations for the adoption of the MNSP from the assessment designer perspective, outlining potential concerns about the complexity of what the OSCE is trying to achieve, particularly around the blueprinting process and the limitations of scoring instruments. We also introduce four potential methods for setting an examinee-centred MNSP standard, and highlight briefly the theoretical advantages and disadvantages of these approaches. There are psychometric arguments for and against the limiting of compensation in OSCEs, but it is clear that many stakeholders value the application of an MNSP standard. This paper adds to the limited literature on this important topic and notes that current MNSP practices are often problematic in high stakes settings. More empirical work is needed to develop understanding of the impact on pass/fail decision-making of the proposed standard setting methods developed in this paper. [ABSTRACT FROM AUTHOR]

Background: Objective Structured Clinical Examination (OSCE) is one of the final assessment components for nursing students. OSCE assessment uses the ON-OSCE (Online Nursing OSCE) application to fulfill the need and overcome the manual OSCE assessment challenges using papers. The manual OSCE assessment with multiple checklists is very detailed and takes a long time in scoring. Besides, the evaluation using papers tends to cause a miscalculation in scoring a total of ratings that can harm the student. Method: This research is a non-experimental study with a descriptive design and cross-sectional approach. The samples were 480 respondents of nursing students. The instrument of assessment of satisfaction was analyzed based on the score accuracy, time to retrieve score, and examiner's attention. Data analysis used descriptive frequency and percentage distributions. Results: Students' satisfaction based on the time to retrieve score revealed that 365 students (76%) were satisfied. Two hundred eighty-five students (59,4 %) were also satisfied with ON-OSCE related to the score accuracy. Regarding the examiners' attention, 273 students (56,9 %) felt the examiners ignored the students' actions. They focused on the laptop to provide an assessment. It could be due to the new ON-OSCE application for the examiners and their unfamiliarity with operating the application. Conclusion: Most students were satisfied with the assessment using the ON-OSCE application to retrieve scores and the score accuracy. However, they were less satisfied with the examiners' attention. The examiners should be more familiar with the ON-OSCE.