Search

Purpose The COVID-19 pandemic has necessitated a quick shift to virtual speech-language services; however, only a small percentage of speech-language pathologists (SLPs) had previously engaged in telepractice. The purpose of this clinical tutorial is (a) to describe how the Early Language and Literacy Acquisition in Children with Hearing Loss study, a longitudinal study involving speech-language assessment with children with and without hearing loss, transitioned from in-person to virtual assessment and (b) to provide tips for optimizing virtual assessment procedures. Method We provide an overview of our decision making during the transition to virtual assessment. Additionally, we report on a pilot study that calculated test-retest reliability from in-person to virtual assessment for a subset of our preschool-age participants. Results Our pilot study revealed that most speech-language measures had high or adequate test-retest reliability when administered in a virtual environment. When low reliability occurred, generally the measures were timed. Conclusions Speech-language assessment can be conducted successfully in a virtual environment for preschool children with hearing loss. We provide suggestions for clinicians to consider when preparing for virtual assessment sessions. Supplemental Material https://doi.org/10.23641/asha.14787834.

Purpose: To assess educational and professional outcomes of an accelerated combined bachelor of science-doctor of medicine (BS-MD) program using data collected from 1968 through 2018., Method: Participants of this longitudinal study included 2,235 students who entered medical school between 1968 and 2014: 1,134 in the accelerated program and 1,101 in the regular curriculum (control group)-matched by year of entrance to medical school, gender, and Medical College Admission Test (MCAT) scores. Outcome measures included performance on medical licensing examinations, academic progress, satisfaction with medical school, educational debt, first-year residency program directors' ratings on clinical competence, specialty choice, board certification, and faculty appointments., Results: The authors found no practically important differences between students in the accelerated program and those in the control group on licensing examination performance, academic progress, specialty choice, board certification, and faculty appointments. Accelerated students had lower mean educational debt (P < .01, effect sizes = 0.81 and 0.45 for, respectively, their baccalaureate debt and medical school debt), lower satisfaction with their second year of medical school (P < .01, effect size = 0.21), and lower global satisfaction with their medical school education (P < .01, effect size = 0.35). Residency program directors' ratings in 6 postgraduate competency areas showed no practically important differences between the students in the accelerated program and those in the control group. The proportion of Asian students was higher among program participants (P < .01, effect size = 0.43)., Conclusions: Students in the accelerated program earned BS and MD degrees at a faster pace and pursued careers that were comparable to students in a matched control who were in a regular MD program. Findings indicate that shortening the length of medical education does not compromise educational and professional outcomes., (Copyright © 2020 by the Association of American Medical Colleges.)

This Invited Commentary is an independent opinion piece and companion to the Perspective by Carmody and Rajasekaran that appears in this issue of Academic Medicine. The National Board of Medical Examiners (NBME), a 501(c)(3) nonprofit, is a powerful gatekeeper to the medical profession in the United States. According to publicly available tax data, the NBME, which has increased its number of income-enhancing products, had revenues of $153.9 million (M) and net assets of $177.6M in 2017, earnings (revenue less expenses) of $39.7M in 2013-2017, and a highly compensated management team. Medical students are ultimately the source of nearly all the NBME's revenue, and the NBME has contributed to the growth of medical student debt. The NBME has operated as a monopoly since its agreement in the early 1990s with the Federation of State Medical Boards to cosponsor the United States Medical Licensing Examination (USMLE). Although the NBME has developed valuable products and is ostensibly governed by a capable board, the NBME has inherent financial conflicts of interest and may be benefiting from the current "Step 1 mania" undermining undergraduate medical education. Here, the author makes 4 recommendations to reestablish the trust of the U.S. medical education community in the NBME: (1) the NBME should recuse itself from current discussions and policy-making decisions related to changes in the score reporting of the USMLE Step 1 exam; (2) the NBME should disclose and be transparent about all aspects of its finances; (3) new NBME products, changes in pricing, and changes to pass thresholds should be approved by an oversight committee, independent of the NBME; and (4) the NBME (and USMLE) should not charge students or residents for retaking any of its licensing examinations.

Though intended to inform a binary decision on initial medical licensure, the United States Medical Licensing Examination (USMLE) is frequently used for screening candidates for residency positions. Some have argued that reporting results as pass/fail would honor the test's purpose while preventing inappropriate use. To date, the USMLE's sponsor organizations have declined to make such a change. In this Perspective, the authors examine the history and mission of the National Board of Medical Examiners (NBME), trace the rise of "Step 1 mania," and consider the current financial incentives for the NBME in implementing a pass/fail score-reporting policy.The NBME was founded in 1915 to address the lack of interstate reciprocity in medical licensure examination. With the creation of the USMLE in 1992, a single pathway for licensure was established, and the organization's original mission was achieved. Yet even after fulfilling its primary purpose, the NBME-classified as a nonprofit organization-has seen its revenues rise dramatically over the past 2 decades. Much of the increased revenue is derived from test products and services not required for medical licensure, with sales driven by the increasing importance of Step 1 scores in residency selection. Revenue from these products and services would likely decline if the NBME reported Step 1 results as pass/fail.A financial conflict of interest occurs when a judgment concerning a primary interest may be influenced by a secondary interest, such as financial gain. The data presented here demonstrate that the NBME has a conflict of interest in its current score-reporting policy. Possible remedies, such as disclosure, recusal, divestiture, and restructuring, are considered.

Introduction: The Objective Structured Clinical Examination (OSCE) is used globally for formative and summative purposes. The objective of this study was to examine the impact of tablet-scoring on sources of validity evidence for an Internal Medicine residency OSCE. Methods: We compared paper-scored OSCEs from 2014 to tablet-scored OSCEs in 2015 for missing data, amount of comments, and time to pass/fail decision. We then examined in 2016 the impact on learning of showing residents their score sheets and asking them to write an action plan immediately after an OSCE. Results: Tablet-scoring significantly reduced stations with missing data from 1.8 to 0.2%, and stations without comments from 42 to 28% with an increase in word count per comment. Time to official results reduced from 3 weeks to 12 h with tablet-scoring. Residents who wrote a learning plan after reviewing their OSCE score sheets were more likely (with medium to large effect sizes) to pursue further studying and/or change their behavior (e.g. history taking or physical examination) in the clinical environment. Conclusions: OSCE tablet-scoring improved many sources of validity evidence, especially educational impact with timeliness of feedback supporting a change in behavior, a hard to achieve goal of educational interventions.

Introduction: Knowledge of anatomy is essential for surgeons. We sought to determine whether it is possible to effectively assess and differentiate the anatomic knowledge of general surgery residents, using pieces of fabric and yarn., Methods: Postgraduate years 2, 3, and 4 general surgery residents were assessed during a simulation-based assessment known as the Surgical X-Games. Residents were allowed 3 minutes to assemble the anatomic structures of the right upper quadrant region and 2 minutes to create the right lower quadrant of the abdomen, using colored felt and yarn. One point (each) was given for naming and placing the structures in the correct position. A checklist was used to assess trainees with a maximum combined score of 150 points., Results: A total of 34 residents (postgraduate year 2 = 16, postgraduate year 3 = 8, postgraduate year 4 = 10) participated in the 2017 fall Surgical X-Games and 31 residents (postgraduate year 2 = 13, postgraduate year 3 = 9, postgraduate year 4 = 9) participated in the spring Surgical X-Games. Total scores increased respective to the level of clinical training in both the fall Surgical X-Games (postgraduate year 2 = 77, postgraduate year 3 = 84, postgraduate year 4 = 93, P = .04) and the spring Surgical X-Games (postgraduate year 2 = 94, postgraduate year 3 = 101, postgraduate year 4 = 109). We observed significant improvement in the right upper quadrant, right lower quadrant, and total scores from the fall to the spring postgraduate year (P < .001)., Conclusion: Surgical residents showed an increase in mean anatomical scores from postgraduate years 2 to postgraduate years 3 to postgraduate years 4, using low-fidelity models. This inexpensive, 5-minute test based on a simple checklist may offer surgical educators insight in to residents' anatomic knowledge and potential readiness for clinical rotations., (Copyright © 2019 Elsevier Inc. All rights reserved.)

Background: The objective structured clinical examination (OSCE) has gained widespread use as a form of performance assessment. However, opportunities for students to participate in practice OSCEs are limited by the financial, faculty and administrative investments required., Objectives: To determine the feasibility and acceptability of a student-run mock OSCE (MOSCE) as a learning experience for medical students of all 4 years., Design: We conducted a five-station MOSCE for third-year students. This involved fourth-year students as examiners and first-/second-year students as standardized patients (SPs). Each examiner scored examinees using a checklist and global rating scale while providing written and verbal feedback. MOSCE stations and checklists were designed by students and reviewed by a faculty supervisor. Following the MOSCE, participants completed surveys which elucidated their perceptions on the roles they took during the MOSCE., Results: Fifty examinees participated in the MOSCE. Of these, 42 (84%) consented to participate in the study and submitted completed questionnaires. Twenty-four examiners participated in the OSCE and consented to participate in the study, with 22 (92%) submitting completed questionnaires. Fifty-three of 60 SPs (88%) agreed to take part in this study, and 51 (85%) completed questionnaires. The internal consistency of the five-station OSCE was calculated as a Cronbach's alpha of 0.443. Students commented positively on having the opportunity to network and engage in mentorship activities and reinforce clinical concepts., Conclusions: Examinees, examiners, and SPs all perceived the MOSCE to be a beneficial learning experience. We found the MOSCE to be a feasible and acceptable means of providing additional OSCE practice to students prior to higher-stakes evaluations.

Competing Interests: Competing interests: None declared.

The pedagogical approach for both didactic and laboratory teaching of anatomy has changed in the last 25 years and continues to evolve; however, assessment of student anatomical knowledge has not changed despite the awareness of Bloom's taxonomy. For economic reasons most schools rely on multiple choice questions (MCQ) that test knowledge mastered while competences such as critical thinking and skill development are not typically assessed. In contrast, open-ended question (OEQ) examinations demand knowledge construction and a higher order of thinking, but more time is required from the faculty to score the constructed responses. This study compares performances on MCQ and OEQ examinations administered to a small group of incoming first year medical students in a preparatory (enrichment) anatomy course that covered the thorax and abdomen. In the thorax module, the OEQ examination score was lower than the MCQ examination score; however, in the abdomen module, the OEQ examination score improved compared to the thorax OEQ score. Many students attributed their improved performance to a change from simple memorization (superficial learning) for cued responses to conceptual understanding (deeper learning) for constructed responses. The results support the view that assessment with OEQs, which requires in depth knowledge, would result in student better performance in the examination. Anat Sci Educ 11: 254-261. © 2017 American Association of Anatomists., (© 2017 American Association of Anatomists.)

Neuroanatomical localization (NL) is a key skill in neurology, but learners often have difficulty with it. This study aims to evaluate a concise NL tool (NLT) developed to help teach and learn NL. To evaluate the NLT, an extended-matching questions (EMQ) test to assess NL was designed and validated. The EMQ was validated with fourth-year medical students and internal medicine and neurology residents. The NLT's usability was evaluated with third- and fourth-year students, and the effectiveness was evaluated with an experimental study of second-year students, using the EMQ as the outcome measure. Students were taught how to use both the NLT and textbook algorithms (control) to perform NL, then randomized into either group, and only allowed to use their assigned tool to complete the EMQ. Primary outcome was the difference in mean EMQ scores expressed as a percentage of total score. For EMQ validation, students (n = 56) scored lower than residents (n = 50) (76.7% ± 1.7 vs. 83.0% ± 1.6; mean ± standard error of mean, P < 0.009). The EMQ demonstrated good reliability (Cronbach's α 0.85) and generalizability (G-coefficient 0.85). Third- (n = 77) and fourth-year (n = 42) students found the NLT user-friendly and helpful in their learning of NL. In the experimental study, scores were significantly higher for NLT group (n = 94) than for controls (n = 101) (42.5 vs. 37.0%, P = 0.014); the effect size (Cohen's d) was 0.36. The EMQ is validated to reliably assess NL and is generalizable, feasible, practical, and of low cost. The concise and user-friendly NLT for NL was effective in aiding medical student performance of NL. Anat Sci Educ 11: 262-269. © 2017 American Association of Anatomists., (© 2017 American Association of Anatomists.)

Background: Fifty percent of graduating U.S. Navy post-graduate year (PGY)-1 physicians will practice in the operational environment before returning to residency training. However, current internship structure is less rotational and focuses more on specialty-specific training. Therefore, these physicians may not be fully prepared for this primary care role., Methods: Based on the U.S. Navy privileges for General Medical Officers, a comprehensive didactic and simulation curriculum was developed. Twenty-three procedural skill competencies (SK) and five validated standardized patient (SP) scenarios were identified. During the SK portion, learners reviewed instructional videos, read reference materials, and practiced with partial task trainers before small-group sessions with subject matter experts (SME). Separate SP round-robin sessions were conducted and feedback provided by SMEs and SPs. Learners demonstrated competency or were remediated., Results: One hundred and three PGY-1 trainees participated over 2 yr. All trainees met requirements during the SK phase. During the SP phase, seven learners required remediation. All learners ultimately met requirements for privileging., Conclusion: The Simulation Training for Operational Medicine Providers curriculum for future General Medical Officers is an effective tool for primary care skill training and credentialing. Plans for export to other Graduate Medical Education sites are underway and further evaluation of skills retention is warranted.

Objective. To examine the extent of financial and faculty resources dedicated to preparing students for NAPLEX and PCOA examinations, and how these investments compare with NAPLEX pass rates. Methods. A 23-item survey was administered to assessment professionals in U.S. colleges and schools of pharmacy (C/SOPs). Institutions were compared by type, age, and student cohort size. Institutional differences were explored according to the costs and types of NAPLEX and PCOA preparation provided, if any, and mean NAPLEX pass rates. Results. Of 134 C/SOPs that received the survey invitation, 91 responded. Nearly 80% of these respondents reported providing some form of NAPLEX preparation. Significantly higher 2015 mean NAPLEX pass rates were found in public institutions, schools that do not provide NAPLEX prep, and schools spending less than $10,000 annually on NAPLEX prep. Only 18 schools reported providing PCOA preparation. Conclusion. Investment in NAPLEX and PCOA preparation resources vary widely across C/SOPs but may increase in the next few years, due to dropping NAPLEX pass rates and depending upon how PCOA data are used.

The purpose of this study was to examine the associations among racial discrimination, generational status, and perceptions of the economic value of education among Latina/o youth. Participants were 400 urban, low-income, Latina/o students from a large Midwestern U.S. city who completed surveys in both 9th and 10th grades. Results revealed that more perceived racial discrimination was associated with more perceived economic limitations of education. When analyzed by generational status, more racial discrimination in 9th grade was significantly related to lower perceived economic value of education in 10th grade for third-generation and later participants, but not for first- or second-generation participants. The results provide evidence for the diverse experiences of racial discrimination and perceived economic value of education across generational groups., (© 2017 The Authors. Journal of Research on Adolescence © 2017 Society for Research on Adolescence.)

Background: In ordinary circumstances, objective structured clinical examination (OSCE) is a resource-intensive assessment method. In case of developing and implementing multidisciplinary OSCE, there is no doubt that the cost will be greater., Aim: Through this study a research project was conducted to develop, implement and evaluate a multidisciplinary OSCE model within limited resources., Methods: This research project went through the steps of blueprinting, station writing, resources reallocation, implementation and finally evaluation., Results: The developed model was implemented in the Primary Health Care (PHC) program which is one of the pillars of the Community-Based undergraduate curriculum of the Faculty of Medicine, Suez Canal University (FOM-SCU). Data for evaluation of the implemented OSCE model were derived from two resources. First, feedback of the students and assessors through self-administered questionnaires was obtained. Second, evaluation of the OSCE psychometrics was done. The deliverables of this research project included a set of validated integrated multi-disciplinary and low cost OSCE stations with an estimated reliability index of 0.6., Conclusion: After having this experience, we have a critical mass of faculty members trained on blueprinting and station writing and a group of trained assessors, facilitators and role players. Also there is a state of awareness among students on how to proceed in this type of OSCE which renders future implementation more feasible.

Background: With the first qualifying examination administered September 15, 2014, complex general surgical oncology (CGSO) is now a board-certified specialty. We aimed to assess the attitudes and perceptions of current and future surgical oncology fellows regarding the recently instituted Accreditation Council for Graduate Medical Education (ACGME) accreditation., Methods: A 29-question anonymous survey was distributed to fellows in surgical oncology fellowship programs and applicants interviewing at our fellowship program., Results: There were 110 responses (79 fellows and 31 candidates). The response rate for the first- and second-year fellows was 66 %. Ninety-percent of the respondents were aware that completing an ACGME-accredited fellowship leads to board eligibility in CGSO. However, the majority (80 %) of the respondents stated that their decision to specialize in surgical oncology was not influenced by the ACGME accreditation. The fellows in training were concerned about the cost of the exam (90 %) and expressed anxiety in preparing for another board exam (83 %). However, the majority of the respondents believed that CGSO board certification will be helpful (79 %) in obtaining their future career goals. Interestingly, candidate fellows appeared more focused on a career in general complex surgical oncology (p = 0.004), highlighting the impact that fellowship training may have on organ-specific subspecialization., Conclusions: The majority of the surveyed surgical oncology fellows and candidates believe that obtaining board certification in CGSO is important and will help them pursue their career goals. However, the decision to specialize in surgical oncology does not appear to be motivated by ACGME accreditation or the new board certification.

Background: The few studies directly comparing the methodological approach of peer role play (RP) and standardized patients (SP) for the delivery of communication skills all suggest that both methods are effective. In this study we calculated the costs of both methods (given comparable outcomes) and are the first to generate a differential cost-effectiveness analysis of both methods., Methods: Medical students in their prefinal year were randomly assigned to one of two groups receiving communication training in Pediatrics either with RP (N = 34) or 19 individually trained SP (N = 35). In an OSCE with standardized patients using the Calgary-Cambridge Referenced Observation Guide both groups achieved comparable high scores (results published). In this study, corresponding costs were assessed as man-hours resulting from hours of work of SP and tutors. A cost-effectiveness analysis was performed., Results: Cost-effectiveness analysis revealed a major advantage for RP as compared to SP (112 vs. 172 man hours; cost effectiveness ratio .74 vs. .45) at comparable performance levels after training with both methods., Conclusions: While both peer role play and training with standardized patients have their value in medical curricula, RP has a major advantage in terms of cost-effectiveness. This could be taken into account in future decisions.

The US National Assessment of Educational Progress (NAEP) measures cognitive competences in reading and mathematics of US students (last 2012 survey N = 50,000). The long-term development based on results from 1971 to 2012 allows a prediction of future cognitive trends. For predicting US averages also demographic trends have to be considered. The largest groups' (White) average of 1978/80 was set at M = 100 and SD = 15 and was used as a benchmark. Based on two past NAEP development periods for 17-year-old students, 1978/80 to 2012 (more optimistic) and 1992 to 2012 (more pessimistic), and demographic projections from the US Census Bureau, cognitive trends until 2060 for the entire age cohort and ethnic groups were estimated. Estimated population averages for 2060 are 103 (optimistic) or 102 (pessimistic). The average rise per decade is dec = 0.76 or 0.45 IQ points. White-Black and White-Hispanic gaps are declining by half, Asian-White gaps treble. The catch-up of minorities (their faster ability growth) contributes around 2 IQ to the general rise of 3 IQ; however, their larger demographic increase reduces the general rise at about the similar amount (-1.4 IQ). Because minorities with faster ability growth also rise in their population proportion the interactive term is positive (around 1 IQ). Consequences for economic and societal development are discussed.

Background: Residency programs are developing new methods to assess resident competence and to improve the quality of formative assessment and feedback to trainees. Simulation is a valuable tool for giving formative feedback to residents., Objective: To develop an objective structured clinical examination (OSCE) to improve formative assessment of senior pediatrics residents., Methods: We developed a multistation examination using various simulation formats to assess the skills of senior pediatrics residents in communication and acute resuscitation. We measured several logistical factors (staffing and program costs) to determine the feasibility of such a program., Results: Thirty-one residents participated in the assessment program over a 3-month period. Residents received formative feedback comparing their performance to both a standard task checklist and to peers' performance. The program required 16 faculty members per session, and had a cost of $624 per resident., Conclusions: A concentrated assessment program using simulation can be a valuable tool to assess residents' skills in communication and acute resuscitation and provide directed formative feedback. However, such a program requires considerable financial and staffing resources.

Background: The American College of Surgeons has developed a reliable and valid OSCE (objective structured clinical examination) to assess the clinical skills of incoming postgraduate year 1 surgery residents, but the cost and logistics of implementation have not been described., Methods: Fixed costs included staff time, medical supplies, facility fee, standardized patient (SP) training time, and one OSCE session. Variable costs were incurred for additional OSCE sessions. Costs per resident were calculated and modeled for increasing the number of test takers., Results: American College of Surgeons OSCE materials and examination facilities were free. Fixed costs included training 11 SPs for 4 hours ($1,540), moulage and simulation material ($469), and administrative effort for 44 hours ($2,200). Variable cost for each session was $1,540 (SP time). Total cost for the first session was $6,649 ($664/resident), decreased to $324/resident for 3 sessions, and projected to further decline to $239/resident for 6 sessions., Conclusions: The cost decreased as the number of residents tested increased. To manage costs, testing more trainees by regional collaboration is recommended., (Copyright © 2015 Elsevier Inc. All rights reserved.)

Objectives: Increasing numbers of educational institutions in the medical field choose to replace their conventional admissions interviews with a multiple mini-interview (MMI) format because the latter has superior reliability values and reduces interviewer bias. As the MMI format can be adapted to the conditions of each institution, the question of under which circumstances an MMI is most expedient remains unresolved. This article systematically reviews the existing MMI literature to identify the aspects of MMI design that have impact on the reliability, validity and cost-efficiency of the format., Methods: Three electronic databases (OVID, PubMed, Web of Science) were searched for any publications in which MMIs and related approaches were discussed. Sixty-six publications were included in the analysis., Results: Forty studies reported reliability values. Generally, raising the number of stations has more impact on reliability than raising the number of raters per station. Other factors with positive influence include the exclusion of stations that are too easy, and the use of normative anchored rating scales or skills-based rater training. Data on criterion-related validities and analyses of dimensionality were found in 31 studies. Irrespective of design differences, the relationship between MMI results and academic measures is small to zero. The McMaster University MMI predicts in-programme and licensing examination performance. Construct validity analyses are mostly exploratory and their results are inconclusive. Seven publications gave information on required resources or provided suggestions on how to save costs. The most relevant cost factors that are additional to those of conventional interviews are the costs of station development and actor payments., Conclusions: The MMI literature provides useful recommendations for reliable and cost-efficient MMI designs, but some important aspects have not yet been fully explored. More theory-driven research is needed concerning dimensionality and construct validity, the predictive validity of MMIs other than those of McMaster University, the comparison of station types, and a cost-efficient station development process., (© 2014 John Wiley & Sons Ltd.)

Objective: The Australian Dental Council is responsible for the assessment of overseas-qualified dentists seeking to practice dentistry in Australia. The aim of this paper is to reflect on the Council's assessment and examination process through the experiences and perceptions of overseas-qualified dentists in Australia., Methods: Qualitative methods were used. Life stories of 49 overseas-qualified dentists from 22 nationalities were analysed to discern significant themes and patterns. We focused on their overall as well as specific experiences of various stages of the examination. The analysis was consistent with a hermeneutic phenomenological approach to social scientific research., Results: Most participants referred to 'cost' of the examination process in terms of lost income, expenses and time. The examination itself was perceived as a tough assessment process. Some participants seemed to recognise the need for a strenuous assessment due to differences in patient management systems in Australia compared with their own country. Significantly, most of the participants stressed the importance of support structures for overseas-qualified dentists involved in or planning to undertake the examination. These considerations about the examination experience were brought together in two themes: (1) 'a tough stressful examination'; and (2) 'need for support.', Conclusion: This paper highlights the importance of support structures for overseas-qualified dentists. Appropriate support (improved information on the examination process, direction for preparation and training, further counselling advice) by recognised bodies may prevent potential exploitation of overseas-qualified dentists. Avenues that have been successful in providing necessary support, such as public sector schemes, offer policy options for limited recruitment of overseas-qualified dentists in Areas of Need locations. Such policies should also be in line with the local concerns and do not reduce opportunities for Australian-qualified dentists.

Objective: To compare educational and vocational outcomes among adults with childhood-onset systemic lupus erythematosus (SLE) and adult-onset SLE., Methods: We used data derived from the 2002–2010 cycles of the Lupus Outcomes Study, a longitudinal cohort of 1,204 adult subjects with SLE. Subjects ages 18–60 years living in the US (n = 929) were included in the analysis and were classified as childhood-onset SLE if age at diagnosis was <18 years (n = 115). Logistic regression was used to assess the unadjusted and adjusted effect of childhood-onset SLE, sex, race/ethnicity, baseline age, urban or rural location, and US region on the likelihood of completing a bachelor's degree. Generalized estimating equations were used to assess the effect of childhood-onset SLE, demographics, education, and disease-related factors on the odds of employment, accounting for multiple observations over the study period., Results: Subjects with childhood-onset SLE were on average younger (mean ± SD 29 ± 10 years versus 44 ± 9 years), with longer disease duration (mean ± SD 15 ± 10 years versus 11 ± 8 years). Subjects with adult-onset SLE and childhood-onset SLE subjects were equally likely to complete a bachelor's degree. However, subjects with childhood-onset SLE were significantly less likely to be employed, independent of demographic and disease characteristics (odds ratio 0.62, 95% confidence interval 0.42–0.91)., Conclusion: While subjects with SLE are just as likely as those with adult-onset SLE to complete college education, childhood-onset SLE significantly increases the risk of not working in adulthood, even when controlling for disease and demographic factors. Exploring reasons for low rates of employment and providing vocational support may be important to maximize long-term functional outcomes in patients with childhood-onset SLE.

Background: The Simulated Colonoscopy Objective Performance Evaluation (SCOPE) was developed to fill the need for a lower-cost, non-virtual-reality (VR)-based assessment tool. This study aimed to evaluate the ability of SCOPE to assess endoscopic skills objectively., Methods: Four tasks were created using the Kyoto Kagaku colonoscopy model (Kyoto Kagaku Co., Ltd., Kyoto, Japan). The SCOPE tasks included Scope Manipulation (SM) requiring torque and tip deflection to align a shape in the colon with a matching shape on the monitor; Tool Targeting (TT) requiring coordination with biopsy forceps to contact a metal target; Loop Management (LM) requiring prevention, recognition, and reduction of a redundant sigmoid colon with navigation to the cecum; and Mucosal Inspection (MI) requiring identification of simulated polyps during withdrawal and retroflexion. Key performance metrics were identified, and a normalized scoring system was developed. For the study, 35 subjects were stratified into three cohorts based on colonoscopy experience: novice (0-50 colonoscopies; n = 11), intermediate (51-139 colonoscopies; n = 13), and experienced (>140 colonoscopies; n = 11). The subjects performed two trials of all four tasks., Results: Across all four tasks, the experienced endoscopists (E) consistently outperformed the intermediates (I), who in turn outperformed the novices (N). The mean normalized scores with 95 % confidence intervals (CI) are as follows: SM: N (54; range, 26-82), I (92; range, 79-106), E (106; range, 93-118) (p = 0.0006). TT: N (40; range, 24-55), I (77; range, 63-91), E (88; range, 72-105) (p < 0.0001). LM: N (51; range, 24-79), I (80; range, 59-101), E (101; range, 98-105) (p = 0.003). MI: N (73; range, 53-92), I (85; range, 76-95), E (100; range, 91-108) (p = 0.013). Total score: N (218; range, 155-280), I (335; range, 299-371), E (395; range, 371-419) (p < 0.0001). The test-retest reliability (0.6) for the expert total score was respectable., Conclusions: The validity evidence from this study shows that scores on SCOPE tasks can differentiate between groups expected to have different levels of technical skill. This model shows promise as a low-technology tool for objective assessment or training of endoscopic skills.

Background: The Objective Structured Clinical Examination (OSCE) was introduced by Harden et al. (1975) trying to answer the problems regarding the assessment of clinical competencies. Despite increasingly widespread use of OSCEs, debate continues with arguments as 'why using such a demanding format if other methods are available?', Aim: To review and synthesize evidence on technical and economic feasibility of OSCE in undergraduate medical studies., Methods: Best Evidence Medical Education methodology was applied by two independent coders to 1083 studies identified by literature search from 1975 until the end of 2008., Key Findings: The OSCE is a feasible approach to the assessment of clinical competence for use in different cultural and geographical contexts; to assess a wide range of learning outcomes; in different specialties and disciplines; for formative and summative purposes; to assess students a curriculum or an educational intervention; in the different phases of education including the early and later years of the undergraduate curriculum; and in different health care professions., Conclusion: Despite being an expensive test format, evidence suggests that the use of OSCE produces reliable results. The study also suggests that one reason for the wide-scale adoption of the OSCE and the feasibility of its use in different contexts and situations is its inherent flexibility in terms of the number of students that can be assessed, the number of examiners included, the type of patients represented and the format of the examination itself, including the length of the examination, the number and duration of stations.

Context: Models of short-term remediation for failing students are typically associated with improvements in candidate performance at retest. However, the process is costly to deliver, particularly for performance retests with objective structured clinical examinations (OSCEs), and there is increasing evidence that these traditional models are associated with the longitudinal underperformance of candidates., Methods: Rather than a traditional OSCE model, sequential testing involves a shorter 'screening' format, with an additional 'sequential' test for candidates who fail to meet the screening standard. For those tested twice, overall pass/fail decisions are then based on results on the full sequence of tests. In this study, the impacts of sequential assessment on student performance, cost of assessment delivery and overall reliability were modelled using data sourced from a final graduating OSCE in an undergraduate medical degree programme., Results: Initial modelling using pre-existing OSCE data predicted significant improvements in reliability in the critical area, reflected in pilot results: 13.5% of students (n = 228) were required to sit the sequential OSCE. One student (0.4%) was identified as representing a false positive result (i.e. under the previous system this student would have passed the OSCE but failed on extended testing). Nine students (3.9%) who would have required OSCE retests under the prior system passed the full sequence and were therefore able to graduate at the normal time without loss of earnings. Overall reliability was estimated as 0.79 for the full test sequence. Significant cost savings were realised., Conclusions: Sequential testing in OSCEs increases reliability for borderline students because the increased number of observations implies that 'observed' student marks are closer to 'true' marks. However, the station-level quality of the assessment needs to be sufficiently high for the full benefits in terms of reliability to be achieved. The introduction of such a system has financial benefits, good validity inferences and has proved acceptable to students and other stakeholders., (© 2013 John Wiley & Sons Ltd.)

Background: The importance of basic surgical skills is entirely agreed among surgical educators. However, restricted by ethical issues, finance etc, the basic surgical skills training is increasingly challenged. Increasing cost gives an impetus to the development of cost-effective training models to meet the trainees' acquisition of basic surgical skills. In this situation, a cost-effective training framework was formed in our department and introduced here., Methods: Each five students were assigned to a 'training unit'. The training was implemented weekly for 18 weeks. The framework consisted of an early, a transitional, an integrative stage and a surgical skills competition. Corresponding training modules were selected and assembled scientifically at each stage. The modules comprised campus intranet databases, sponge benchtop, nonliving animal tissue, local dissection specimens and simulating reality operations. The training outcomes used direct observation of procedural skills as an assessment tool. The training data of 50 trainees who were randomly selected in each year from 2006 to 2011 year, were retrospectively analysed., Results: An excellent and good rate of the surgical skills is from 82 to 88%, but there is no significant difference among 6 years (P > 0.05). The skills scores of the contestants are markedly higher than those of non-contestants (P < 0.05). The average training cost per trainee is about $21.85-34.08., Conclusion: The present training framework is reliable, feasible, repeatable and cost-effective. The skills competition can promote to improve the surgical skills level of trainees., (© 2012 The Authors. ANZ Journal of Surgery © 2012 Royal Australasian College of Surgeons.)

Objective: To evaluate how the utility (reliability, validity, acceptability, feasibility, cost and educational impact) of a communication-OSCE was influenced by whether or not station-specific (StSp) checklists were used together with a generic instrument and whether or not narrative feedback was provided to students., Methods: At ten stations, faculty members rated standardized patient-student interactions using the common ground (CG) instrument (at all stations) and StSp-checklists. Both raters and patients provided written feedback. The impact of changing the design on the various utility parameters was assessed: reliability by means of a generalizability study, cost using the Reznick model and the other utility parameters by means of a survey., Results: Use of the generic instrument (CG) proved more reliable (G coefficient=0.67) than using the StSp-checklists (G=0.47) or both (G=0.65) while there was a high correlation between both scale scores (Pearsons'r=0.86). The cost was 6.5% higher when StSp-checklists were used and 5% higher when narrative feedback was provided., Conclusion: The utility of a communication OSCE can be enhanced by omitting StSp-checklists and by providing narrative feedback to students., Practice Implications: The same generic assessment scale can be used in all stations of a communication OSCE. Providing feedback to students is promising but it increases the costs., (Copyright © 2012 Elsevier Ireland Ltd. All rights reserved.)

Purpose: To compare three separate assessment stations used for selection to Core Medical Training (CMT) and to determine the effect of reducing the number from three to two., Methods: Quantitative analysis of candidates' assessment station scores, financial analysis of costs of the selection process and quantitative and qualitative surveys of candidates and assessors., Results: The assessment stations used for selection to CMT were reliable and valid for assessing suitability for employment as a CMT trainee. There was no significant difference in candidate ranking if only two assessment stations were used rather than three, i.e. there was no change in the likelihood of receiving a job offer. All of the assessment stations were perceived to have face validity by candidates and assessors. The efficiency of the selection process could be improved without loss of quality if two stations were used rather than three., Conclusions: Using two assessment stations rather than three would appear to improve the efficiency and maintain the quality of the CMT selection process while reducing costs.

The OSCE (Objective Structured Clinical Examination) is considered the most valid and reliable method for assessing the clinical skills of students training for health professions, but its use is limited by the related high costs. We analyzed the cost retrospectively of using an OSCE designed for second-year students (2009) in our degree course, adopting the Reznick et al. guidelines (1993), which recommend assessing both high-end costs and low-end costs. The high-end costs adopting the OSCE amounted to € 145.23 per student, while the low-end costs were € 31.51 per student. Considering the economic crisis and the cost-containment measures applied also in nursing education, strategies for further reducing costs are discussed., (Copyright © 2011 Elsevier Ltd. All rights reserved.)

The new membership examination of the Royal College of General Practitioners (MRCGP) has been in existence since autumn 2007. The MRCGP assessments aim to assure the RCGP of the competence of a general practitioner specialty trainee (GPST). Once the MRCGP is obtained, the GPST is then able to apply for a certificate of completion of training (CCT) from the General Medical Council (GMC). The MRCGP demonstrates the commonly described educational theory model of Miller's pyramid (see Figure 1). The Applied Knowledge Test (AKT), as the name suggests, aims for more than mere recall and arguably is at the 'knows how' level. Current literature suggests that the 'shows how' level constitutes competency-based assessment rather than performance. In the MRCGP, this takes the form of Clinical Skills Assessment (CSA). The level of 'does' is assessed via workplace-based assessments. These are performed throughout the three years of GPST training. They also take into account that the skill of performance incorporates not only competence, but also a combination of other influences mentioned in the Cambridge model for delineating performance and competence. This article will focus on the AKT. It is a three-hour 200-item multi-choice question (MCQ) examination that covers the three broad domains described in Box 1. The questions are in one of three formats: single best answer, extended matching questions and completion of algorithms. Literature exists that looks at the characteristics of a good assessment system. However, it is the work of van der Vleuten that is widely cited. He suggested that validity, reliability, educational impact, cost-effectiveness and acceptability must all be considered when constructing an assessment. I will now focus on the AKT in relation to each of the factors.

Introduction: An objective structured clinical examination (OSCE) has been implemented in preclinical dentistry. It was taken at an early stage (propaedeutics course). The objectives of this study were to evaluate the reliability, validity, and feasibility of the examination, and the effect of circuit number on OSCE score., Methods: The OSCE was designed by an expert committee on the basis of pre-reviewed blueprints and checklists. Eleven stations formed an interdisciplinary circuit. Six groups of students (n = 62) passed sequentially round the same circuit. Statistical analysis was performed by using SPSS. Reliability was determined by measurement of internal consistency (Cronbach's α, Guttman's λ(2) ), standard error of measurement (SEM) (comprising generalisability index α, dependability index ϕ and pass 150;fail reliability p(c) ), consistency coefficient κ, item 150;scale correlation (Pearson correlation), and, because the unidimensionality of the stations could not be assumed, factor analysis including varimax rotation. Convergent validity (Pearson correlation, t-test), and predictive validity for future preclinical courses and the final preclinical examination were assessed by analysis of variance (ANOVA). The effect of the circuit number on score improvement was calculated, including a correction for the general competence of the students (ANOVA). Cost was calculated on the basis of the time invested., Results: Fifty-three out of sixty-two students passed the OSCE (mean score: 67%, SD 7.7, range, 47-81). Scores for each station correlated significantly with total scores (r = 0.35-0.54, P < 0.01). For internal consistency, α = 0.75 (relative SEM 3.8) and λ(2) = 0.766. The dependability index was ϕ = 0.694 (absolute SEM 4.4), p(c) = 0.89 and κ = 0.61. Factor analysis yielded two components: dental-materials-oriented stations and all other stations (explained variance 43%). Scores correlated significantly with success in passing practical tests (i.e. performing dental procedures under examination conditions) (known group validity, P < 0.01) and with scores for subsequent courses and the final preclinical examination (Physikum) (predictive validity, P < 0.001). Later groups performed 4% better on average (CI 95%: 1.2-6.8%; P < 0.01). The cost was 181 Euro per student., Conclusions: The OSCE is reliable and valid in the context of preclinical dentistry. The cost is substantial. The problem of improvement of students' results with ascending circuit number has to be addressed., (© 2011 John Wiley & Sons A/S.)