Search

Introduction: Medical students who are diagnosed with a specific learning difficulty (SpLD) will typically receive a reasonable adjustment within examinations in the form of modified assessment provision (MAP). This study investigated whether the timing of SpLD diagnosis and subsequent implementation of MAP has an impact on performance in applied medical knowledge multiple choice question (MCQ) assessments., Method: The MCQ performance of 108 students diagnosed with SpLD who received a MAP was monitored and compared with 1960 students who received an unmodified assessment, over 5 years of a medical program. Students who received a SpLD diagnosis in the latter years of the program were identified as not receiving a MAP in assessments prior to diagnosis., Results: Differences were found between declaration and diagnosis, with 44.4% of students who declared and 48.1% who did not declare subsequently receiving a diagnosis. Students with SpLD who receive a MAP increase their applied medical knowledge assessment performance, although there is a delay of up to a year for this impact to reach significance., Conclusion: Early diagnosis of SpLD is necessary to ensure the intended benefit is received from MAP.

Background: The preparation for colonoscopy is elaborate and complex. In the context of colorectal cancer screening, up to 11% of patients do not keep their colonoscopy appointments and up to 33% of those attending their appointments have inadequately cleansed bowels that can delay cancer diagnosis and treatment. A smartphone app may be an acceptable and wide-reaching tool to improve patient adherence to colonoscopy., Objective: The aim of this qualitative study was to employ a user-centered approach to design the content and features of a smartphone app called colonAPPscopy to support individuals preparing for their colonoscopy appointments., Methods: We conducted 2 focus group discussions (FGDs) with gastroenterology patients treated at the McGill University Health Centre in Montreal, Canada. Patients were aged 50 to 75 years, were English- or French-speaking, and had undergone outpatient colonoscopy in the previous 3 months; they did not have inflammatory bowel disease or colorectal cancer. FGDs were 75 to 90 min, conducted by a trained facilitator, and audiotaped. Participants discussed the electronic health support tools they might use to help them prepare for the colonoscopy, the content needed for colonoscopy preparation, and the features that would make the smartphone app useful. Recordings of FGDs were transcribed and analyzed using thematic analysis to identify key user-defined content and features to inform the design of colonAPPscopy., Results: A total of 9 patients (7 male and 2 female) participated in one of 2 FGDs. Main content areas focused on bowel preparation instructions, medication restrictions, appointment logistics, communication, and postcolonoscopy expectations. Design features to make the app useful and engaging included minimization of data input, reminders and alerts for up to 7 days precolonoscopy, and visual aids. Participants wanted a smartphone app that comes from a trusted source, sends timely and tailored messages, provides reassurance, provides clear instructions, and is simple to use., Conclusions: Participants identified the need for postcolonoscopy information as well as reminders and alerts in the week before colonoscopy, novel content, and features that had not been included in previous smartphone-based strategies for colonoscopy preparation. The ability to tailor instructions made the smartphone app preferable to other modes of delivery. Study findings recognize the importance of including potential users in the development phase of building a smartphone app., (©Maida J Sewitch, Carlo A Fallone, Peter Ghali, Ga Eun Lee. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 02.07.2019.)

Objectives: Undergraduate medical students experience a considerable amount of stress and anxiety due to frequent exams. The goal of the present study was to examine the development of exam related anxiety and to test for a correlation between anxiety and learning approaches., Methods: A whole class of 212 medical students was invited to participate in the study. During their first term, trait anxiety and learning approaches were assessed by use of the state-trait-anxiety inventory (STAI-T) and the approaches-and-study-skills-inventory-for-students (ASSIST), respectively. Acute state anxiety was assessed twice in the course of the second term. To that extent, the STAI-S in combination with measuring salivary cortisol were employed immediately before two oral anatomy exams., Results: Our most important results were that a surface learning approach correlated significantly with anxiety as a trait and that students with a predominantly strategic approach to learning were the least anxious yet academically most successful., Conclusion: As surface learners are at risk of being academically less successful and because anxiety is a prerequisite for burn-out, we suggest that medical faculties place particular emphasis on conveying strategies for both, coping with stress and successful learning., Competing Interests: The authors have declared that no competing interests exist.

Objective: To develop a scientifically sound and standardized medical language examination for the State of Bavaria according to the requirements set forth by the 87 th Conference of State Health Ministers. This Sprachtest für Ausländische Mediziner (SAM, Language Test for Foreign Physicians) ought to become part of the licensing procedure for foreign physicians in Germany. Using testing stations that are situation-based, it will assess medical language competence and communication skills at the proficiency level of C1. Methods: Case scenarios for four mini-interviews of 10 minutes each were developed. For the written part of the exam, consisting of two separate testing stations with a combined duration of 40 minutes, one video of a physician taking a patient's history and one annotated set of laboratory results were developed. Based on the analysis of existing scientific literature as well as real-life examples, features and characteristics of professional medical language were identified. This served as the basis for the development of itemized rating scales for each of the testing stations. The exam was validated in three simulated trial runs. Each run was video-recorded and subsequently graded by a team of test-raters. Results: 19 participants took part in the three trial runs. A benchmark (gold standard) could be set for 18 of these. A ROC-analysis yielded an AUC-value of .83. This confirmed the predictive quality of the SAM-test. The reliability of the SAM-test could be calculated for only ten participants. The internal consistency, calculated with the use of Cronbach's Alpha, was .85. The pass/fail mark was calculated based on the Youden-Index and yielded a result of >60%. Conclusion: The SAM-test presents a statistically valid medical language examination with a high level of objectivity. As required, it tests language proficiency at the level of C1 and uses authentic communication scenarios within a standardized test setting. Additional studies with larger test samples will help to further validate this test and thus guarantee a higher degree of reliability., Competing Interests: The authors declare that they have no competing interests.

Introduction: For assessment to fill an educational role, students must see the results generated by assessment as valuable, and actively engage with this feedback in order to support learning. Few studies include examinees as stakeholders in validation beyond general notions of acceptability. Here, we explore students as stakeholders in the validation of a newly implemented assessment., Methods: A student-relevant validity framework based on the unified theory of validity was created and adapted to a survey format. Likert-style items were used to examine first- and second-year medical students' perceptions of a new cumulative assessment, with several open-ended items. Analysis included: mean ratings per subscale of validity evidence, thematic analysis of comments, and a correlation between questionnaire subscores and exam performance., Results: Seventy-seven students participated (20.5%). Student perceptions of the assessment were favourable, with significantly different ratings across validity evidence (Response Process (4.8 (SD = 0.7); scored/6), Content (4.6(0.9)), Consequential (4.4(0.8)), Internal Structure (4.2(0.9)), and Relationship to Other Variables (4.0(1.0))). Exam performance correlated with subscores for Relationship to Other Variables (r = 0.34, p < 0.005) and Response Process (r = 0.24, p < 0.05)., Discussion: Students perceived the assessment as facilitating learning, providing 'checkpoints', and were disappointed when it did not meet their expectations regarding the purpose of assessment. If students perceive that results do not reflect their future performance in clinical environments, or do not align with their perceived purpose of assessment, the educational value of assessment may be limited. It is critical to understand when, and how students engage in interpreting and integrating assessment-generated feedback to ensure that assessment contributes positively to learning.

Introduction: Ongoing monitoring of cohort demographic variation is an essential part of quality assurance in medical education assessments, yet the methods employed to explore possible underlying causes of demographic variation in performance are limited. Focussing on properties of the vignette text in single-best-answer multiple-choice questions (MCQs), we explore here the viability of conducting analyses of text properties and their relationship to candidate performance. We suggest that such analyses could become routine parts of assessment evaluation and provide an additional, equality-based measure of an assessment's quality and fairness., Methods: We describe how a corpus of vignettes can be compiled, followed by examples of using Microsoft Word's native readability statistics calculator and the koRpus text analysis package for the R statistical analysis environment for estimating the following properties of the question text: Flesch Reading Ease (FRE), Flesch-Kincaid Grade Level (Grade), word count, sentence count, and average words per sentence (WpS). We then provide examples of how these properties can be combined with equality and diversity variables, and the process automated to provide ongoing monitoring., Conclusions: Given the monitoring of demographic differences in assessment for assurance of equality, the ability to easily include textual analysis of question vignettes provides a useful tool for exploring possible causes of demographic variations in performance where they occur. It also provides another means of evaluating assessment quality and fairness with respect to demographic characteristics. Microsoft Word provides data comparable to the specialized koRpus package, suggesting routine use of word processing software for writing items and assessing their properties is viable with minimal burden, but that automation for ongoing monitoring also provides an additional means of standardizing MCQ assessment items, and eliminating or controlling textual variables as a possible contributor to differential attainment between subgroups.

Background: It has been documented that international students face diverse challenges due to language and cultural barriers. International medical students suffer from personal distress, a lack of support and perform poorer than local fellow-students in clinical examinations. It has been documented that international medical students benefit from peer-led tutorials in their first year. We investigated the effectiveness of a tutorial offered for international medical students in their second year. Methods: A peer-guided examination preparation course with interactive elements for second year international medical students was designed, learning objectives were defined. Two evaluations were undertaken: In a quantitative assessment, students were asked to fill out five multiple-choice-questions at the beginning of every session of the tutorial (pre-test) as well as to participate in a post-test at the end of the semester in which all former multiple-choice-questions were re-used. Using a qualitative approach, participants were asked for their thoughts and comments in a semi-structured interview at the end of the semester. Results: International students (N=12) showed significantly better results in the post- than in the pre-test (t(11)=-8.48, p<.001, d=1.95). Within the interviews, international students (N=10) reported to have benefited from technical and didactic, as well as social learning experiences. The individual lectures students were asked to contribute were discussed controversially. Conclusion: Our peer-guided tutorial for second year international medical students is an effective and well accepted possibility to prepare these students for examinations.

The objective of this study was to compare the examination scores before and after implementation of a study program based on high yield topics on the Council of Resident Education in Obstetrics and Gynecology (CREOG) In-Training Examination. This prospective cohort study compared scores from academic years 2012 to 2014 of University of Hawai'i obstetrics and gynecology residents who participated in a directed study program based on selected high yield topics from the CREOG Test Item Summary Booklet. Topics were considered high yield if more than 75% of the program residents answered the topic questions incorrectly during the immediately preceding CREOG In-Training Examination administration. Residents were assigned topics to research and present at monthly teaching sessions. The presentations were made accessible in a wiki website. The intervention was initiated in 2012 and discontinued in 2013. The primary outcome was the difference among CREOG In-Training Examination scores before the study program, during the study program, and after the study program was discontinued. Only scores of residents who sat for all three exams were included. Eleven residents were present during the duration of the study period and sat for all three CREOG examinations. During the year of the educational activity, paired individual resident CREOG exams scores increased significantly from the 2012 CREOG administration (mean = 194.7) to the 2013 CREOG administration (mean = 208.2). These findings demonstrate that the CREOG Test Item Summary Booklet and the wiki platform can be used to effectively direct educational efforts resulting in improvements in CREOG examination performance., Competing Interests: None of the authors identify any conflict of interest.

Introduction: Tablet-based assessments offer benefits over scannable-paper assessments; however, there is little known about the impact to the variability of assessment scores., Methods: Two studies were conducted to evaluate changes in rating technology. Rating modality (paper vs tablets) was manipulated between candidates (Study 1) and within candidates (Study 2). Average scores were analyzed using repeated measures ANOVA, Cronbach's alpha and generalizability theory. Post-hoc analyses included a Rasch analysis and McDonald's omega., Results: Study 1 revealed a main effect of modality (F (1,152) = 25.06, p < 0.01). Average tablet-based scores were higher, (3.39/5, 95% CI = 3.28 to 3.51), compared with average scan-sheet scores (3.00/5, 95% CI = 2.90 to 3.11). Study 2 also revealed a main effect of modality (F (1, 88) = 15.64, p < 0.01), however, the difference was reduced to 2% with higher scan-sheet scores (3.36, 95% CI = 3.30 to 3.42) compared with tablet scores (3.27, 95% CI = 3.21 to 3.33). Internal consistency (alpha and omega) remained high (>0.8) and inter-station reliability remained constant (0.3). Rasch analyses showed no relationship between station difficulty and rating modality., Discussion: Analyses of average scores may be misleading without an understanding of internal consistency and overall reliability of scores. Although updating to tablet-based forms did not result in systematic variations in scores, routine analyses ensured accurate interpretation of the variability of assessment scores., Conclusion: This study demonstrates the importance of ongoing program evaluation and data analysis.

Objectives: The current study aimed to carry out a post-validation item analysis of multiple choice questions (MCQs) in medical examinations in order to evaluate correlations between item difficulty, item discrimination and distraction effectiveness so as to determine whether questions should be included, modified or discarded. In addition, the optimal number of options per MCQ was analysed., Methods: This cross-sectional study was performed in the Department of Paediatrics, Arabian Gulf University, Manama, Bahrain. A total of 800 MCQs and 4,000 distractors were analysed between November 2013 and June 2016., Results: The mean difficulty index ranged from 36.70-73.14%. The mean discrimination index ranged from 0.20-0.34. The mean distractor efficiency ranged from 66.50-90.00%. Of the items, 48.4%, 35.3%, 11.4%, 3.9% and 1.1% had zero, one, two, three and four nonfunctional distractors (NFDs), respectively. Using three or four rather than five options in each MCQ resulted in 95% or 83.6% of items having zero NFDs, respectively. The distractor efficiency was 91.87%, 85.83% and 64.13% for difficult, acceptable and easy items, respectively ( P <0.005). Distractor efficiency was 83.33%, 83.24% and 77.56% for items with excellent, acceptable and poor discrimination, respectively ( P <0.005). The average Kuder-Richardson formula 20 reliability coefficient was 0.76., Conclusion: A considerable number of the MCQ items were within acceptable ranges. However, some items needed to be discarded or revised. Using three or four rather than five options in MCQs is recommended to reduce the number of NFDs and improve the overall quality of the examination., Competing Interests: CONFLICT OF INTEREST The authors declare no conflicts of interest.

Background Studying regularly has been a known method for better memory and long term retention of a subject. Getting students to study on a regular basis has been tried in many ways and its impact is increased if they understand the importance and value the efficiency of it. Objectives With large number of students, it would be a time consuming process to follow one on one interactions or study sessions. Hence, we introduced a new strategy, the "Question of the Day", which is interesting, challenging and most of all; it has produced positive influence on the examination performance of the students who performed this task. Main objective of this study was to make students answer questions related to the topics taught every day. Method Regular studying and updating the subject knowledge was provided through Question of the Day (Annexure-1). The students were given the questions on a regular basis during dissection classes and were asked to document the answers for those questions. Result The students who worked on the "Question of the Day" scored significantly higher in the exams than those who did not. The results of the opinion survey clearly indicated that this exercise was liked by most of the students. Conclusion "Question of the day" did have a positive influence on the student's performance. Most of the students felt that this exercise had a positive impact on their study methods, interaction with peers and teachers, retention and recollection of the knowledge.

Background and Objectives: Our objective was to examine the predictive validity of the American Board of Family Medicine's (ABFM) In-Training Examination (ITE) with regard to predicting outcomes on the ABFM certification examination., Methods: This study used a repeated measures design across three levels of medical training (PGY1--PGY2, PGY2--PGY3, and PGY3--initial certification) with three different cohorts (2010--2011, 2011--2012, and 2012--2013) to examine: (1) how well the residents' ITE scores correlated with their test scores in the following year, (2) what the typical score increase was across training years, and (3) what was the sensitivity, specificity, positive predictive value, and negative predictive value of the PGY3 scores with regard to predicting future results on the MC-FP Examination., Results: ITE scores generally correlate at about .7 with the following year's ITE or with the following year's certification examination. The mean growth from PGY1 to PGY2 was 52 points, from PGY2 to PGY3 was 34 points, and from PGY3 to initial certification was 27 points. The sensitivity, specificity, positive predictive value, and negative predictive value were .91, .47, .96, and .27, respectively., Conclusion: The ITE is a useful predictor of future ITE and initial certification examination performance.

Objective: To examine the relationship between study strategies and performance on a high stakes medical licensing exam entitled the United States Medical Licensing Examination Step 1., Methods: The action research project included seventy nine student participants at the Texas A & M Health Science Center College of Medicine during their pre-clinical education. Data collection included pre-matriculation and matriculation academic performance data, standardized exam data, and the Learning and Study Strategies Instrument. Multiple regression analyses were conducted. For both models, the dependent variable was the Step 1 score, and the independent variables included Medical College Admission Test, Undergraduate Grade Point Average, Year 1 Average, Year 2 Average, Customized National Board of Medical Examiners Average, Comprehensive Basic Science Exam score, and Learning and Study Strategy Instrument sub-scores. Model 2 added Comprehensive Basic Science Self-Assessment average., Results: Concentration (Model 1 - β = .264; Model 2 - β = .254) was the only study strategy correlated with Step 1 performance. The other statistically significant predictors were Customized National Board of Medical Examiners Average (β = .315) and Year 2 Average (β = .280) in Model 1 and Comprehensive Basic Science Self-Assessment Average (β = .338) in Model 2., Conclusions: There does appear to be a relationship between the study strategy concentration and Step 1 licensing exam performance. Teaching students to practice and utilize certain techniques to improve concentration skills when preparing for and taking exams may help improve licensing exam scores.

Background: Whole slide images (WSIs) used in medical education can provide new insights into how histological slides are viewed by students. We created software infrastructure which tracks viewed WSI areas, used it during a practical exam in oral pathology and analyzed collected data to discover students' viewing behavior., Methods: A view path tracking solution, which requires no specialized equipment, has been implemented on a virtual microscopy software platform (WebMicroscope, Fimmic Ltd, Helsinki, Finland). Our method dynamically tracks view paths across the whole WSI area and all zoom levels, while collecting the viewing behavior data centrally from many simultaneous WSI users. We used this approach during the exam to track how all students (N = 88) viewed WSIs (50 per student) when answering exam questions (with no time limit). About 74,000 records with information about subsequently displayed WSI areas were saved in the central database. Gathered data was processed and analyzed in multiple ways. Generated images and animations showed view fields and paths marked on WSI thumbnails, either for a single student or multiple students answering the same question. A set of statistics was designed and implemented to automatically discover certain viewing patterns, especially for multiple students and WSIs. Calculated metrics included average magnification level on which a WSI was displayed, dispersion of view fields, total viewing time, total number of view fields and a measure depicting how much a student was focused on diagnostic areas of a slide., Results: Generated visualizations allowed us to visually discover some characteristic viewing patterns for selected questions and students. Calculated measures confirmed certain observations and enabled generalization of some findings across many students or WSIs. In most questions selected for the analysis, students answering incorrectly tended to view the slides longer, go through more view fields, which were also more dispersed - all compared to students who answered the questions correctly., Conclusions: Designed and implemented view path tracking appeared to be a useful method of uncovering how students view WSIs during an exam in oral pathology. Proposed analysis methods, which include visualizations and automatically calculated statistics, were successfully used to discover viewing patterns., Virtual Slides: The virtual slide(s) for this article can be found here: http://www.diagnosticpathology.diagnomx.eu/vs/13000&#95;2014&#95;208.

Introduction: Medical education is perceived as stressful. As excessive stress hampers students' performance, stress management is required for medical students. This study was aimed to assess the effect of Mind Sound Technology (MST), an intelligence enhancing program, on psychological well-being of medical undergraduates during exam stress., Materials and Methods: Forty-two medical students were recruited and Dukes Health Profile scoring was done at baseline and during Exam Stress (ES). After pre-intervention measurements, the students were randomized into two groups: non-practitioners and MST practitioners. Post-intervention measurement was done at the end of 6 weeks when the students had examination., Results: Students showed a significant increase (P < 0.001) in negative health scores like perceived health scores, anxiety, and depression and a significant decrease (P < 0.001) in positive health scores like Self-Esteem, Mental Health Score, Social Health Score, and General Health Score during exam when compared with baseline. MST practice increased positive health scores (P < 0.001) and decreased perceived health score (P < 0.01), anxiety, depression, and anxiety-depression scores significantly (P < 0.001) when compared with ES score. Non practitioners did not show any significant change in any of the scores when compared with ES score. Six weeks of MST practice by medical students have improved the academic scores (P < 0.05) when compared with their non-practitioner counterpart., Conclusion: Thus, practice of MST has helped in coping up the stress that occurs during examination and improved academic performance in medical undergraduates.

Two experiments were used to assess the efficacy of the keyword mnemonic method in adults. In Experiment 1, immediate and delayed recall (at a one-day interval) were assessed by comparing the results obtained by a group of adults using the keyword mnemonic method in contrast to a group using the repetition method. The mean age of the sample under study was 59.35 years. Subjects were required to learn a list of 16 words translated from Latin into Spanish. Participants who used keyword mnemonics that had been devised by other experimental participants of the same characteristics, obtained significantly higher immediate and delayed recall scores than participants in the repetition method. In Experiment 2, other participants had to learn a list of 24 Latin words translated into Spanish by using the keyword mnemonic method reinforced with pictures. Immediate and delayed recall were significantly greater in the keyword mnemonic method group than in the repetition method group.