Search

Summary Four, and later five, of the medical schools in the University of London collaborated in administering one multiple choice question (MCQ) paper in Obstetrics and Gynaecology to their students at the end of the Obstetrics and Gynaecology courses. The paper was amended twice after intervals of approximately 12 months. The results showed differences in performance between the five schools on questions and alternatives within questions, which were common to all editions of the paper (the ‘short’ paper). These differences were also shown in the first two editions of the full paper (the ‘long’ paper), but were not apparent in the third. There was a significant improvement in performance from the first to the second paper edition of the long by approximately 11%, but this was reversed from the second to the third edition, where there was a significant decrease in performance by approximately 4%. We cannot here exclude the possibility that this decrease in the third edition has resulted from sample bias. The rank order of the schools may result from differences in methods of teaching. It seems that such relative performance between London medical schools is contributed to by the differences in course length and by the occurrence of a mid-course test. It appears also that student performance in any of the schools has not been uniform over the period of the study, there being a peak during the period when the second edition of the paper was used. Variations in student selection occurring between schools may effect some of these differences. There was also a marked difference in students' performance between Obstetrics and Gynaecology. The short papers appeared easier than the full papers. Whilst this may be due to the greater clarity of phrasing for the questions of the short paper, we also believe the subject matter of the short paper may be more relevant. A distinct effect could also be shown, over the period, due to improvements in the wording of the questions. It is possible that students perform better at Obstetrics than at Gynaecology.

Context Multiple-choice questions (MCQs) are frequently used to assess students in health science disciplines. However, few educators have formal instruction in writing MCQs and MCQ items often have item-writing flaws. The purpose of this study was to examine the impact of item-writing flaws on student achievement in high-stakes assessments in a nursing programme in an English-language university in Hong Kong. Methods From a larger sample, we selected 10 summative test papers that were administered to undergraduate nursing students in 1 nursing department. All test items were reviewed for item-writing flaws by a 4-person consensus panel. Items were classified as 'flawed' if they contained > or = 1 flaw. Items not containing item-writing violations were classified as 'standard'. For each paper, 2 separate scales were computed: a total scale which reflected the characteristics of the assessment as administered and a standard scale which reflected the characteristics of a hypothetical assessment including only unflawed items. Results The proportion of flawed items on the 10 test papers ranged from 28-75%; 47.3% of all items were flawed. Fewer examinees passed the standard scale than the total scale (748 [90.6%] versus 779 [94.3%]). Conversely, the proportion of examinees obtaining a score > or = 80% was higher on the standard scale than the total scale (173 [20.9%] versus 120 [14.5%]). Conclusions Flawed MCQ items were common in high-stakes nursing assessments but did not disadvantage borderline students, as has been previously demonstrated. Conversely, high-achieving students were more likely than borderline students to be penalised by flawed items.

Summary The examination performance of a cohort of clinical undergraduates has been scrutinized. The cohort was notable for a high failure rate at the university preclinical examination, but this failure rate did not reappear consistently in all four final clinical subjects. The assaying examinations were not only valid for their objectives, but also measured a range of other attributes. Multiple choice question papers scores detect both quantitative and qualitative differences in knowledge between the high and low fail risk candidates. The effect of A-level grade is variable but is carried up to the final qualifying examinations. High performers tend to have good A-levels. The contribution of the preclinical course to the clinical qualifying examination was small, varied between the preclinical subjects and also between the clinical subjects and two streams of students. It is shown that lack of an adequate standard in science subjects before entry to university is associated with subsequent disadvantage in clinical training. Such disadvantage could be reduced by an introductory premedical course. The contributions made by the differing educational experiences within the cohort could be detected in many of the examinations; in particular, poor performance at the end of first clinical year pathology multiple choice questions paper examination was associated with poor performance at finals. Failure in the first part of finals indicated a likelihood of further failure, factors such as sex and motivation are shown to affect multiple choice question performance whilst others such as work-overload do not. The high risk fail candidates can be detected 2 years before finals and might be given special educational treatment designed to improve their chance of first time success as there is no efficient way of identifying them before acceptance into the medical school.

OBJECTIVE Modern postgraduate medical training requires both accurate and reliable selection procedures. An essential first step is to conduct detailed job analysis studies. This paper reports data on a series of job analyses to develop a competency model for three secondary care specialties (anaesthesia, obstetrics and gynaecology, and paediatrics). METHODS Three independent job analysis studies were conducted. The content validity of the resulting competency domains was tested using a questionnaire-based study with specialty trainees (specialist registrars [SpRs]) and consultants drawn from the three specialties. Job analysis was carried out in the Yorkshire and the Humber region in the UK. The validation study was administered with additional participants from the West Midlands and Trent regions in the UK. This was an exploratory study. The outcome is a set of competency domains with data on their importance at senior house officer, SpR and consultant grade in each specialty. RESULTS The study produced a model comprising 14 general competency domains that were common to all the three specialties. However, there were significant between-specialty differences in both definitions of domains and the ratings of importance attached to them. CONCLUSIONS The results indicate that a wide range of attributes beyond clinical knowledge and academic achievement need to be considered in order to ensure doctors train and work within a specialty for which they have a particular aptitude. This has significant implications for developing selection criteria for specialty training. Future research should explore the content validity of these competency domains in other secondary care specialties.

Objectives This study investigates: (1) which personality traits are typical of medical students as compared to other students, and (2) which personality traits predict medical student performance in pre-clinical years. Design This paper reports a cross-sectional inventory study of students in nine academic majors and a prospective longitudinal study of one cohort of medical students assessed by inventory during their first preclinical year and by university examination at the end of each pre-clinical year. Subjects and methods In 1997, a combined total of 785 students entered medical studies courses in five Flemish universities. Of these, 631 (80AE4%) completed the NEO-PI-R (i.e. a measure of the Five-Factor Model of Personality). This was also completed by 914 Year 1 students of seven other academic majors at Ghent University. Year end scores for medical students were obtained for 607 students in Year 1, for 413 in Year 2, and for 341 in Year 3. Results Medical studies falls into the group of majors where students score highest on extraversion and agreeableness. Conscientiousness (i.e. self-achievement and self-discipline) significantly predicts final scores in each pre-clinical year. Medical students who score low on conscientiousness and high on gregariousness and excitement-seeking are significantly less likely to sit examinations successfully. Conclusions The higher scores for extraversion and agreeableness, two dimensions defining the interpersonal dynamic, may be beneficial for doctors’ collaboration and communication skills in future professional practice. Because conscientiousness affects examination results and can be reliably assessed at the start of a medical study career, personality assessment may be a useful tool in student counselling and guidance.

Distance education (DE) has been used as a teaching mode for over 100 years for a variety of academic, vocational and recreational courses. This paper describes a postgraduate clinical epidemiology course conducted by DE for Australian and international health professionals, and compares the educational experiences of three groups of students (on-campus part-time, Australian DE and international DE). The groups were not expected to differ in academic achievement (marks) in eight subjects or in degree completion rates. The marks in each subject were not significantly different and typically were within five marks across all student groups. Approximately 40–50% of Australian DE students complete the graduate diploma coursework requirements in the minimum 2 years study. The completion rate increases to 70–80% as the period of candidature increases. The completion rates of Australian DE students were not significantly different from those of on-campus part-time students. Evaluation of the course by degree completion rates, comparison of academic achievement between off-campus and on-campus students, and qualitative feedback from students demonstrates that DE is as successful as on-campus teaching in providing training in clinical epidemiology at the postgraduate level. The flexibility of this mode of training as a means of providing public health training is discussed.

Summary This paper reports an investigation of the incremental validity of noncognitive tests. Incremental validity is the predictive ability of a measure when entered into a regression equation after the routine predictors have first been included. Three noncognitive tests—Rotter Locus of Control, Adjective Check List, and Student Orientations Survey—were administered to a first-year medical school class. When entered after the usual academic predictors of success in medical school—Undergraduate Grade Point Average and Medical College Admission Test—the three noncognitive tests added little to the prediction of first-year medical school Grade Point Average. It is concluded that while noncognitive measures are useful in characterizing a medical school class and in discovering nonacademic correlates of academic success in medical school, limited incremental validity related to first-year academic performance was demonstrated.