Search

Absolute standard setting procedures are recommended for assessment in medical education. Absolute, test-centred standard setting procedures were introduced for written assessments in the Liverpool MBChB in 2001. The modified Angoff and Ebel methods have been used for short answer question-based and extended matching question-based papers, respectively. Data collected has been analysed to investigate whether reliable standards can be achieved for small-scale, medical school-based assessments, to establish the minimum number of judges required and the effect of a discussion phase on reliability. The root mean squared error (RMSE) has been used as a measure of reliability and used to compute 95% confidence intervals for comparison to the examination statistics. The RMSE has been used to calculate the minimum number of judges required to obtain a predetermined minimum level of reliability, and the effect of the number of judges and number of items have been examined. Values of the RMSE obtained vary from 0.9 to 2.2%. Using average variances across each paper type, the minimum number of judges to obtain a RMSE of less than 2% is 10 or more judges before discussion or 6 or more judges after discussion. The results indicate that including a discussion phase improves the reliability and reduces the minimum number of judges required. Decision studies indicate that increasing the number of questions included in the assessments would not significantly improve the reliability of the standard setting.

Introduction In 2008 the BTS published the first National Emergency Oxygen Guidance. Subsequent national audits in 2008 and 2010 have shown poor compliance with these guidelines. In order to improve the use of emergency oxygen Liverpool Medical School has included this topic in a new Patient Safety Programme (PSP) developed to provide practical skills for the 335 fifth year students as a preparation for their Foundation Programme. Methodology (1) Computer based knowledge test (CBKT) of 60 MCQ questions, based on the BTS guidance and BNF are accessed on-line by the students. Each student has to correctly answer 80% of 10 randomly selected questions. (2) Practical oxygen management session including demonstration of oxygen delivery systems and blood gas sampling in a Clinic Skills Department. (3) Ward based supervised skills including completion of five oxygen prescriptions and blood gas samples. Results The CBKT score was low with an average of 6.2 of 10 questions answered correctly after the first attempt. After 10 attempts only 72% students passed thus 28% (94 students) were unable to reach the pass threshold. The Abstract P75 table 1 relates the pass rate to the numbers of attempts shows only a modest incremental increase in the cumulative pass rate. In contrast, all students rapidly gained the skills to undertake blood gas sampling and were able to write oxygen prescriptions based on SpO2 results. Conclusions This study confirms that medical students, like other staff, have a poor basic knowledge about the use of emergency oxygen. The Liverpool PSP is addressing this knowledge gap but in view of the results further education (eg, seminars and e-learning) will be provided prior to the CBKT. We recommend this type of programme to other Medical Schools and clinical staff. [ABSTRACT FROM AUTHOR]

Professional values and behaviours are intrinsic to all medical practice, yet remain one of the most difficult subjects to integrate explicitly into a curriculum. Professionalism for the twenty-first century raises challenges not only to adapting the course to changing societal values but also for instilling skills of ongoing self-directed continuous development in trainees for future revalidation. This Guide is based on the contemporary available literature and focuses on instilling Professionalism positively into both undergraduate and postgraduate training deliberately avoiding the more negative aspects of Fitness to Practise. The literature on Professionalism is extensive. An evidence-based approach has been taken throughout. We have selected only some of the available publications to offer practical advice. Comprehensive reviews are available elsewhere (van Mook et al. 2009a-g). This Guide takes a structured stepwise approach and sequentially addresses: (i) agreeing an institutional definition, (ii) structuring the curriculum to integrate learning across all years, (iii) suggesting learning models, (iv) harnessing the impact of the formal, informal and hidden curricula and (v) assessing the learning. Finally, a few well-evaluated case studies for both teaching and assessment have been selected to illustrate our recommendations.