Search

Multi-source feedback (MSF) has become the accepted mechanism of ensuring the appropriate professional behaviour of doctors. It is part of the mandatory assessment of doctors in training and is to be utilized as part of the revalidation of trained doctors. There is significant variation in the models of MSF currently used within the National Health Service and new models of MSF are being designed by various specialties. No single model has been recognized as the 'gold standard'. However, there is a large published literature concerning MSF, both in the context of health systems and, more extensively, within industry. This published literature is reviewed, drawing attention to aspects of MSF systems in which there is consensus on effective approaches as well as other aspects in which there is doubt about the optimum approach. In the light of the review 10 principles key in the development of effective MSF models have been produced.

Multi-source feedback, or 360-degree assessment, is an important part of the assessment of people in the workplace, in both health and industry. Almost all published work concentrates on content validity and generalizability. However, an assessment system needs construct validity, and has to have practicability and acceptability, without sacrificing fitness for purpose, content validity or inter-rater reliability. This was a six-year study of the first UK-wide hospital-based multi-source feedback system, in the specialty of obstetrics and gynaecology. This paper describes the development of the assessment tool, its use and the analyses of the results in several areas. These are picking up poor performance, congratulating good behaviour, construct validity, the number of domains to be measured, and the minimum number of raters. The study demonstrated that the Team Observation system in reality only measured a very limited number of attributes, and that the main construct under scrutiny is interpersonal behaviour. The system can identify those who may have a problem, using less than 10 raters, and yet the process can be a positive experience for the large majority of people who have been assessed.

Higher Specialist Training (HST) has been radically reformed in the last decade, and specialties have been urged to conduct further reform. This paper reports on an explorative investigation into curriculum provision and assessment within four specialties in the West Midlands. Methods include analysis of curriculum documents; observation of the RITA process; and interviews with consultants (4), specialist registrars (5) and members of the regional higher specialist training management team (3). Three major themes identified within the interviews are discussed: the content and use of the curriculum; the impact of service-based learning; and issues around assessment.

This study evaluates a pilot careers service for junior doctors provided by a non-medical careers adviser. Evaluation forms and interviews indicate high satisfaction with this service. The junior doctors valued help with CVs and interviews, information regarding career routes and meetings with senior doctors arranged by the adviser. A longitudinal survey suggests that these doctors became clearer about their career pathways compared with controls, although they did not view the adviser as influencing their career choices. The employment of non-medical careers advisers is proposed. The evaluation was principally conducted by the Centre for Research in Medical and Dental Education, School of Education, University of Birmingham, UK.

Background: Introduced in 2006, the objective structured public health examination (OSPHE) is a part of the examination for the UK Membership of the Faculty of Public Health. Designed to simulate real work in public health, the same five generic competencies were assessed simultaneously in each of six stations in 1 h 48 min. This study estimates the examination reliability of the OSPHE. Methods: Data from all 198 candidates, 54 questions, 39 examiners over 9 test days in 2006 were analysed for reliability and sources of error using multilevel cross-classified analysis and generalisability theory. Results: A reliability co-efficient of 0.85 was estimated (95% confidence interval 0.82–0.88). Little variance was accounted for by questions or examiners. Twelve questions would increase reliability to 0.92. More than one examiner per question has negligible impact on reliability. Conclusion: Multilevel modelling estimates examination reliability which is more rigorous than Cronbach's alpha. Traditional generalisability methods based on the analysis of variance (ANOVA) are unable to do this. The reliability of this examination compares favourably with the literature despite having just six test stations and under 2 h per candidate. Other specialties could consider the use of parallel testing of the same key competencies at all stations within simulated workplace events.

This study was to see if the team assessment of behaviours (TAB) 360 degree assessment tool was able to identify interpersonal behaviour problems in doctors in training, to see if feedback was useful, to gauge the value of the process by those involved, and to learn lessons about implementing the process for the future.TAB was administered to assess interpersonal behaviours of senior house officers in four hospitals in the West Midlands, UK. In addition, questionnaires were sent to all participants, some were interviewed about the whole process, and records kept of the time involved.One hundred and seventy-one SHO volunteers received 1378 assessments. The median number of ratings per SHO was 8 (mode 9). Sixty-four percent of SHOs received 'no concern' ratings in all four behaviours (domains) assessed. Twenty-one percent received one 'some concern' rating. Fifteen percent received more than one 'concern' rating.Assessors and trainees found the process practical, valuable and fair. Educational supervisors found it valuable, although only 23% learned something new about their trainees. Clinical tutors valued the system. Administrative staff found it time consuming. The TAB four-domain rating form with its single pass category identified specific concern about volunteer trainees' professional behaviour. Not all trainees received skilled feedback.

The new regional, centralized system of selecting GP trainees in the West Midlands aims to be fair and non-discriminatory in the processes of recruiting the most suitable candidates and allocating the successful candidates to their preferred posts. This paper provides an audit of the achievements of all (n = 165) the candidates in the October 2000 selection round. Initial analysis suggests that gender is not a significant factor in determining outcomes. In this paper, the data are examined with respect to candidates' ethnic background and their country of qualification. Two levels of analysis are possible, based on average 'success' or placement rates, and average scores achieved at each stage of the selection process. The analysis, using one-sided significance tests, suggests that White candidates, and those qualifying in the UK, are more likely to succeed in the selection process. Possible explanations are offered from a more detailed interrogation of the selection criteria. Finally, the implications for the next selection round are discussed in the light of these findings.