Search

Professionalism is an important topic in medical education today. While much work has focused on defining professionalism and teaching medical students the appropriate interpersonal behaviours, relatively little research has looked at meaningful ways of assessing the relevant attributes.The Educational Commission for Foreign Medical Graduates (ECFMG®) clinical skills assessment (CSA®) uses standardised patients (SPs) to evaluate the readiness of graduates of international medical schools to enter accredited graduate training programmes in the USA. Doctor interpersonal skills, including professional qualities such as rapport, are evaluated as part of the CSA. Attentiveness, attitude and empathy, all facets of professional behaviour, are specifically targeted as part of the assessment.To date, over 35 000 candidates have been assessed, encompassing more than 370 000 individual SP encounters. Based on a 1-year cohort of examinees, the reliability of the individual professionalism-related component scores ranged from 0.61 to 0.70. Doctors who had graduated from medical school more recently, or were younger, generally obtained higher ratings. Professional qualities were only marginally related to measures of basic science and clinical science proficiency. Female candidates were rated significantly higher than male candidates on all dimensions.While some professional behaviours are probably best measured using formats such as surveys, self-assessment and critical incident techniques, certain aspects of the domain can be reliably and validly measured in SP examinations. [ABSTRACT FROM AUTHOR]

Objectives In multiple-choice tests using a `don't-know' option the number of correct minus incorrect answers was used as the test score (formula scoring) in order to reduce the measurement error resulting from random guessing. In the literature diverging results are reported when comparing formula scoring and number-right scoring, the scoring method without the don't-know option. Design To investigate which method was most appropriate, both scoring methods were used in true–false tests (block tests) taken at the end of a second- and third-year educational module (block). The students were asked to answer each item initially by choosing from the response options true, false or don't know, and secondly to replace all don't-know answers by a true–false answer. Setting Maastricht University, The Netherlands. Subjects Medical students. Results The correct scores for the don't-know answered items were found to be 4·5% and 5·9%, respectively, higher than expected with pure random guesswork. This represents a source of bias with formula scoring, because students who were less willing to guess obtained lower scores. The average difference in the correct minus incorrect score for the two scoring methods (2·5%, P < 0·001, and 3·4%, P < 0·001, respectively) indicates the size of the bias (compare: the standard deviation of the score equals 11%). Test reliability was higher with formula scoring (0·72 vs. 0·66 and 0·74 vs. 0·66), but the difference decreased when the test was restricted to items which were close to the core content of the block (0·81 vs. 0·77, resp. 0·75 vs. 0·70). Conclusions In deciding what scoring method to use, less bias (number-right scoring) has to be weighed against higher reliability (formula scoring). Apart from these psychometric reasons educational factors must be considered. [ABSTRACT FROM AUTHOR]