Back to Search
Start Over
Workplace-based assessments of junior doctors: do scores predict training difficulties?
- Source :
- Medical Education; Dec2011, Vol. 45 Issue 12, p1190-1198, 9p, 4 Charts, 2 Graphs
- Publication Year :
- 2011
-
Abstract
- Medical Education 2011: 45: 1190-1198 Objectives Workplace-based assessment (WPBA) is an increasingly important part of postgraduate medical training and its results may be used as evidence of professional competence. This study evaluates the ability of WPBA to distinguish UK Foundation Programme (FP) doctors with training difficulties and its effectiveness as a surrogate marker for deficiencies in professional competence. Methods We conducted a retrospective observational study using anonymised records for 1646 trainees in a single UK postgraduate deanery. Data for WPBAs conducted from August 2005 to April 2009 were extracted from the e-portfolio database. These data included all scores submitted by trainees in FP years 1 and 2 on mini-clinical evaluation exercise (mini-CEX), case-based discussion (CbD), direct observation of procedural skills (DOPS) and mini-peer assessment tool (mini-PAT) assessments. Records of trainees in difficulty, as identified by their educational supervisors, were tagged as index cases. Main outcome measures were odds ratios (ORs) for associations between mean WPBA scores and training difficulties. Further analyses by the reported aetiology of the training difficulty (health-, conduct- or performance-related) were performed. Results Of the 1646 trainees, 92 had been identified as being in difficulty. Mean CbD and mini-CEX scores were lower for trainees in difficulty and an association was found between identified training difficulties and average scores on the mini-CEX (OR = 0.54; p = 0.034) and CbD (OR = 0.39; p = 0.002). A receiver operator characteristic curve analysis of mean WPBA scores for diagnosing 'in difficulty' status yielded an area under the curve of 0.64, indicating weak predictive value. There was no statistical evidence that mean scores on DOPS and mini-PAT assessments differed between the two groups. Conclusions Analysis of a large dataset of WPBA scores revealed significant associations between training difficulties and lower mean scores on both the mini-CEX and CbD. Models show that using WPBA scores is, however, not a valid way of screening for trainees in difficulty. Workplace-based assessments have value as formative assessments that prompt supervision, feedback and reflection. They should not be relied upon to certify competence and their use for such ends may reduce their effectiveness in training. Their results should be interpreted in the context of multiple other methods of assessment, with the aim of achieving a genuinely holistic and representative assessment of professional competence. [ABSTRACT FROM AUTHOR]
- Subjects :
- CLINICAL competence
NATIONAL competency-based educational tests
ANALYSIS of variance
CONFIDENCE intervals
EDUCATIONAL tests & measurements
EPIDEMIOLOGY
STUDY & teaching of medicine
SCIENTIFIC observation
RESEARCH funding
LOGISTIC regression analysis
DATA analysis
RETROSPECTIVE studies
RECEIVER operating characteristic curves
DATA analysis software
Subjects
Details
- Language :
- English
- ISSN :
- 03080110
- Volume :
- 45
- Issue :
- 12
- Database :
- Complementary Index
- Journal :
- Medical Education
- Publication Type :
- Academic Journal
- Accession number :
- 67480176
- Full Text :
- https://doi.org/10.1111/j.1365-2923.2011.04056.x