Paper trials: a qualitative study exploring the place of portfolios in making revalidation recommendations for Responsible Officers.

Background: A portfolio of supporting information (SI) reflecting a doctor's entire medical practice is now a central aspect of UK appraisal for revalidation. Medical revalidation, introduced in 2012, is an assessment of a doctor's competence and passing results in a five yearly license to practice medicine. It assesses of a doctor's professional development, workplace performance and reflection and aims to provide assurance that doctors are up-to-date and fit to practice. The dominant assessment mechanism is a portfolio. The content of the revalidation portfolio has been increasingly prescribed and the assessment of the SI is a fundamental aspect of the appraisal process which ultimately allows Responsible Officers (ROs) to make recommendations on revalidation. ROs, themselves doctors, were the first to undergo UK revalidation. This qualitative study explored the perceptions of ROs and their appraisers about the use of this portfolio of evidence in a summative revalidation appraisal.
Methods: 28 purposefully sampled London ROs were interviewed following their revalidation appraisal and 17 of their appraisers participated in focus groups and interviews. Thematic analysis was used to identify commonalities and differences of experience.
Results: SI was mostly easy to provide but there were challenges in gathering certain aspects. ROs did not understand in what quantities they should supply SI or what it should look like. Appraisers were concerned about making robust judgements based on the evidence supplied. A lack of reflection from the process of collating SI and preparing for appraisal was noted and learning came more from the appraisal interview itself.
Conclusions: More explicit guidance must be available to both appraisee and appraiser about what SI is required, how much, how it should be used and, how it will be assessed. The role of SI in professional learning and revalidation must be clarified and further empirical research is required to examine how best to use this evidence to make judgments as part of this type of appraisal.