Search

The applied knowledge test (AKT) is a computer-delivered multiple choice test offered three times a year. It forms part of the Membership of the Royal College of General Practitioners (MRCGP) examination, which licenses doctors for UK general practice. This paper investigates the effect of current quality assurance processes in the AKT on the reliability of questions which are not pretested. Test questions in the AKT are based on the RCGP curriculum blueprint, referenced to current evidence, peer-reviewed and critically appraised before they are added to the question bank. There is a standardised process for question selection and test construction. Standard setting for the AKT follows a modified Angoff process where a panel of experts make individual judgements on each question of the test estimating the likelihood of the 'just-passing candidate' answering correctly. In the three examinations delivered in 2011 (AKT 11 (January), 12 (May) and 13 (October)) Cronbach's alpha was 0.88, 0.92, 0.91, respectively. For pretested questions, the predicted alpha was 0.90, 0.93, 0.92 and for non-pretested questions 0.84, 0.91, 0.92, respectively adjusting for test length using the Spearman-Brown formula. Pretesting of questions is unnecessary provided there is a systematic process of question and test construction which quality assures the AKT. This ensures that pretesting does not delay the use of questions to examine current knowledge and guidelines. [ABSTRACT FROM AUTHOR]

The article reports a study which was conducted to identify the organisational cultural dimensions in two general practices and their relationship to the development of Medical Leadership Competency Framework (MLCF) competencies. The two practices studied demonstrated features of rational outcomes-based culture; and a developmental culture, characterized as adaptive, flexible, and with an external focus. General practices need to take account of existing culture(s) before implementing the MLCF.

Background A new computer-based Applied Knowledge Test (AKT) has been developed for the licensing examination for general practice administered by the Royal College of General Practitioners. Aim The aim of this evaluation was to assess the acceptability, feasibility, and validity of the test as well as its transfer to a computerised format at local test centres. Design of study Computer-based test and postal questionnaire. Participants and setting Panel of examiners, Membership of the Royal College of General Practitioners (MRCGP) examination, UK. Method Self-administered postal questionnaires were sent to examiners not involved with the development of the test after completing it. Their performance scores were compared with those of candidates. Results The majority of participants (80.9%) were satisfied with the new computer-based test. Responses relating to content and attitudes to the test were also positive overall, but some problems with content were highlighted. Fewer examiners (61.9%) were positive about the physical comfort of the test centre, including seating, heating, and lighting. Examiners had significantly higher scores (mean 83.3%, range 69 to 93%, 95% confidence interval [CI] = 81.9 to 84.7%) than 'real' candidates (mean 75.0%, range 45 to 94%, 95% CI = 74.6 to 75.5%), who subsequently took an identical test. Conclusion The new computer-based licensing test (the AKT) was found to be acceptable to the majority of examiners. The pass-fail standard, determined by routine methods including an Angoff procedure, was supported by the higher success rate of examiners compared with candidates. The use of selected groups to assess high-stakes (licensing) examinations can be useful for assessing test validity. [ABSTRACT FROM AUTHOR]