Search

Objective: The participation of students from both undergraduate medical education (UGME) and postgraduate medical education (PGME) in independent patient care contributes to the development of knowledge, skills and the professional identity of students. A continuing collaboration between students and their preceptor might contribute to opportunities for students to independently provide patient care. In this systematic review, we aim to evaluate whether longitudinal training models facilitate the independent practice of students and what characteristics of longitudinal training models contribute to this process. Method: This systematic review was performed according to the PRISMA guidelines. In May 2020, we performed a search in three databases. Articles evaluating the impact of longitudinal training models on the independent practice of students from both UGME and PGME programmes were eligible for the study. A total of 68 articles were included in the study. Quality of the included studies was assessed using the Quality Assessment Tool for Studies with Diverse Designs (QATSDD). Results: Both UGME and PGME students in longitudinal training models are more frequently allowed to provide patient care independently when compared with their block model peers, and they also feel better prepared for independent practice at the end of their training programme. Several factors related to longitudinal training models stimulate opportunities for students to work independently. The most important factors in this process are the longitudinal relationships with preceptors and with the health care team. Conclusion: Due to the ongoing collaboration between students and their preceptor, they develop an intensive and supportive mutual relationship, allowing for the development of a safe learning environment. As a result, the professional development of students is fostered, and students gradually become part of the health care team, allowing them the opportunity to engage in independent patient care.

Het opleiden van huisartsen is net zoals dokteren een vak met een wetenschappelijke basis. Daarom wordt er onderzoek naar medisch onderwijs gedaan, dat bijvoorbeeld gericht is op interprofessioneel leren, docentprofessionalisering en toetsen. Een centrale conclusie uit dergelijk onderzoek is dat de ontwikkeling van de huisarts in spe het meest gebaat is bij het opdoen van klinische ervaring in de huisartsenpraktijk. De kwaliteit van deze ervaring blijkt bovendien voor een groot deel af te hangen van de rol van de opleider. De huisartsopleiding ontwikkelt zich op basis van nieuwe medische en onderwijskundige inzichten, en maatschappelijke ontwikkelingen. Willen opleidingen ook in de toekomst goede huisartsen afleveren, dan blijft onderzoek belangrijk.

Although learning-centred education would be most effective if all medical educators held learning-centred beliefs, many educators still hold teaching-centred beliefs. A previously developed theoretical model describes a relationship between beliefs, educational identity and ‘mission,’ meaning that which inspires and drives educators. To increase our understanding of why educators hold certain beliefs, we explored the empirical relationship between educators’ beliefs and their awareness of their educational identity and mission. A qualitative study was conducted using in-depth interviews with medical educators. We performed a deductive thematic analysis employing two existing models to examine educators’ beliefs about teaching and learning and their awareness of their educational identity and mission. Educators demonstrated both teaching-centred and learning-centred beliefs, which aligned with an awareness of their educational identity and mission. While educators who were unaware of both their identity and mission displayed teaching-centred beliefs, educators aware of their identity and mission displayed learning-centred beliefs. Those who were aware of their identity, but not their mission, displayed either teaching- or learning-centredness. Medical educators’ awareness of identity and mission are related to their beliefs about education. Further research is needed into whether beliefs can change over time by increasing identity and mission awareness.

Introduction: In competency-based medical education, direct observation (DO) of residents’ skills is scarce, notwithstanding its undisputed importance for credible feedback and assessment. A growing body of research is investigating this discrepancy. Strikingly, in this research, DO as a concrete educational activity tends to remain vague. In this study, we concretised DO of technical skills in postgraduate longitudinal training relationships. Methods: Informed by constructivist grounded theory, we performed a focus group study among general practice residents. We asked residents about their experiences with different manifestations of DO of technical skills. A framework describing different DO patterns with their varied impact on learning and the training relationship was constructed and refined until theoretical sufficiency was reached. Results: The dominant DO pattern was ad hoc, one-way DO. Importantly, in this pattern, various unpredictable, and sometimes unwanted, scenarios could occur. Residents hesitated to discuss unwanted scenarios with their supervisors, sometimes instead refraining from future requests for DO or even for help. Planned bi-directional DO sessions, though seldom practiced, contributed much to collaborative learning in a psychologically safe training relationship. Discussion and conclusion: Patterns matter in DO. Residents and supervisors should be made aware of this and educated in maintaining an open dialogue on how to use DO for the benefit of learning and the training relationship.

Background The educational beliefs of medical educators influence their teaching practices. Insight into these beliefs is important for medical schools to improve the quality of education they provide students and to guide faculty development. Several studies in the field of higher education have explored the educational beliefs of educators, resulting in classifications that provide a structural basis for diverse beliefs. However, few classification studies have been conducted in the field of medical education. We propose a framework that describes faculty beliefs about teaching, learning, and knowledge which is specifically adapted to the medical education context. The proposed framework describes a matrix in which educational beliefs are organised two dimensionally into belief orientations and belief dimensions. The belief orientations range from teaching-centred to learning-centred; the belief dimensions represent qualitatively distinct aspects of beliefs, such as ‘desired learning outcomes’ and ‘students’ motivation’. Methods We conducted in-depth semi-structured interviews with 26 faculty members, all of whom were deeply involved in teaching, from two prominent medical schools. We used the original framework of Samuelowicz and Bain as a starting point for context-specific adaptation. The qualitative analysis consisted of relating relevant interview fragments to the Samuelowicz and Bain framework, while remaining open to potentially new beliefs identified during the interviews. A range of strategies were employed to ensure the quality of the results. Results We identified a new belief dimension and adapted or refined other dimensions to apply in the context of medical education. The belief orientations that have counterparts in the original Samuelowicz and Bain framework are described more precisely in the new framework. The new framework sharpens the boundary between teaching-centred and learning-centred belief orientations. Conclusions Our findings confirm the relevance of the structure of the original Samuelowicz and Bain beliefs framework. However, multiple adaptations and refinements were necessary to align the framework to the context of medical education. The refined belief dimensions and belief orientations enable a comprehensive description of the educational beliefs of medical educators. With these adaptations, the new framework provides a contemporary instrument to improve medical education and potentially assist in faculty development of medical educators.

ObjectivesAt the very start of medical residency training, entrustment of clinical tasks may be a major challenge, on which current scientific knowledge is scarce. This study therefore aimed to gain insight into the process of supervisors’ decision making underlying the entrustment of clinical tasks at the start of one-on-one supervisor–trainee working relationships.SettingThis study was performed in a general practice (GP) training department in the Netherlands.ParticipantsFor this study, we recruited supervisor–trainee pairs who were just about to start the first year of GP residency training. Of 10 eligible supervisor–trainee pairs, 4 participated.DesignWe used a qualitative, social-constructivist research approach. Data from naturalistic non-participant observations and semistructured interviews with supervisors in four GP practices were triangulated and analysed using a thematic analysis approach.ResultsSupervisors’ early entrustment decisions were based on generic trainee qualities such as self-reflexivity, knowing one’s limitations and asking for help in time, rather than on task-specific performance. At the start of residency training, supervisors’ primary concern was to create a safe working and learning environment in which trainees could and would ask for timely supervision while being entrusted with challenging tasks. Supervisors used idiosyncratic entrustment strategies that were influenced by their propensity to trust, previous experiences with trainees, and their perspective on learning and teaching.ConclusionEntrustment decisions require high levels of safety for all stakeholders involved. Especially at the beginning of supervisor–trainee relationships, establishing an educational alliance in which entrustment is the subject of ongoing trainee–supervisor conversations is essential to achieve and maintain and optimal balance between trainee learning and patient safety. Additional research may further our understanding of early entrustment decision making and the role of generic trainee qualities in different settings.

Introduction Entrustable Professional Activities (EPAs) are developed to support the practical application of competency frameworks in postgraduate medical education (PGME) programmes. EPAs are used for the assessment of the trainees' competence development, which takes place by means of an entrustment decision, aiming to stimulate learning and independent practice in trainees. In this pilot study, we explore the feasibility and validity of EPA-based assessment in a General Practice (GP) training programme. Methods We used questionnaires to evaluate trainers' and trainees' experiences with the use of six EPAs for trainee learning, assessment and independent practice at the Out-of-Hours GP Center. Data were analysed quantitatively and qualitatively. Additionally, we examined the inter-item correlation between scores on EPA-based assessment and competency-based assessment using Spearman's Rho. Results EPA-based assessment provided opportunities for giving concrete feedback and substantiating competency-based assessment. No consistent correlation between EPA-based assessment and competency-based assessment could be detected. Only later in the course of the training programme a correlation was found between the EPA scores and the degree of independence of trainees. Discussion Results of this pilot study confirm the theories behind EPAs, as well as earlier research on EPAs in the workplace regarding trainee learning, assessment and independent practice. An important limitation of this study was the COVID-19 pandemic, as it influenced the results through reduced inclusion and follow-up, and through the impact on the workplace and trainee learning possibilities. Further research is needed to determine how EPAs support independent practice of trainees, as well as the assessment of trainee competency development.

BackgroundIt is assumed that portfolios contribute to self-regulated learning (SRL). Presence of these SRL processes within the documentation kept in portfolios is presupposed in common educational practices, such as the assessment of reflective entries. However, questions can be asked considering the presence of SRL within portfolios. The aim of this study was to gain insight into the documentation of SRL processes within the electronic (e)-portfolio content of medical trainees. SRL consists of numerous processes, for this study the focus was on self-assessment via reflection and feedback, goal-setting and planning, and monitoring, as these are the processes that health professions education research mentions to be supported by portfolios.MethodsA database containing 1022 anonymous e-portfolios from General Practitioner trainees was used to provide descriptive statistics of the various available e-portfolio forms. This was followed by a quantitative content analysis of 90 e-portfolios, for which, a codebook was constructed to rate the documentation of the included SRL processes.ResultsThe numbers of forms in the e-portfolios varied to a great extent. Content analysis showed a limited documentation of reflective entries, and available entries mainly described events and experiences without explanations and context. Feedback was generally limited to comments on what went well and lacked specificity, context and suggestions for future action. Learning goals and plans were short of specificity, but did contain challenging topics and different goals were compatible with each other. 75% of the e-portfolios showed (limited) signs of monitoring.ConclusionsThe e-portfolio content showed limited documentation of SRL processes. As documentation of SRL requires time and asks for a high level of introspection and writing skills, one cannot expect documentation of SRL processes to appear in e-portfolio content without efforts.

ObjectivesTrust plays an important role in workplace-based postgraduate medical education programmes. Trainers must trust their trainees for granting them greater independence. Trainees must trust their trainer for a safe learning environment. As trainers’ and trainees’ trust in each other plays an important role in trainee learning and development, the authors aimed to explore the development of the mutual trust relationship between trainers and trainees.SettingThis study was performed in a general practice training department in the Netherlands.ParticipantsAll trainers and trainees of the general practice training department were invited to participate. Fifteen trainers and 34 trainees, voluntarily participated in focus group discussions.Outcome measuresThe authors aimed to gain insight in the factors involved in the development of the mutual trust relationship between trainers and trainees, in order to be able to create a model for the development of a mutual trust relationship between trainers and trainees. The risk-based view of trust was adopted as leading conceptual framework.ResultsIn the first stage of trust development, trainers and trainees develop basic trust in each other. Basic trust forms the foundation of the trust relationship. In the second stage, trainers develop trust in trainees taking into account trainees’ working and learning performance, and the context in which the work is performed. Trainees trust their trainer based on the trainer’savailability and accessibility and the personal relationship between the trainee and their trainer. Trainee self-confidence modifies the development of a trust relationship.ConclusionThe development of a mutual trust relationship between trainers and trainees is a complex process that involves various stages, goals, factors and interactive aspects. As the mutual trust relationship influences the learning environment for trainees, greater emphasis on the mutual trust relationship may improve learning outcomes. Further research may explore the effect of long-term and short-term educational relationships on the trust relationship between trainers and trainees.

Context Direct observation (DO) of residents’ performance, despite the importance that is ascribed to it, does not readily fit in with the practice of postgraduate medical education (PGME); it is infrequent and the quality of observation may be poor in spite of ongoing efforts towards improvement. In recent literature, DO is mostly portrayed as a means to gather information on the performance of residents for purposes of feedback and assessment. The role of DO in PGME is likely to be more complex and poorly understood in the era of outcome‐based education. By exploring the possible complexity of DO in workplace learning, our research aims to contribute to a better use of DO in the practice of PGME. Methods Constructivist grounded theory informed our data collection and analysis. Data collection involved focus group sessions with supervisors in Dutch general practice who were invited to discuss the manifestations, meanings and effects of DO of technical skills. Theoretical sufficiency was achieved after four focus groups, with a total of 28 participants being included. Results We found four patterns of DO of technical skills: initial planned DO sessions; resident‐initiated ad hoc DO; supervisor‐initiated ad hoc DO, and continued planned DO sessions. Different patterns of DO related to varying meanings, such as checking or trusting, and effects, such as learning a new skill or experiencing emotional discomfort, all of them concerning the training relationship, patient safety or residents’ learning. Conclusions Direct observation, to supervisors, means much more than gathering information for purposes of feedback and assessment. Planned DO sessions are an important routine during the initiation phase of a training relationship. Continued planned bidirectional DO sessions, although infrequently practised, potentially combine most benefits with least side‐effects of DO. Ad hoc DO, although much relied upon, is often hampered by internal tensions in supervisors, residents or both., The authors offer explanations for the lack of direct observation (DO) in residency by exploring its manifestations, meanings and effects in training relationships; DO may benefit from pre‐planning and bi‐directionality.

Item does not contain fulltext Workplace-Based Assessment (WBA) plays a pivotal role in present-day competency-based medical curricula. Validity in WBA mainly depends on how stakeholders (e.g. clinical supervisors and learners) use the assessments-rather than on the intrinsic qualities of instruments and methods. Current research on assessment in clinical contexts seems to imply that variable behaviours during performance assessment of both assessors and learners may well reflect their respective beliefs and perspectives towards WBA. We therefore performed a Q methodological study to explore perspectives underlying stakeholders' behaviours in WBA in a postgraduate medical training program. Five different perspectives on performance assessment were extracted: Agency, Mutuality, Objectivity, Adaptivity and Accountability. These perspectives reflect both differences and similarities in stakeholder perceptions and preferences regarding the utility of WBA. In comparing and contrasting the various perspectives, we identified two key areas of disagreement, specifically 'the locus of regulation of learning' (i.e., self-regulated versus externally regulated learning) and 'the extent to which assessment should be standardised' (i.e., tailored versus standardised assessment). Differing perspectives may variously affect stakeholders' acceptance, use-and, consequently, the effectiveness-of assessment programmes. Continuous interaction between all stakeholders is essential to monitor, adapt and improve assessment practices and to stimulate the development of a shared mental model. Better understanding of underlying stakeholder perspectives could be an important step in bridging the gap between psychometric and socio-constructivist approaches in WBA.

Background: The educational beliefs of medical academics influence how they act in class and thus influence student learning. One component of these are beliefs academics hold about the qualities of teachers themselves. These teacher qualities range from behaviours and competencies to more personal attributes such as the teacher's identity and mission. However, it is unclear what medical academics believe to be key teacher qualities. Therefore, this study explored the variety of medical academics' beliefs about 'teacher qualities', aiming to identify and characterise profiles of academics with similar beliefs.Methods: We interviewed 26 expert academics from two medical schools to explore their beliefs about teacher qualities. A concentric onion-model focusing on teacher qualities was used to analyse and categorise the data deductively. Within each theme we developed subthemes inductively. To gain insight into the variety of beliefs we then clustered the participants into teacher profiles according to the themes. To better understand each of the profiles we carried out a quantitative study of the differences between profiles regarding subthemes, contextual and personal factors, and analysed statistical significance using Fisher's exact- and Student's t-tests for categorical and continuous data, respectively.Results: Four profiles of medical academics were identified, corresponding to the most central theme that each participant had reflected on: the 'Inspirer', 'Role-model', 'Practitioner', and 'Critic'. The focus of the profiles varied from external constraining factors within the 'Critic' profile to affective personal qualities within the 'Role-model' and 'Inspirer' profiles. The profiles could be regarded as hierarchically ordered by inclusiveness. Educational institute was the only significant factor related to the profiles.Conclusions: Besides the relevance of affective teacher qualities, the 'Inspirer' profile demonstrates the importance of developing a clear mission as a teaching academic, centred around student learning and professional development. In our view, academics who inspire their students continue to be inspired themselves.The practical implications are described for faculty development programmes, and for the potential value of using these profiles within medical schools. In the discourse on educational beliefs, the authors argue that more attention should be paid to affective qualities, in particular to explicating the educational mission of academics.

Contains fulltext : 153007.pdf (Publisher’s version ) (Closed access) Communication assessment in real-life consultations is a complex task. Generic assessment instruments help but may also have disadvantages. The generic nature of the skills being assessed does not provide indications for context-specific behaviour required in practice situations; context influences are mostly taken into account implicitly. Our research questions are: 1. What factors do trained raters observe when rating workplace communication? 2. How do they take context factors into account when rating communication performance with a generic rating instrument? Nineteen general practitioners (GPs), trained in communication assessment with a generic rating instrument (the MAAS-Global), participated in a think-aloud protocol reflecting concurrent thought processes while assessing videotaped real-life consultations. They were subsequently interviewed to answer questions explicitly asking them to comment on the influence of predefined contextual factors on the assessment process. Results from both data sources were analysed. We used a grounded theory approach to untangle the influence of context factors on GP communication and on communication assessment. Both from the think-aloud procedure and from the interviews we identified various context factors influencing communication, which were categorised into doctor-related (17), patient-related (13), consultation-related (18), and education-related factors (18). Participants had different views and practices on how to incorporate context factors into the GP(-trainee) communication assessment. Raters acknowledge that context factors may affect communication in GP consultations, but struggle with how to take contextual influences into account when assessing communication performance in an educational context. To assess practice situations, raters need extra guidance on how to handle specific contextual factors.

PurposeEntrustment has mainly been conceptualized as delegating discrete professional tasks. Because residents provide most of their patient care independently, not all resident performance is visible to supervisors; the entrustment process involves more than granting discrete tasks. This study explored how supervisors made entrustment decisions based on residents' performance in a long-term family medicine training program.MethodA qualitative nonparticipant observational study was conducted in 2014-2015 at competency-based family medicine residency programs in the Netherlands. Seven supervisor-resident pairs participated. During two days, one researcher observed first-year residents' patient encounters, debriefing sessions, and supervisor-resident educational meetings and interviewed them separately afterwards. Data were collected and analyzed using iterative, phenomenological inductive research methodology.ResultsThe entrustment process developed over three phases. Supervisors based their initial entrustment on prior knowledge about the resident. In the ensuing two weeks, entrustment decisions regarding independent patient care were derived from residents' observed general competencies necessary for a range of health problems (clinical reasoning, decision making, relating to patients); medical knowledge and skills; and supervisors' intuition. Supervisors provided supervision during and after encounters. Once residents performed independently, supervisors kept reevaluating their decisions, informed by residents' overall growth in competencies rather than by adhering to a predefined set of tasks.ConclusionsSupervisors in family medicine residency training took a holistic approach to trust, based on general competencies, knowledge, skills, and intuition. Entrustment started before training and developed over time. Building trust is a mutual process between supervisor and resident, requiring a good working relationship.

Van den Hombergh P, Schalk-Soekar S, Kramer A, Bottema B, Campbell S, Braspenning J. Hogere kwaliteit bij praktijken van huisartsopleiders. Huisarts Wet 2013;56(9):438-43. Huisartsen-in-opleiding mogen verwachten dat ze worden opgeleid door gekwalificeerde opleiders in modelpraktijken. Dit onderzoek kijkt naar de verschillen tussen huisartsopleiders en niet-opleiders op het gebied van kwaliteit van zorg (infrastructuur, team, informatie, kwaliteitsbeleid). We toetsten 203 huisartsenpraktijken (115 opleidingspraktijken) met 512 huisartsen (177 opleiders), die vrijwillig deelnamen aan de NHG-Praktijkaccreditering®, met het Visitatie Instrument Praktijkvoering op 369 items (142 huisartsniveau; 227 praktijkniveau). Per niveau hebben we de verschillen geanalyseerd (ANOVA, ANCOVA). Opleiders scoorden op alle items (behalve een) hoger en significant hoger op 47 items (13 items na covariatencorrectie). Opleidingspraktijken scoorden hoger op alle items en significant hoger op 61 items (23 items na covariatencorrectie). Het verschil met reguliere praktijken zit in een breder diagnostisch en therapeutisch aanbod, de ruimere inzet van teamkwaliteiten, een betere praktijkorganisatie, een groter aanbod voor mensen met chronische ziekten en kwaliteitsbeleid. Verder rapporteerden opleiders meer betrokkenheid en werkvreugde, en minder ervaren werkdruk dan reguliere huisartsen. Huisartsenopleiders en hun praktijken onderscheiden zich positief van de reguliere huisartsenzorg in zowel de breedte van het aanbod als de kwaliteit ervan. Deelname aan de NHG-Praktijkaccreditering® helpt opleidingsinstituten de kwaliteit van de opleiding te monitoren en te borgen. Bij het promoten van het opleiderschap kan ook de werkvreugde worden ingebracht.

Item does not contain fulltext Clinical workplaces offer postgraduate trainees a wealth of opportunities to learn from experience. To promote deliberate and meaningful learning self-regulated learning skills are foundational. We explored trainees' learning activities related to patient encounters to better understand what aspects of self-regulated learning contribute to trainees' development, and to explore supervisor's role herein. We conducted a qualitative non-participant observational study in seven general practices. During two days we observed trainee's patient encounters, daily debriefing sessions and educational meetings between trainee and supervisor and interviewed them separately afterwards. Data collection and analysis were iterative and inspired by a phenomenological approach. To organise data we used networks, time-ordered matrices and codebooks. Self-regulated learning supported trainees to increasingly perform independently. They engaged in self-regulated learning before, during and after encounters. Trainees' activities depended on the type of medical problem presented and on patient, trainee and supervisor characteristics. Trainees used their sense of confidence to decide if they could manage the encounter alone or if they should consult their supervisor. They deliberately used feedback on their performance and engaged in reflection. Supervisors appeared vital in trainees' learning by reassuring trainees, discussing experience, knowledge and professional issues, identifying possible unawareness of incompetence, assessing performance and securing patient safety. Self-confidence, reflection and feedback, and support from the supervisor are important aspects of self-regulated learning in practice. The results reflect how self-regulated learning and self-entrustment promote trainees' increased participation in the workplace. Securing organized moments of interaction with supervisors is beneficial to trainees' self-regulated learning.

Medical Education 2012: 46:604–612 Objectives Feedback in workplace-based clinical settings often relies on expert trainers’ judgements of directly observed trainee performance. There is ample literature on effective feedback, but in practice trainees in workplace-based training are not regularly observed. We aimed to examine external conditions that impact feedback in observational workplace-based assessment (WBA). Methods Interviews were conducted and the resulting data analysed using a qualitative, phenomenological approach. Between October 2009 and January 2010, we interviewed 22 postgraduate general practice trainees at two institutions in the Netherlands. Three researchers analysed the transcripts of the interviews. Results A three-step scheme emerged from the data. Feedback as part of WBA is of greater benefit to trainees if: (i) observation and feedback are planned by the trainee and trainer; (ii) the content and delivery of the feedback are adequate, and (iii) the trainee uses the feedback to guide his or her learning by linking it to learning goals. Negative emotions reported by almost all trainees in relation to observation and feedback led to different responses. Some trainees avoided observation, whereas others overcame their apprehension and actively sought observation and feedback. Active trainers were able to help trainees overcome their fears. Four types of trainer–trainee pairs were distinguished according to their engagement in observation and feedback. External requirements set by training institutions may stimulate inactive trainers and trainees. Conclusions In line with the literature, our results emphasise the importance of the content of feedback and the way it is provided, as well as the importance of its incorporation in trainees’ learning. Moreover, we highlight the step before the actual feedback itself. The way arrangements for feedback are made appears to be important to feedback in formative WBA. Finally, we outline several factors that influence the success or failure of feedback but precede the process of observation and feedback.

Item does not contain fulltext We reviewed the literature on instruments for work-based assessment in single clinical encounters, such as the mini-clinical evaluation exercise (mini-CEX), and examined differences between these instruments in characteristics and feasibility, reliability, validity and educational effect. A PubMed search of the literature published before 8 January 2009 yielded 39 articles dealing with 18 different assessment instruments. One researcher extracted data on the characteristics of the instruments and two researchers extracted data on feasibility, reliability, validity and educational effect. Instruments are predominantly formative. Feasibility is generally deemed good and assessor training occurs sparsely but is considered crucial for successful implementation. Acceptable reliability can be achieved with 10 encounters. The validity of many instruments is not investigated, but the validity of the mini-CEX and the 'clinical evaluation exercise' is supported by strong and significant correlations with other valid assessment instruments. The evidence from the few studies on educational effects is not very convincing. The reports on clinical assessment instruments for single work-based encounters are generally positive, but supporting evidence is sparse. Feasibility of instruments seems to be good and reliability requires a minimum of 10 encounters, but no clear conclusions emerge on other aspects. Studies on assessor and learner training and studies examining effects beyond 'happiness data' are badly needed.

Contains fulltext : 88541.pdf (Publisher’s version ) (Closed access) BACKGROUND: The Nijmegen Professionalism Scale, an instrument for assessing professional behaviour of general practitioner (GP) trainees, consists of four domains: professional behaviour towards patients, other professionals, society and oneself. The purpose of the instrument is to provide formative feedback. AIM: The aim of this study was to examine the psychometric properties of the Nijmegen Professionalism Scale. METHODS: Both GP trainers and their GP trainees participated. Factor analysis was conducted for each domain. Factor structures of trainee and trainer groups were compared. Measure of congruence used was Tucker's phi. Cronbach's alpha was used to establish reliability. RESULTS: Factor structures of the instrument used by GP trainers and trainees were similar. Two factors for each domain were found: domain 1, Respecting patient's interests and Professional distance; domain 2, Collaboration skills and Management skills; domain 3, Responsibility and Quality management; and domain 4, Reflection and learning and Dealing with emotions. Congruence measures were substantial (>0.90). Reliability ranged from 0.78 to 0.95. CONCLUSION: This study to validate the instrument represents one further step. To construct a sound validity argument, a much broader range of evidence is required. Nevertheless, this study shows that the Nijmegen Professionalism Scale is a reliable tool for assessing professional behaviour.

Pelgrim, Elisabeth A M Kramer, Anneke W M Comment England Med Educ. 2013 Mar;47(3):225-7. doi: 10.1111/medu.12128.

Several standard-setting methods can be used for authentic performance-based assessment using direct observation. Rational methods (e.g., the Angoff method) use judgments of test items to set a cutscore, whereas empirical methods (e.g., the borderline group or contrasting groups procedure) use judgments of examinees to obtain a cutscore. When comparing standard-setting methods, a critical concern is the precision of the obtained cutscore. With the Angoff method, cutscore precision can be estimated by applying generalizability analysis to the item judgment data. The empirical methods, however, lack a generally applicable procedure to estimate the precision. In this study a method is developed to estimate the cutscore precision for empirical methods by using resampling. The method was illustrated for a regression based standard-setting method applied to two actual datasets. The results show the applicability of the method: The precision estimate obtained with the resampling method was in reasonable agreemen...

Purpose Earlier studies of absolute standard setting procedures for objective structured clinical examinations (OSCEs) show inconsistent results. This study compared a rational and an empirical standard setting procedure. Reliability and credibility were examined first. The impact of a reality check was then established. Methods The OSCE included 16 stations and was taken by trainees in their final year of postgraduate training in general practice and experienced general practitioners. A modified Angoff (independent judgements, no group discussion) with and without a reality check was used as a rational procedure. A method related to the borderline group procedure, the borderline regression (BR) method, was used as an empirical procedure. Reliability was assessed using generalisability theory. Credibility was assessed by comparing pass rates and by relating the passing scores to test difficulty. Results The passing scores were 73AE4% for the Angoff procedure without reality check (Angoff I), 66AE0% for the Angoff procedure with reality check (Angoff II) and 57AE6% for the BR method. The reliabilities (expressed as root mean square errors) were 2AE1% for Angoffs I and II, and 0AE6% for the BR method. The pass rates of the trainees and GPs were 19% and 9% for Angoff I, 66% and 46% for Angoff II, and 95% and 80% for the BR method, respectively. The correlation between test difficulty and passing score was 0AE69 for Angoff I, 0AE88 for Angoff II and 0AE86 for the BR method. Conclusion The BR method provides a more credible and reliable standard for an OSCE than a modified Angoff procedure. A reality check improves the credibility of the Angoff procedure but does not improve its reliability.

PURPOSE: To examine the validity of a written knowledge test of skills for performance on an OSCE in postgraduate training for general practice. METHODS: A randomly-selected sample of 47 trainees in general practice took a knowledge test of skills, a general knowledge test and an OSCE. The OSCE included technical stations and stations including complete patient encounters. Each station was checklist rated and global rated. RESULTS: The knowledge test of skills was better correlated to the OSCE than the general knowledge test. Technical stations were better correlated to the knowledge test of skills than stations including complete patient encounters. For the technical stations the rating system had no influence on the correlation. For the stations including complete patient encounters the checklist rating correlated better to the knowledge test of skills than the global rating. CONCLUSION: The results of this study support the predictive validity of the knowledge test of skills. In postgraduate training for general practice a written knowledge test of skills can be used as an instrument to estimate the level of clinical skills, especially for group evaluation, such as in studies examining the efficacy of a training programme or as a screening instrument for deciding about courses to be offered. This estimation is more accurate when the content of the test matches the skills under study. However, written testing of skills cannot replace direct observation of performance of skills.

Huisartsen in opleiding leren zowel in de praktijk als op het instituut. Maar vaak sluiten deze twee leeromgevingen niet op elkaar aan. Een recent systematisch literatuuronderzoek beschrijft manieren om het leren in de praktijk en op het instituut beter op elkaar aan te laten sluiten.

Item does not contain fulltext Self-regulated learning is essential for professional development and lifelong learning. As self-regulated learning has many inaccuracies, the need to support self-regulated learning has been recommended. Supervisors can provide such support. In a prior study trainees reported on the variation in received supervisor support. This study aims at exploring supervisors' perspectives. The aim is to explore how supervisors experience self-regulated learning of postgraduate general practitioners (GP) trainees and their role in this, and what helps and hinders them in supervising. In a qualitative study using a phenomenological approach, we interviewed 20 supervisors of first- and third-year postgraduate GP trainees. Supervisors recognised trainee activity in self-regulated learning and adapted their coaching style to trainee needs, occasionally causing conflicting emotions. Supervisors' beliefs regarding their role, trainees' role and the usefulness of educational interventions influenced their support. Supervisors experienced a relation between patient safety, self-regulated learning and trainee capability to learn. Supervisor training was helpful to exchange experience and obtain advice. Supervisors found colleagues helpful in sharing supervision tasks or in calibrating judgments of trainees. Busy practice occasionally hindered the supervisory process. In conclusion, supervisors adapt their coaching to trainees' self-regulated learning, sometimes causing conflicting emotions. Patient safety and entrustment are key aspects of the supervisory process. Supervisors' beliefs about their role and trainees' role influence their support. Supervisor training is important to increase awareness of these beliefs and the influence on their behaviour, and to improve the use of educational instruments. The results align with findings from other (medical) education, thereby illustrating its relevance.

Samenvatting In het kader van de evaluatie van de driejarige huisartsopleiding werd onderzocht of de huisartsen-in-opleiding (haio’s) aan het einde van hun opleiding medisch-technische vaardigheden voldoende beheersen.

Item does not contain fulltext Background: Although the literature suggests that reflection has a positive impact on learning, there is a paucity of evidence to support this notion. Aim: We investigated feedback and reflection in relation to the likelihood that feedback will be used to inform action plans. We hypothesised that feedback and reflection present a cumulative sequence (i.e. trainers only pay attention to trainees' reflections when they provided specific feedback) and we hypothesised a supplementary effect of reflection. Method: We analysed copies of assessment forms containing trainees' reflections and trainers' feedback on observed clinical performance. We determined whether the response patterns revealed cumulative sequences in line with the Guttman scale. We further examined the relationship between reflection, feedback and the mean number of specific comments related to an action plan (ANOVA) and we calculated two effect sizes. Results: Both hypotheses were confirmed by the results. The response pattern found showed an almost perfect fit with the Guttman scale (0.99) and reflection seems to have supplementary effect on the variable action plan. Conclusions: Reflection only occurs when a trainer has provided specific feedback; trainees who reflect on their performance are more likely to make use of feedback. These results confirm findings and suggestions reported in the literature.

Background Family Physician (FP) trainees are expected to be provided with high quality training in well organized practice settings. This study examines differences between FP trainers and non-trainers and their practices to see whether there are differences in trainers and non-trainers and in how their practices are organized and their services are delivered. Method 203 practices (88 non-training and 115 training) with 512 FPs (335 non-trainers and 177 trainers) were assessed using the “Visit Instrument Practice organization (VIP)” on 369 items (142 FP-level; 227 Practice level). Analyses (ANOVA, ANCOVA) were conducted for each level by calculating differences between FP trainees and non-trainees and their host practices. Results Trainers scored higher on all but one of the items, and significantly higher on 47 items, of which 13 remained significant after correcting for covariates. Training practices scored higher on all items and significantly higher on 61 items, of which 23 remained significant after correcting for covariates. Trainers (and training practices) provided more diagnostic and therapeutic services, made better use of team skills and scored higher on practice organization, chronic care services and quality management than non-training practices. Trainers reported more job satisfaction and commitment and less job stress than non-trainers. Discussion There are positive differences between FP trainers and non-trainers in both the level and the quality of services provided by their host practices. Training institutions can use this information to promote the advantages of becoming a FP trainer and training practice as well as to improve the quality of training settings for FPs.

Contains fulltext : 118212.pdf (Publisher’s version ) (Open Access) BACKGROUND: Assessment of medical communication performance usually focuses on rating generically applicable, well-defined communication skills. However, in daily practice, communication is determined by (specific) context factors, such as acquaintance with the patient, or the presented problem. Merely valuing the presence of generic skills may not do justice to the doctor's proficiency.Our aim was to perform an exploratory study on how assessment of general practitioner (GP) communication performance changes if context factors are explicitly taken into account. METHODS: We used a mixed method design to explore how ratings would change. A random sample of 40 everyday GP consultations was used to see if previously identified context factors could be observed again. The sample was rated twice using a widely used assessment instrument (the MAAS-Global), first in the standard way and secondly after context factors were explicitly taken into account, by using a context-specific rating protocol to assess communication performance in the workplace. In between first and second rating, the presence of context factors was established. Item score differences were calculated using paired sample t-tests. RESULTS: In 38 out of 40 consultations, context factors prompted application of the context-specific rating protocol. Mean overall score on the 7-point MAAS-Global scale increased from 2.98 in standard to 3.66 in the context-specific rating (p

Item does not contain fulltext OBJECTIVE: Acquiring adequate communication skills is an essential part of general practice (GP) specialty training. In assessing trainee proficiency, the context in which trainees communicate is usually not taken into account. The present paper aims to explore what context factors can be found in regular GP trainee consultations and how these influence their communication performance. METHODS: In a randomly selected sample of 44 videotaped, real-life GP trainee consultations, we searched for context factors previously identified in GP consultations and explored how trainee ratings change if context factors are taken into account. Trainee performance was rated twice using the MAAS-Global, first without and then with incorporating context factors. Item score differences were calculated using a paired samples t-test and effect sizes were computed. RESULTS: All previously identified context factors were again observed in GP trainee consultations. In communication assessment scores, we found a significant difference in 5 out of 13 MAAS-Global items, mostly in a positive direction. The effect size was moderate (0.57). CONCLUSIONS: GP trainee communication is influenced by contextual factors; they seem to adapt to context in a professional way. PRACTICE IMPLICATIONS: GP specialty training needs to focus on a context-specific application of communication skills. Communication raters need to be taught how to incorporate context factors into their assessments.

Contains fulltext : 109243.pdf (Publisher’s version ) (Open Access) ABSTRACT: BACKGROUND: Research has shown that narrative feedback, (self) reflections and a plan to undertake and evaluate improvements are key factors for effective feedback on clinical performance. We investigated the quantity of narrative comments comprising feedback (by trainers), self-reflections (by trainees) and action plans (by trainer and trainee) entered on a mini-CEX form that was modified for use in general practice training and to encourage trainers and trainees to provide narrative comments. In view of the importance of specificity as an indicator of feedback quality, we additionally examined the specificity of the comments. METHOD: We collected and analysed modified mini-CEX forms completed by GP trainers and trainees. Since each trainee has the same trainer for the duration of one year, we used trainer-trainee pairs as the unit of analysis. We determined for all forms the frequency of the different types of narrative comments and rated their specificity on a three-point scale: specific, moderately specific, not specific. Specificity was compared between trainee-trainer pairs. RESULTS: We collected 485 completed modified mini-CEX forms from 54 trainees (mean of 8.8 forms per trainee; range 1-23; SD 5.6). Trainer feedback was more frequently provided than trainee self-reflections, and action plans were very rare. The comments were generally specific, but showed large differences between trainee-trainer pairs. CONCLUSION: The frequency of self-reflection and action plans varied, all comments were generally specific and there were substantial and consistent differences between trainee-trainer pairs in the specificity of comments. We therefore conclude that feedback is not so much determined by the instrument as by the users. Interventions to improve the educational effects of the feedback procedure should therefore focus more on the users than on the instruments.

Contains fulltext : 108146.pdf (Publisher’s version ) (Open Access) ABSTRACT: BACKGROUND: Self-regulation is essential for professional development. It involves monitoring of performance, identifying domains for improvement, undertaking learning activities, applying newly learned knowledge and skills and self-assessing performance. Since self-assessment alone is ineffective in identifying weaknesses, learners should seek external feedback too. Externally regulated educational interventions, like reflection, learning portfolios, assessments and progress meetings, are increasingly used to scaffold self-regulation.The aim of this study is to explore how postgraduate trainees regulate their learning in the workplace, how external regulation promotes self-regulation and which elements facilitate or impede self-regulation and learning. METHODS: In a qualitative study with a phenomenologic approach we interviewed first- and third-year GP trainees from two universities in the Netherlands. Twenty-one verbatim transcripts were coded. Through iterative discussion the researchers agreed on the interpretation of the data and saturation was reached. RESULTS: Trainees used a short and a long self-regulation loop. The short loop took one week at most and was focused on problems that were easy to resolve and needed minor learning activities. The long loop was focused on complex or recurring problems needing multiple and planned longitudinal learning activities. External assessments and formal training affected the long but not the short loop. The supervisor had a facilitating role in both loops. Self-confidence was used to gauge competence.Elements influencing self-regulation were classified into three dimensions: personal (strong motivation to become a good doctor), interpersonal (stimulation from others) and contextual (organizational and educational features). CONCLUSIONS: Trainees did purposefully self-regulate their learning. Learning in the short loop may not be visible to others. Trainees should be encouraged to actively seek and use external feedback in both loops. An important question for further research is which educational interventions might be used to scaffold learning in the short loop. Investing in supervisor quality remains important, since they are close to trainee learning in both loops.

Item does not contain fulltext OBJECTIVE: The aim of this study is to develop gender criteria that can be included in communication skills assessment in medical education. METHODS: A three-round Delphi study was conducted. The invited 59 participants were experts in the field of gender medicine education (n=28) and doctor-patient communication (n=31). Each Delphi round comprised a questionnaire, an analysis, and a feedback report. In the first round, gender experts explored gender themes in doctor-patient communication from which initial gender criteria were defined. The second and third rounds were used to validate the importance and feasibility of gender criteria. Consensus was defined as a 75% panel agreement and a mean of 4 or higher on a 5-point Likert scale. RESULTS: Four gender criteria achieved consensus after the third round. The importance of including the gender criteria in communication skills assessment was rated consistently higher than its feasibility. Gender criteria relating to the patients' perspective, to gathering information and to gender and power were considered the most important. CONCLUSION: Using a Delphi study, we have developed gender criteria for inclusion in communication skills assessment to promote good communication between doctors and patients. PRACTICE IMPLICATIONS: Gender influences medical communication. Incorporating gender in communication skills assessment may be useful to improve the teaching and learning of communication skills. 01 augustus 2012

Item does not contain fulltext OBJECTIVE: In medical education the focus has shifted from gaining knowledge to developing competencies. To effectively monitor performance in practice throughout the entire training, a new approach of assessment is needed. This study aimed to evaluate an instrument that monitors the development of competencies during postgraduate training in the setting of training of general practice: the Competency Assessment List (Compass). METHODS: The distribution of scores, reliability, validity, responsiveness and feasibility of the Compass were evaluated. RESULTS: Scores of the Compass ranged from 1 to 9 on a 10-point scale, showing excellent internal consistency ranging from .89 to .94. Most trainees showed improving ratings during training. Medium to large effect sizes (.31-1.41) were demonstrated when we compared mean scores of three consecutive periods. Content validity of the Compass was supported by the results of a qualitative study using the RAND modified Delphi Method. The feasibility of the Compass was demonstrated. CONCLUSION: The Compass is a competency based instrument that shows in a reliable and valid way trainees' progress towards the standard of performance. PRACTICE IMPLICATIONS: The programmatic approach of the Compass could be applied in other specialties provided that the instrument is tailored to specific needs of that specialism.

Contains fulltext : 97982.pdf (Author’s version postprint ) (Open Access) ABSTRACT: BACKGROUND: Communication is a key competence for health care professionals. Analysis of registrar and GP communication performance in daily practice, however, suggests a suboptimal application of communication skills. The influence of context factors could reveal why communication performance levels, on average, do not appear adequate. The context of daily practice may require different skills or specific ways of handling these skills, whereas communication skills are mostly treated as generic. So far no empirical analysis of the context has been made. Our aim was to identify context factors that could be related to GP communication. METHODS: A purposive sample of real-life videotaped GP consultations was analyzed (N = 17). As a frame of reference we chose the MAAS-Global, a widely used assessment instrument for medical communication. By inductive reasoning, we analyzed the GP behaviour in the consultation leading to poor item scores on the MAAS-Global. In these cases we looked for the presence of an intervening context factor, and how this might explain the actual GP communication behaviour. RESULTS: We reached saturation after having viewed 17 consultations. We identified 19 context factors that could potentially explain the deviation from generic recommendations on communication skills. These context factors can be categorized into doctor-related, patient-related, and consultation-related factors. CONCLUSIONS: Several context factors seem to influence doctor-patient communication, requiring the GP to apply communication skills differently from recommendations on communication. From this study we conclude that there is a need to explicitly account for context factors in the assessment of GP (and GP registrar) communication performance. The next step is to validate our findings.

‘Dokteren doe je niet alleen’ is het congresthema van dit jaar. Daar zijn wij het hartgrondig mee eens. Maar zijn onze jonge dokters wel voldoende voorbereid om hieraan goed vorm te geven? Wij denken dat daarin nog een weg te gaan is.

Contains fulltext : 51707.pdf (Publisher’s version ) (Closed access) OBJECTIVE: To examine the increase in self-perceived clinical competence during a three-year postgraduate training in general practice and to explore the relation between the growth of self-perceived competence and several background variables. DESIGN: Cohort, 1995-1998. SETTING: Three-year Postgraduate Training for General practice in the Netherlands PARTICIPANTS: All Dutch trainees who followed postgraduate training from September 1995 to September 1998 (N=191). INTERVENTION: We asked the trainees at the start and at the end of their postgraduate training to complete a questionnaire, which assessed their self-perceived knowledge, clinical skills and consultations skills. We collected information about potentially influencing background variables. Amongst these were variables such as: age, gender, prior medical experience, the effort someone has spent upon her/his education, insight in weak and strong areas of clinical competence and knowledge and skills levels. MAIN OUTCOME MEASURE: Self-perceived competence. RESULTS: A total of 127 trainees completed both questionnaires (190 at the first administration and 128 at the second one). We found statistically significant growth of self-perceived clinical competence. Self-perceived consultation skills increased more than self-perceived knowledge and clinical skills. The afore mentioned background variables did not relate in any way with the growth of self-perceived clinical competence. CONCLUSION: This study shows that a 3-year postgraduate training in general practice enhances self-perceived clinical competence. However, we still do not know how to explain this improvement. Further study into the theoretical concept of self-assessment in medical education and into the factors contributing to the feeling of being competent, is required.

Contains fulltext : 57666.pdf (Publisher’s version ) (Closed access) PURPOSE: The evidence suggests that a longitudinal training of communication skills embedded in a rich clinical context is most effective. In this study we evaluated the acquisition of communication skills under such conditions. METHODS: In a longitudinal design the communication skills of a randomly selected sample of 25 trainees of a three-year postgraduate training programme for general practice were assessed at the start and at the end of training. Eight videotaped real life consultations were rated per measurement and per trainee, using the MAAS-Global scoring list. The results were compared with each other and with those of a reference group of 94 experienced GPs. RESULTS: The mean score of the MAAS-Global was slightly increased at the end of training (2.4) compared with the start (2.2). No significant difference was found between the final results of the trainees and the reference group. According to the criteria of the rating scale the performance of both trainees and GPs was unsatisfactory. CONCLUSION: The results of this study indicate that communication skills do not improve in a three-year postgraduate training comprising both a rich clinical context and a longitudinal training of communication skills, and that an unsatisfactory level still exists at the end of training. Moreover, GPs do not acquire communication skills during independent practice as they perform comparably to the trainees. Further research into the measurement of communication skills, the teaching procedures, the role of the GP-trainer as a model and the influence of rotations through hospitals and the like, is required.

Item does not contain fulltext BACKGROUND: Postgraduate training in general practice aims to develop clinical competence. However, little is known about its effect on trainees' development of competence. OBJECTIVE: Our aim was to investigate whether 3 years of postgraduate training in general practice leads to a higher level of knowledge than 2 years training while maintaining the same structure and educational objectives. METHODS: This retrospective study had a mixed longitudinal design. Trainees of the Dutch postgraduate training in general practice participated. Knowledge was assessed by written progress testing of knowledge relevant to general practice embedded in real life situations. Test results were collected from 1992 to 1999. The results of trainees who received the 2-year and 3-year curriculum were compared. RESULTS: Both curricula started with the same entrance level and showed the largest acquisition of knowledge at the start and towards the end of training. Both curricula showed stagnation in growth at the end of the training period in which trainees rotate through hospitals, nursing homes and mental health institutions. The level of knowledge at the end of training was significantly higher for the 3-year curriculum. CONCLUSION: This study shows that postgraduate training contributes to an increase in knowledge and that a 3-year programme leads to a higher level of knowledge than a 2-year programme. The stagnation in growth found at the end of rotations through hospitals, nursing homes and mental health institutions questions the impact of these rotations on the development of competence and the efficacy of the training as a whole. Further study is needed to draw more firm conclusions.

Contains fulltext : 138017.pdf (Publisher’s version ) (Open Access) BACKGROUND: The literature provides some insight into the role of feedback givers, but little information about within-trainer factors influencing 'feedback-giving behaviours'. We looked for relationships between characteristics of feedback givers (self-efficacy, task perception, neuroticism, extraversion, agreeableness and conscientiousness) and elements of observation and feedback (frequency, quality of content and consequential impact). METHODS: We developed and tested several hypotheses regarding the characteristics and elements in a cross-sectional digital survey among GP trainers and their trainees in 2011 and 2012. We conducted bivariate analysis using Pearson correlations and performed multiple regression analysis. RESULTS: Sixty-two trainer-trainee couples from three Dutch institutions for postgraduate GP training participated in the study. Trainer scores on 'task perception' and on a scale of the trait 'neuroticism' correlated positively with frequency of feedback and quality of feedback content. Multiple regression analysis supported positive correlations between task perception and frequency of feedback and between neuroticism and quality of feedback content. No other correlations were found. CONCLUSION: This study contributes to the literature on feedback giving by revealing factors that influence feedback-giving behaviour, namely neuroticism and task perception. Trainers whose task perception included facilitation of observation and feedback (task perception) and trainers who were concerned about the safety of their patients during consultations with trainees (neuroticism) engaged more frequently in observation and feedback and gave feedback of higher quality.

Inleiding: Doel van het onderzoek was na te gaan in hoeverre de driejarige vervolgopleiding tot huisarts bijdraagt aan de klinische competentie zoals die door aios zelf ervaren wordt. Daarnaast onderzochten we of we een relatie konden vinden tussen de groei van zelf ervaren competentie en verschillende achtergrondvariabelen.

A variety of assessment instruments have been developed with the aim of improving the use of feedback on observed performance in the workplace [1]. Literature shows that feedback should preferably be specific and clear, and more recently the importance of reflection on feedback has been emphasized as well. Reflection should lead to a dialogue between trainer and trainee, aimed at better processing and translation of feedback into action plans [2]. To investigate how this theory plays out in practice, we analyzed the frequency, specificity and interdependence of reflections, feedback and action plans reported through an instrument for observational assessment.