Search

To understand expertise and expertise development, interactions between knowledge, cognitive processing and task characteristics must be examined in people at different levels of training, experience, and performance. Interviewing is widely used in the initial exploration of domain expertise. Work and cognitive task analysis chart the knowledge, skills, and strategies experts employ to perform effectively in representative tasks. Interviewing may also shed light on the learning processes involved in acquiring and maintaining expertise and the way experts deal with critical incidents. Interviews may focus on specific tasks, events, scenarios, and examples, but they do not directly tap the representations involved in task performance. Methods that collect verbal protocols during and immediately after task performance better probe the ongoing processes in representing problems and accomplishing tasks. This article provides practical guidelines and examples to help researchers to prepare, conduct, analyse, and report expertise studies using interviews and verbal protocols that are derived from thinking aloud, dialogues or group discussions, free recall, explanation, and retrospective reports. In a multi-method approach, these methods and other techniques need to be combined to fully grasp the nature of expertise. This article shows how the cognitive processes in data collection constrain data quality and highlights how research questions guide the development of coding schemes that enable meaningful interpretation of the rich data obtained. It focuses on professional expertise and provides examples from medicine including visual tasks. This comprehensive review of qualitative research methods aims to contribute to the advancement of expertise.

Purpose: This study aims to investigate quality of feedback as offered by supervisor-assessors with varying levels of assessor expertise following assessment of performance in residency training in a health care setting. It furthermore investigates if and how different levels of assessor expertise influence feedback characteristics. Design/methodology/approach: Experienced (n = 18) and non-experienced (n = 16) supervisor-assessors with different levels of assessor expertise in general practice (GP) watched two videotapes, each presenting a trainee in a "real-life" patient encounter. After watching each videotape, participants documented performance ratings, wrote down narrative feedback comments and verbalized their feedback. Deductive content analysis of feedback protocols was used to explore quality of feedback. Between-group differences were assessed using qualitative-based quantitative analysis of feedback data. Findings: Overall, specificity and usefulness of both written and verbal feedback was limited. Differences in assessor expertise did not seem to affect feedback quality. Research limitations/implications: Results of the study are limited to a specific setting (GP) and assessment context. Further study in other settings and larger sample sizes may contribute to better understanding of the relation between assessor characteristics and feedback quality. Practical implications: Findings suggest that even with supervisor-assessors with varying levels of assessor expertise who are trained in performance assessment and the provision of performance feedback, high-quality feedback is not self-evident; coaching "on the job" of feedback providers and continuous evaluation of feedback processes in performance management systems is crucial. Instruments should facilitate provision of meaningful feedback in writing. Originality/value: The paper investigates quality of feedback immediately following assessment of performance, and links feedback quality to assessor expertise. Findings can contribute to improvement of performance management systems and assessments for developmental purposes. (Contains 2 tables and 2 figures.)

Background: Family physicians perceive that gut feelings, i.e. a 'sense of reassurance' or a 'sense of alarm', play a substantial role in diagnostic reasoning. A measuring instrument is desirable for further research. Our objective is to validate a questionnaire measuring the presence of gut feelings in diagnostic reasoning. Methods: We constructed 16 case vignettes from real practice situations and used the accompanying 'sense of reassurance' or the 'sense of alarm' as reference labels. Based on the results of an initial study (26 family physicians), we divided the case vignettes into a group involving a clear role for the sense of reassurance or the sense of alarm and a group involving an ambiguous role. 49 experienced family physicians evaluated each 10 vignettes using the questionnaire. Construct validity was assessed by testing hypotheses and an internal consistency procedure was performed. Results: As hypothesized we found that the correlations between the reference labels and corresponding items were high for the clear-case vignettes (0.59 - 0.72) and low for the ambiguous-case vignettes (0.08 - 0.23). The agreement between the classification in clear sense of reassurance, clear sense of alarm and ambiguous case vignettes as derived from the initial study and the study population's judgments was substantial (Kappa = 0.62). Factor analysis showed one factor with opposites for sense of reassurance and sense of alarm items. The questionnaire's internal consistency was high (0.91). We provided a linguistic validated English-language text of the questionnaire. Conclusions: The questionnaire appears to be valid. It enables quantitative research into the role of gut feelings and their diagnostic value in family physicians' diagnostic reasoning. [ABSTRACT FROM AUTHOR]