Search

The interviewing practice of survey interviewers has long been recognized as an important contributor to measurement error in survey data. In the current article, we compare two approaches that can be used to identify interviewers whose task performance might be inadequate and damaging to data quality. The first approach assesses interviewing behavior through the use of audio-recorded interviews. Behavioral assessments capture actual behavior in an interview, but typically rely only on “slices” of observed behavior. The second approach is based on interview time paradata, a type of “trace” data that can easily be aggregated in summary measurements such as average interview speed at the interviewer level. In the current study, we use data from the Dutch-speaking subsample of interviewers employed in two survey rounds of the European Social Survey in Belgium to evaluate how successful the two above approaches are for predicting interviewers’ contributions to interviewer variance. The results show that interviewers who deviate from a larger number of standardized interviewing practices in one (early) audio-recorded interview, as well as those who tend to accelerate their interviewing speed over the course of an interview, tend to contribute more to interviewer variance. The two types of performance assessments appear to be independent, additive predictors of interviewers’ variance contributions. While statistically significant, the effects are nevertheless modest in size. The implication for practice is that interviewer monitoring would benefit from well-considered combinations of both behavioral and paradata-based assessments.

Survey interviews are conducted to produce objective, accurate information in which interviewers ask questions as worded and their discretionary speech is carefully managed. To limit interviewer influence over answers and reduce between-interviewer variance, Standardized Interviewing (SI) requires interviewers to administer nondirective probes – scripted utterances designed to elicit acceptable answers without leading respondents. To promote the intended interpretation of questions, and thus response accuracy, Conversational Interviewing (CI) authorizes interviewers to clarify questions when they suspect respondents have misunderstood. This article reviews evidence from 12 studies about the effectiveness of these two approaches. Findings consistently show that CI leads to considerably more accurate question interpretation and response accuracy than SI across different samples, modes, and languages, and does not increase interviewer variance. CI generally leads to longer interviews than SI, requiring practitioners to weigh increased response accuracy against longer interview duration. Several online implementations of CI have produced initially promising results. [ABSTRACT FROM AUTHOR]

Abstract Background Involving patients in decisions about their pharmacotherapy is crucial for a satisfactory treatment outcome. Information and opinions about medicines are available from a variety of sources. The Wise List is the drug formulary of recommended essential medicines for the Stockholm healthcare region and is issued by the Drug and Therapeutics Committee (DTC). To inform the public about treatment for common diseases and the concept of recommended medicines, a patient edition of the Wise List was developed. The aim of this study was to explore patients’ knowledge, needs and attitudes to the Wise List, DTC and information about medicines in general. Methods To examine patient knowledge about recommended medicines a survey (n = 312) was carried out at four large primary healthcare centres in Stockholm, Sweden. To further elucidate the patients’ needs of the information on recommended medicines and medicines in general, three focus group discussions (FGDs) were performed. Results Of the respondents 57% did not recognise the Wise List, 26% recognised but did not use it and 17% used it. A total of 63% reported that they search for information about medicines. The most common information source was “asking their doctor” (36%) followed by searching the internet (31%). The FGDs revealed that the patients were not interested in medicines in general, only in the medicines they use themselves. They did not understand the aim of the Wise List or how they could benefit from information about recommended medicines. The patients expressed a wish to access all information they need about their own care as well as public healthcare information at one location. Conclusion The intended aim of the DTC with providing information to the public was not achieved as the patients have difficulties to understand the information and how they should use it. The patients were not interested in medicines in general, they wanted information tailored to their specific needs. The findings highlight the importance of creating tools for patients in collaboration with them and evaluate the concept continuously.

A rich and diverse literature exists on the effects that human interviewers can have on different aspects of the survey data collection process. This research synthesis uses the Total Survey Error (TSE) framework to highlight important historical developments and advances in the study of interviewer effects on a variety of important survey process outcomes, including sample frame coverage, contact and recruitment of potential respondents, survey measurement, and data processing. Included in the scope of the synthesis is research literature that has focused on explaining variability among interviewers in these effects and the different types of variable errors that they can introduce, which can ultimately affect the efficiency of survey estimates. We first consider common tasks with which human interviewers are often charged and then use the TSE framework to organize and synthesize the literature discussing the variable errors that interviewers can introduce when attempting to execute each task. Based on our synthesis, we identify key gaps in knowledge and then use these gaps to motivate an organizing model for future research investigating explanations for interviewer effects on different aspects of the survey data collection process. [ABSTRACT FROM AUTHOR]

This study investigates how an onscreen virtual agent's dialog capability and facial animation affect survey respondents' comprehension and engagement in "face-to-face" interviews, using questions from US government surveys whose results have far-reaching impact on national policies. In the study, 73 laboratory participants were randomly assigned to respond in one of four interviewing conditions, in which the virtual agent had either high or low dialog capability (implemented through Wizard of Oz) and high or low facial animation, based on motion capture from a human interviewer. Respondents, whose faces were visible to the Wizard (and videorecorded) during the interviews, answered 12 questions about housing, employment, and purchases on the basis of fictional scenarios designed to allow measurement of comprehension accuracy, defined as the fit between responses and US government definitions. Respondents answered more accurately with the high-dialog-capability agents, requesting clarification more often particularly for ambiguous scenarios; and they generally treated the high-dialog-capability interviewers more socially, looking at the interviewer more and judging high-dialog-capability agents as more personal and less distant. Greater interviewer facial animation did not affect response accuracy, but it led to more displays of engagement--acknowledgments (verbal and visual) and smiles--and to the virtual interviewer's being rated as less natural. The pattern of results suggests that a virtual agent's dialog capability and facial animation differently affect survey respondents' experience of interviews, behavioral displays, and comprehension, and thus the accuracy of their responses. The pattern of results also suggests design considerations for building survey interviewing agents, which may differ depending on the kinds of survey questions (sensitive or not) that are asked. [ABSTRACT FROM AUTHOR]

Abstract: We conducted an experiment to evaluate the use of embodied survey bots (i.e., software-controlled avatars) as a novel method for automated data collection in 3D virtual worlds. A bot and a human-controlled avatar carried out a survey interview within the virtual world, Second Life, asking participants about their religion. In addition to interviewer agency (bot vs. human), we tested participants’ virtual age, that is, the time passed since the person behind the avatar joined Second Life, as a predictor for response rate and quality. The human interviewer achieved a higher response rate than the bot. Participants with younger avatars were more willing to disclose information about their real life than those with older avatars. Surprisingly, the human interviewer received more negative responses than the bot. Affective reactions of older avatars were also more negative than those of younger avatars. The findings provide support for the utility of bots as virtual research assistants but raise ethical questions that need to be considered carefully. [Copyright &y& Elsevier]

This paper describes ways in which dialect variation may have important effects on the implementation and results of surveys in particular linguistic settings. Specifically, such differences of speech across populations within countries may significantly interfere with survey cooperation and with normal processes of understanding and responding to survey questions. The Swiss context is presented as a case study of how complications in survey implementation due to language and dialect variation may jeopardise data quality. Greater awareness among researchers of the potential impact of dialect variation on survey participation and responding is needed. Also, there is a need for more research internationally on the nature and scope of potential problems due to dialect, and on how to develop targeted strategies and remedies. [ABSTRACT FROM AUTHOR]

This paper suggests a framework for maximizing response in survey research with poor African-American families. Highlighting the affinity between research and direct practice, strategies involving persuasion, credibility, giving, and motivation are discussed. The utility of these ideas for researchers concerned with data gathering in minority communities for policy and practice is addressed. [ABSTRACT FROM AUTHOR]