Search

In this research note, we address the potentials of using interviewer-observed paradata, typically collected during face-to-face-only interviews, in mixed-mode and innovative data collection methods that involve an interviewer at some stage (e.g., during the initial contact or during the interview). To this end, we first provide a systematic overview of the types and purposes of the interviewer-observed paradata most commonly collected in face-to-face interviews—contact form data, interviewer observations, and interviewer evaluations—using the methodology of evidence mapping. Based on selected studies, we illustrate the main purposes of interviewer-observed paradata we identified—including fieldwork management, propensity modeling, nonresponse bias analysis, substantive analysis, and survey data quality assessment. Based on this, we discuss the possible use of interviewer-observed paradata in mixed-mode and innovative data collection methods. We conclude with thoughts on new types of interviewer-observed paradata and the potential of combining paradata from different survey modes.

Während in den quantitativen Sozialwissenschaften Umfragedaten seit jeher das Herzstück der Informationsgewinnung bilden, spielten Beobachtungsdaten und andere Datenquellen eine eher untergeordnete Rolle. Soziale Medien und mobile Endgeräte lassen nun digitale Verhaltensdaten immer mehr in den Mittelpunkt sozialwissenschaftlicher Forschung rücken. Doch selbst die innovativsten und umfangreichsten Datenmengen sind unzureichend, wenn sie nicht von hoher Qualität sind. Dieser Artikel diskutiert anhand eingängiger Beispiele die grundlegenden Herausforderungen bei der Analyse digitaler Verhaltensdaten und präsentiert einen zentralen Ansatz zur Evaluation ihrer Qualität., easy social sciences No. 68 (2023): Diegitale Gesellschaft(en): Neue Forschungsansätze zur Digitalen Transformation Starting Point and Frequency: Year: 1989, Issues per Year: 2, easy social sciences

Response probabilities are used in adaptive and responsive survey designs to guide data collection efforts, often with the goal of diversifying the sample composition. However, if response probabilities are also correlated with measurement error, this approach could introduce bias into survey data. This study analyzes the relationship between response probabilities and data quality in grid questions. Drawing on data from the probability-based GESIS panel, we found low propensity cases to more frequently produce item nonresponse and nondifferentiated answers than high propensity cases. However, this effect was observed only among long-time respondents, not among those who joined more recently. We caution that using adaptive or responsive techniques may increase measurement error while reducing the risk of nonresponse bias., Survey Research Methods, Vol 15 No 1 (2021)

Typically, interviewer training is implemented in order to minimize interviewer effects and ensure that interviewers are well prepared to administer the survey. Leading professional associations in the survey research landscape recommend the standardized implementation of interviewer training. Some large-scale multinational survey programs have produced their own training guidelines to ensure a comparable level of quality in the implementation of training across participating countries. However, the length, content, and methodology of interviewer training guidelines are very heterogeneous. In this paper, we provide a comparative overview of general and study-specific interviewer training guidelines of three multinational survey programs (ESS, PIAAC, SHARE). Using total survey error (TSE) as a conceptual framework, we map the general and study-specific training guidelines of the three multinational survey programs to components of the TSE to determine how they target the reduction of interviewer effects. Our results reveal that unit nonresponse error is covered by all guidelines; measurement error is covered by most guidelines; and coverage error, sampling error, and processing error are addressed either not at all or sparsely. We conclude, for example, that these guidelines could be an excellent starting point for new – small as well as large-scale – surveys to design their interviewer training, and that interviewer training guidelines should be made publicly available in order to provide a high level of transparency, thus enabling survey programs to learn from each other., methods, data, analyses, Vol 14, No 1 (2020)

For many years, web surveys have already been the most frequently used survey mode in Germany and elsewhere (ADM, 2018; ESOMAR, 2018). Moreover, respondents increasingly use mobile devices, especially smartphones (or less often tablets), to access the Internet and participate in surveys. Because of those new developments within the Internet usage landscape, this contribution expands an earlier Survey Guideline on web surveys (Bandilla, 2015) by addressing methodological advantages and disadvantages of mixed-device as well as mobile web surveys. Moreover, it provides best practice advice on the implementation of such surveys in the areas of sampling, questionnaire design, paradata collection, and software solutions. Seit vielen Jahren sind Online-Umfragen der populärste Umfragemodus im In- und Ausland (ADM, 2018; ESOMAR, 2018). Zunehmend benutzen Befragte mobile Endgeräte, insbesondere Smartphones (seltener Tablets), um auf das Internet zuzugreifen und an Befragungen teilzunehmen. Aufgrund dieser neuen Entwicklungen im Nutzungsverhalten erweitert dieser Beitrag eine frühere Guideline für Web-Umfragen (Bandilla, 2015), indem er sich mit den methodischen Vor- und Nachteilen von Mixed-Device-Befragungen und Umfragen auf mobilen Endgeräten befasst. Darüber hinaus behandelt er bewährte Verfahrensweisen zur Durchführung solcher Umfragen in den Bereichen Stichprobenziehung, Fragebogendesign, Paradatenerfassung und Softwarelösungen.

Detailed and precise documentation is the key to scientific research and of particular importance for high-quality data collection. An important parameter for each survey is the response rate. It reflects how many persons initially invited to participate finally took part in a survey interview. In any case, every survey based on a probability sample should report its response rate, since it provides first and easily assessable information about the data collection process. The American Association for Public Opinion Research (AAPOR) has proposed Standard Definitions that include final disposition codes as guidelines for reporting of response rates. These definitions take various modes of data collection into account. However, the final disposition codes proposed by AAPOR are based on sampling and data collection practices in the context of the United States. To date, there is no clear guidance on how to adapt these definitions in a comparable manner for the German context. This survey guideline aims to close this information gap and offers operational definitions for the AAPOR final disposition codes that are applicable in the German context.