Search

In this research note, we address the potentials of using interviewer-observed paradata, typically collected during face-to-face-only interviews, in mixed-mode and innovative data collection methods that involve an interviewer at some stage (e.g., during the initial contact or during the interview). To this end, we first provide a systematic overview of the types and purposes of the interviewer-observed paradata most commonly collected in face-to-face interviews—contact form data, interviewer observations, and interviewer evaluations—using the methodology of evidence mapping. Based on selected studies, we illustrate the main purposes of interviewer-observed paradata we identified—including fieldwork management, propensity modeling, nonresponse bias analysis, substantive analysis, and survey data quality assessment. Based on this, we discuss the possible use of interviewer-observed paradata in mixed-mode and innovative data collection methods. We conclude with thoughts on new types of interviewer-observed paradata and the potential of combining paradata from different survey modes.

Social Networking Sites (SNS) offer survey scientists a relatively new tool to recruit participants, especially among otherwise hard-to-reach populations. Facebook and Instagram, in particular, allow the distribution of advertisements to specific subsets of their users at low cost. Researchers can use such targeted advertisements to guide participants to their online questionnaires. In recent years, an increasing number of studies have shown that this approach can be successfully applied to a range of different target groups. However, a certain familiarity with the tools and mechanisms provided by Meta is necessary to employ this sampling method. Therefore, in this guideline, we will first give a general introduction to sampling via advertisements on Facebook and Instagram before providing detailed instructions on the implementation of such a recruitment campaign. This will be followed by a brief summary of a recent study conducted by GESIS using Meta's platforms to recruit professionals in the German health care sector. Finally, we provide recommendations with respect to the reporting of methodological parameters when using this approach, propose a flowchart to visualize sample sizes at different points during the recruitment process and offer a glossary containing definitions of essential terms researchers are confronted with when using Meta's advertisement interface. Soziale Netzwerkseiten (SNS) stellen eine vergleichsweise neue Möglichkeit dar, Teilnehmende für wissenschaftliche Befragungsprojekte zu rekrutieren. Besonderes Potential hat der Ansatz mit Blick auf die Rekrutierung anderweitig schwer erreichbarer Zielgruppen. So ermöglichen Facebook und Instagram die Schaltung kostengünstiger Werbung für genau definierte Teilgruppen der Nutzenden dieser Netzwerke. Entsprechende Werbunganzeigen können von Forschenden genutzt werden, um ausgewählte Personen zu ihren Onlinefragebögen zu leiten. In den letzten Jahren hat eine wachsende Zahl von Studien gezeigt, dass dieser Ansatz erfolgreich auf eine Reihe unterschiedlicher Zielgruppen angewendet werden kann. Allerdings ist eine gewisse Vertrautheit mit den von Meta bereitgestellten Instrumenten und Mechanismen erforderlich, um dieses Verfahren anzuwenden. Daher bieten wir in dieser Guideline zunächst eine allgemeine Einführung in das Sampling über Werbeanzeigen auf Facebook und Instagram, bevor wir eine detaillierte Anleitung für die Durchführung einer solchen Rekrutierungskampagne geben. Danach folgt eine kurze Zusammenfassung einer kürzlich von GESIS durchgeführten Studie, bei der die Meta-Plattformen zur Rekrutierung von Fachkräften aus dem deutschen Gesundheitswesen genutzt wurden. Abschließend geben wir Empfehlungen dazu, welche Parameter in Publikationen berichtet werden sollten, um die Vergleichbarkeit der Ergebnisse zu gewährleisten. Mit diesem Ziel stellen wir auch ein Flussdiagramm zur Visualisierung der Stichprobengrößen zu verschiedenen Zeitpunkten des Rekrutierungsprozesses zur Verfügung. Schließlich fassen wir in einem Glossar Definitionen wesentlicher Parameter zusammen, mit denen Forscher bei der Verwendung von Metas Werbeschnittstelle konfrontiert werden.

Während in den quantitativen Sozialwissenschaften Umfragedaten seit jeher das Herzstück der Informationsgewinnung bilden, spielten Beobachtungsdaten und andere Datenquellen eine eher untergeordnete Rolle. Soziale Medien und mobile Endgeräte lassen nun digitale Verhaltensdaten immer mehr in den Mittelpunkt sozialwissenschaftlicher Forschung rücken. Doch selbst die innovativsten und umfangreichsten Datenmengen sind unzureichend, wenn sie nicht von hoher Qualität sind. Dieser Artikel diskutiert anhand eingängiger Beispiele die grundlegenden Herausforderungen bei der Analyse digitaler Verhaltensdaten und präsentiert einen zentralen Ansatz zur Evaluation ihrer Qualität., easy social sciences No. 68 (2023): Diegitale Gesellschaft(en): Neue Forschungsansätze zur Digitalen Transformation Starting Point and Frequency: Year: 1989, Issues per Year: 2, easy social sciences

Response probabilities are used in adaptive and responsive survey designs to guide data collection efforts, often with the goal of diversifying the sample composition. However, if response probabilities are also correlated with measurement error, this approach could introduce bias into survey data. This study analyzes the relationship between response probabilities and data quality in grid questions. Drawing on data from the probability-based GESIS panel, we found low propensity cases to more frequently produce item nonresponse and nondifferentiated answers than high propensity cases. However, this effect was observed only among long-time respondents, not among those who joined more recently. We caution that using adaptive or responsive techniques may increase measurement error while reducing the risk of nonresponse bias., Survey Research Methods, Vol 15 No 1 (2021)

Typically, interviewer training is implemented in order to minimize interviewer effects and ensure that interviewers are well prepared to administer the survey. Leading professional associations in the survey research landscape recommend the standardized implementation of interviewer training. Some large-scale multinational survey programs have produced their own training guidelines to ensure a comparable level of quality in the implementation of training across participating countries. However, the length, content, and methodology of interviewer training guidelines are very heterogeneous. In this paper, we provide a comparative overview of general and study-specific interviewer training guidelines of three multinational survey programs (ESS, PIAAC, SHARE). Using total survey error (TSE) as a conceptual framework, we map the general and study-specific training guidelines of the three multinational survey programs to components of the TSE to determine how they target the reduction of interviewer effects. Our results reveal that unit nonresponse error is covered by all guidelines; measurement error is covered by most guidelines; and coverage error, sampling error, and processing error are addressed either not at all or sparsely. We conclude, for example, that these guidelines could be an excellent starting point for new – small as well as large-scale – surveys to design their interviewer training, and that interviewer training guidelines should be made publicly available in order to provide a high level of transparency, thus enabling survey programs to learn from each other., methods, data, analyses, Vol 14, No 1 (2020)

For many years, web surveys have already been the most frequently used survey mode in Germany and elsewhere (ADM, 2018; ESOMAR, 2018). Moreover, respondents increasingly use mobile devices, especially smartphones (or less often tablets), to access the Internet and participate in surveys. Because of those new developments within the Internet usage landscape, this contribution expands an earlier Survey Guideline on web surveys (Bandilla, 2015) by addressing methodological advantages and disadvantages of mixed-device as well as mobile web surveys. Moreover, it provides best practice advice on the implementation of such surveys in the areas of sampling, questionnaire design, paradata collection, and software solutions. Seit vielen Jahren sind Online-Umfragen der populärste Umfragemodus im In- und Ausland (ADM, 2018; ESOMAR, 2018). Zunehmend benutzen Befragte mobile Endgeräte, insbesondere Smartphones (seltener Tablets), um auf das Internet zuzugreifen und an Befragungen teilzunehmen. Aufgrund dieser neuen Entwicklungen im Nutzungsverhalten erweitert dieser Beitrag eine frühere Guideline für Web-Umfragen (Bandilla, 2015), indem er sich mit den methodischen Vor- und Nachteilen von Mixed-Device-Befragungen und Umfragen auf mobilen Endgeräten befasst. Darüber hinaus behandelt er bewährte Verfahrensweisen zur Durchführung solcher Umfragen in den Bereichen Stichprobenziehung, Fragebogendesign, Paradatenerfassung und Softwarelösungen.

Detailed and precise documentation is the key to scientific research and of particular importance for high-quality data collection. An important parameter for each survey is the response rate. It reflects how many persons initially invited to participate finally took part in a survey interview. In any case, every survey based on a probability sample should report its response rate, since it provides first and easily assessable information about the data collection process. The American Association for Public Opinion Research (AAPOR) has proposed Standard Definitions that include final disposition codes as guidelines for reporting of response rates. These definitions take various modes of data collection into account. However, the final disposition codes proposed by AAPOR are based on sampling and data collection practices in the context of the United States. To date, there is no clear guidance on how to adapt these definitions in a comparable manner for the German context. This survey guideline aims to close this information gap and offers operational definitions for the AAPOR final disposition codes that are applicable in the German context.

Documentation of research results is an essential process within the research lifecycle, which includes the steps of study planning and developing the survey instruments, data collection and preparation, data analysis, and data archiving. Primary researchers have to ensure that the collected data and all accompanying materials are properly documented and archived. This enables the scientific community to understand and reproduce the results of a scientific project. The purpose of this survey guideline is to provide a brief introduction and an overview about data preparation and data documentation in order to help primary researchers to make their data and other study-related materials long-term accessible. This overview will therefore help researchers to comply with the principles of reproducibility as a crucial aspect of good scientific practice. This guideline will be useful for researchers who are in the stages of planning a study as well as for those who have already collected data and would like to prepare it for archiving.