Search

Learning effects due to repeated interviewing, also known as panel conditioning, are a major threat to response quality in later waves of a panel study. To date, research has not provided a clear picture regarding the circumstances, mechanisms, and dimensions of potential panel conditioning effects. In particular, the effects of conditioning frequency, that is, different levels of experience within a panel, on response quality are underexplored. Against this background, we investigated the effects of panel conditioning by using data from the GESIS Panel, a German mixed-mode probability-based panel study. Using two refreshment samples, we compared three panel cohorts with differing levels of experience on several response quality indicators related to the mechanisms of reflection, satisficing, and social desirability. Overall, we find evidence for both negative (i.e., disadvantageous for response quality) and positive (i.e., advantageous for response quality) panel conditioning. Highly experienced respondents were more likely to satisfice by speeding through the questionnaire. They also had a higher probability of refusing to answer sensitive questions than less experienced panel members. However, more experienced respondents were also more likely to optimize the response process by needing less time compared to panelists with lower experience levels (when controlling for speeding). In contrast, we did not find significant differences with respect to the number of “don’t know” responses, nondifferentiation, the selection of first response categories and mid-responses, and the number of nontriggered filter questions. Of the observed differences, speeding showed the highest magnitude with an average increase of 6.0 percentage points for highly experienced panel members compared to low experienced panelists.

Probability sample (PS) surveys are considered the gold standard for population-based inference but face many challenges due to decreasing response rates, relatively small sample sizes, and increasing costs. In contrast, the use of nonprobability sample (NPS) surveys has increased significantly due to their convenience, large sample sizes, and relatively low costs, but they are susceptible to large selection biases and unknown selection mechanisms. Integrating both sample types in a way that exploits their strengths and overcomes their weaknesses is an ongoing area of methodological research. We build on previous work by proposing a method of supplementing PSs with NPSs to improve analytic inference for logistic regression coefficients and potentially reduce survey costs. Specifically, we use a Bayesian framework for inference. Inference relies on a probability survey with a small sample size, and through the prior structure we incorporate supplementary auxiliary information from a less-expensive (but potentially biased) NPS survey fielded in parallel. The performance of several strongly informative priors constructed from the NPS information is evaluated through a simulation study and real-data application. Overall, the proposed priors reduce the mean-squared error (MSE) of regression coefficients or, in the worst case, perform similarly to a weakly informative (baseline) prior that does not utilize any nonprobability information. Potential cost savings (of up to 68 percent) are evident compared to a probability-only sampling design with the same MSE for different informative priors under different sample sizes and cost scenarios. The algorithm, detailed results, and interactive cost analysis are provided through a Shiny web app as guidance for survey practitioners.

Survey researchers frequently use supplementary data sources, such as paradata, administrative data, and contextual data to augment surveys and enhance substantive and methodological research capabilities. While these data sources can be beneficial, integrating them with surveys can give rise to ethical and data privacy issues that have not been completely resolved. In this research synthesis, we review ethical considerations and empirical evidence on how privacy concerns impact participation in studies that collect these novel data sources to supplement surveys. We further discuss potential approaches for safeguarding participants’ data privacy during data collection and dissemination that may assuage their concerns. Finally, we conclude with open questions and suggested avenues for future research.

If respondents recognize repeated survey questions and remember their previous responses, this can result in measurement error. Most studies to date that have investigated respondents’ recall of their prior answers have done so in the context of repeated measurements within one cross-sectional survey. The present study extends this research to a longitudinal panel context by investigating whether respondents remember their previous responses to different types of survey questions (beliefs, attitudes, and behaviors) from a previous wave in a probability-based online panel in Germany. We find evidence that some respondents remember their responses from a previous panel wave even after four months, but at a considerably lower rate than previous studies found within a single cross-sectional survey. Respondents who could not remember their response were most commonly off by only a single scale point. Respondents remembered their responses to different types of questions at different rates and were more likely remember an extreme response. Female respondents were more likely to remember their responses, but we find no link to age, education, perceived response burden, survey enjoyment or online panel experience. As respondents could not remember their previous responses in most cases and we find little evidence for a systematic variation of memory effects across groups of respondents, we conclude that the potential for measurement error due to memory effects across panel waves is low after four or more than four months.

Satisficing response behavior can be a threat to the quality of survey responses. Past research has provided broad empirical evidence on the existence of satisficing and its consequences on data quality, however, relatively little is known about the extent of satisficing over the course of a panel study and its impact on response quality in later waves. Drawing on panel conditioning research, we use question design experiments to investigate whether learning effects across waves of a panel study cause changes in the extent of satisficing and if so, whether general survey experience (process learning) or familiarity with specific question content (content learning) accounts for those changes. We use data from a longitudinal survey experiment comprising six panel waves administered within a German non-probability online access panel. To investigate the underlying mechanism of possible learning effects, the experimental study randomly assigned respondents to different frequencies of receiving identical question content over the six panel waves. Our results show the existence of satisficing in every panel wave, which is in its magnitude similar to the extent of satisficing in the probability-based GESIS Panel that we use as a benchmark study. However, we did not find changes in the extent of satisficing across panel waves, nor did we find moderation effects of the interval between the waves, respondents’ cognitive ability, or motivation. Additional validity analyses showed that satisficing does not only affect the distribution of individual estimates by 15 percent or more but also can have an effect on associations between variables.

Recently, a new workflow has been introduced that allows academic researchers to partner with individuals interested in donating their digital trace data for academic research purposes (Boeschoten, Ausloos, et al., 2022). In this workflow, the digital traces of participants are processed locally on their own devices in such a way that only the subset of participants’ digital trace data that is of legitimate interest to a research project are shared with the researcher, which can only occur after the participant has provided their informed consent. This data donation workflow consists of the following steps: First, the participant requests a digital copy of their personal data at the platform of interest, such as Google, Meta, Twitter and other digital platforms, i.e., their Data Download Package (DDP). Platforms, as data controllers, are required as per the European Union’s General Data Protection Regulation (GDPR) to share a digital copy with each participant requesting such a copy. Second, they download the DDP onto their personal device. Third, by means of local processing, only the data points of interest to the researcher are extracted from that DDP. Fourth, the participant inspects the extracted data points after which the participant can consent to donate. Only after providing this consent, the donated data is sent to a storage location and can be accessed by the researcher, which would mean that the storage location can be accessed for further analysis. In this paper, we introduce Port. Port is a software tool that allows researchers to configure the local processing step of the data donation workflow, allowing the researcher to collect exactly the digital traces needed to answer their research question. When using Port, a researcher can decide: • Which digital platforms are investigated; • Which digital traces are collected; • How the extracted digital traces are visually presented to the participant

This study investigates how an auto-forward design, where respondents navigate through a web survey automatically, affects response times and navigation behavior in a long mixed-device web survey. We embedded an experiment in a health survey administered to the general population in The Netherlands to test the auto-forward design against a manual-forward design. Analyses are based on detailed paradata that keep track of the respondents' behavior in navigating the survey. We find that an auto-forward design decreases completion times and that questions on pages with automated navigation are answered significantly faster compared to questions on pages with manual navigation. However, we also find that respondents use the navigation buttons more in the auto-forward condition compared to the manual-forward condition, largely canceling out the reduction in survey duration. Furthermore, we also find that the answer options 'I don't know' and 'I rather not say' are used just as often in the auto-forward condition as in the manual-forward condition, indicating no differences in satisficing behavior. We conclude that auto-forwarding can be used to reduce completing times, but we also advice to carefully consider mixing manual and auto-forwarding within a survey.

Survey researchers are often confronted with the question of how long to set the length of the field period. Longer fielding time might lead to greater participation yet requires survey managers to devote more of their time to data collection efforts. With the aim of facilitating the decision about the length of the field period, we investigated whether a longer fielding time reduces the risk of nonresponse bias to judge whether field periods can be ended earlier without endangering the performance of the survey. By using data from six waves of a probability-based mixed-mode (online and mail) panel of the German population, we analyzed whether the risk of nonresponse bias decreases over the field period by investigating how day-by-day coefficients of variation develop during the field period. We then determined the optimal cut-off points for each mode after which data collection can be terminated without increasing the risk of nonresponse bias and found that the optimal cut-off points differ by mode. Our study complements prior research by shifting the perspective in the investigation of the risk of nonresponse bias to panel data as well as to mixed-mode surveys, in particular. Our proposed method of using coefficients of variation to assess whether the risk of nonresponse bias decreases significantly with each additional day of fieldwork can aid survey practitioners in finding the optimal field period for their mixed-mode surveys.