Search

We investigate panel conditioning effects in a long-running probability-based online panel of the general population through a large-scale experiment conducted in 2020. Our experiment was specifically designed to study the effect of intensifying the surveying frequency for the treatment group (N = 5,598 panel members) during a 16-week corona study while keeping the control group (N = 799 panel members) at the usual bi-monthly surveying frequency. Our results show that panel conditioning is only a minor issue when increasing surveying frequency and only matters for survey questions directly related to the corona study. With this study, we contribute to the general panel conditioning literature, which is still unclear about the conditions under which conditioning effects may occur. In addition, while many panel surveys have conducted add-on studies with increased surveying frequency during the COVID-19 pandemic, ours is among the very few which can provide experimental evidence regarding the potential impact of the add-on study on the underlying panel data infrastructure.

If respondents recognize repeated survey questions and remember their previous responses, this can result in measurement error. Most studies to date that have investigated respondents’ recall of their prior answers have done so in the context of repeated measurements within one cross-sectional survey. The present study extends this research to a longitudinal panel context by investigating whether respondents remember their previous responses to different types of survey questions (beliefs, attitudes, and behaviors) from a previous wave in a probability-based online panel in Germany. We find evidence that some respondents remember their responses from a previous panel wave even after four months, but at a considerably lower rate than previous studies found within a single cross-sectional survey. Respondents who could not remember their response were most commonly off by only a single scale point. Respondents remembered their responses to different types of questions at different rates and were more likely remember an extreme response. Female respondents were more likely to remember their responses, but we find no link to age, education, perceived response burden, survey enjoyment or online panel experience. As respondents could not remember their previous responses in most cases and we find little evidence for a systematic variation of memory effects across groups of respondents, we conclude that the potential for measurement error due to memory effects across panel waves is low after four or more than four months.

A crucial assumption of survey measurements is that respondents carefully perceive, reflect upon, and provide a response to a given question and that this process is independent of respondents’ memory of their responses to previous questions. A violation of this assumption may considerably affect parameter estimations. Despite this possibility, there is little research on when and to what extent such violations occur. To shed light on such memory effects, we investigate the ability of respondents to recall their answers to three types of survey questions (attitudes, behaviors, and beliefs) within one wave of an online panel survey. We find that respondents’ ability to recall their answers is highest for behavior questions and lowest for belief questions, and higher for respondents with extreme responses than respondents with non-extreme responses. We find no difference across respondents with different levels of panel experience and the devices used to complete the survey.

A number of countries in Europe and beyond have established online or mixed mode panels with a web component based upon probability samples. This paper aims to assess the first ever attempt to do this cross-nationally using an input harmonised approach representing a major innovation in cross-national survey methodology. The European Social Survey established a panel using the face-to-face interview as the basis for recruitment to the panel. This experiment was conducted in Estonia, Slovenia and Great Britain using an input harmonised approach to the design throughout something never done before across multiple countries simultaneously. The paper outlines how the experiment was conducted. It then moves on to compare the web panel respondents to the ESS achieved face-to-face sample in each country, as well as comparing the web panel achieved sample to external benchmarks. Most importantly, since the literature is very scarce, the differences in attitudinal and behavioural characteristics are also assessed. By comparing the answers of the total achieved sample in the ESS to the subset who also answered the CRONOS web panel we assess changes in representativeness and substantive answers without confounding the findings with other changes such as mode effects. This approach is only possible where ‘piggybacking’ recruitment has been used. This in itself is rare at the national level but this is the first time survey methodologists have employed this cross-nationally allowing such an analytical approach. Our findings suggest that the CRONOS sample is not too divergent from the target population and to the ESS with the exception of the oldest age groups. However, there are cross-national differences suggesting cross-national comparability might be different when compared to estimates from a face-to-face survey.

The outbreak of the COVID-19 pandemic has a massive impact on society. To curb the spread of the SARS-CoV-2 virus, unprecedented containment measures are being taken by governments around the world. These measures and the fear of the disease itself are likely affecting the economy, social inequality, mental and physical health, and even people’s perception of good democratic governance. Equally unprecedented is the speed at which these massive changes take place and the lack of statistical evidence that accompanies them. Within days of the first containment measures in Germany, the German Internet Panel (GIP) launched the Mannheim Corona Study (MCS), a daily rotating panel study of the general adult population of approximately 3,600 respondents. Its data and reports now inform the crisis cabinet of the German government and are the basis for groundbreaking social and economic research. This paper gives insights into the MCS methodology and data quality.