Search

The article examines two important aspects of data quality in self-completion surveys of young people, taking advantage of a unique data source: Understanding Society: the United Kingdom Household Longitudinal Study. Young persons aged 10–15 are asked to complete a self-administered paper questionnaire at annual intervals. The number of completed interviews varies over waves from 4,049 to 5,020. Data are also collected from parents, providing important explanatory covariates for our analysis. Stronger parent–child relationship and higher mother’s involvement in education were associated with lower item nonresponse rate and lower inconsistency throughout waves. We also found some evidence for a negative panel conditioning effect with an increase of social desirability bias and measurement errors in the subsequent waves. There was a higher level of inconsistent responses and a higher probability of social desirability bias throughout waves in more sensitive items.

Prospect theory states that the influential power of avoiding negative outcomes is stronger than that of achieving positive outcomes. In a survey context, this theory has been tested with respect to not only participation in a CATI survey, but also giving consent to data linkage in CATI surveys. No study, however, has tested the theory with respect to participation in a CAPI or web survey. This study does so in a mixed-mode panel context; it also tests the moderating effects of time-in-panel, response history, and mode protocol. Results show that the framing of the survey participation request influences participation propensity in a way consistent with prospect theory, but only for relatively recent panel entrants. The opposite effect is found for long-term panel participants. No difference is found between mode protocols.

This article is concerned with the extent to which the propensity to participate in a web face-to-face sequential mixed-mode survey is influenced by the ability to communicate with sample members by e-mail in addition to mail. Researchers may be able to collect e-mail addresses for sample members and to use them subsequently to send survey invitations and reminders. However, there is little evidence regarding the value of doing so. This makes it difficult to decide what efforts should be made to collect such information and how to subsequently use it efficiently. Using evidence from a randomized experiment within a large mixed-mode national survey, we find that using a respondent-supplied e-mail address to send additional survey invites and reminders does not affect survey response rate but is associated with an increased proportion of responses by web rather than face-to-face and, hence, lower survey costs.

Adaptive survey designs can be used to allocate sample elements to alternative data collection protocols in order to achieve a desired balance between some quality measure and survey costs. We compare four alternative methods for allocating sample elements to one of two data collection protocols. The methods differ in terms of the quality measure that they aim to optimize: response rate, R-indicator, coefficient of variation of the participation propensities, or effective sample size. Costs are also compared for a range of sample sizes. The data collection protocols considered are CAPI single-mode and web-CAPI sequential mixed-mode. We use data from a large experiment with random allocation to one of these two protocols. For each allocation method we predict outcomes in terms of several quality measures and costs. Although allocating the whole sample to single-mode CAPI produces a higher response rate than allocating the whole sample to the mixed-mode protocol, we find that two of the targeted allocations achieve a better response rate than single-mode CAPI at a lower cost. We also find that all four of the targeted designs out-perform both single-protocol designs in terms of representativity and effective sample size. For all but the smallest sample sizes, the adaptive designs bring cost savings relative to CAPI-only, though these are fairly modest in magnitude.

Sequential mixed-mode designs are increasingly considered as an alternative to interviewer-administered data collection, allowing researchers to take advantage of the benefits of each mode. We assess the effects of the introduction of a sequential web-face-to-face mixed-mode design over three waves of a longitudinal survey in which members were previously interviewed face-to-face. Findings are reported from a large-scale randomised experiment carried out on the UK Household Longitudinal Study. No differences are found between the mixed-mode design and face-to-face design in terms of cumulative response rates and only minimal differences in terms of sample composition. On the other hand, potential cost savings are evident.

In multi-national surveys, different countries usually implement different sample designs. The sample designs affect the variance of estimates of differences between countries. When making such estimates, analysts often fail to take sufficient account of sample design. This failure occurs sometimes because variables indicating stratification, clustering, or weighting are unavailable, partially available, or in a form that is unsuitable for cross-national analysis. In this article, we demonstrate how complex sample design should be taken into account when estimating differences between countries, and we provide practical guidance to analysts and to data producers on how to deal with partial or inappropriately-coded sample design indicator variables. Using EU-SILC as a case study, we evaluate the inverse misspecification effect (imeff ) that results from ignoring clustering or stratification, or both in a between-country comparison where countries’ sample designs differ. We present imeff for estimates of between-country differences in a number of demographic and economic variables for 19 European Union Member States. We assess the magnitude of imeff and the associated impact on standard error estimates. Our empirical findings illustrate that it is important for data producers to supply appropriate sample design indicators and for analysts to use them.

Previous research into survey invitation letters and prenotification letters has sought to identify optimal features of a standard letter that can be sent to all sample members. In the spirit of adaptive design, this study seeks to establish whether it can be advantageous to target different versions of a letter to different sample subgroups. Specifically, a paragraph intended to heighten perceptions of relevance of the survey is varied among six subgroups in a panel survey. Random allocation to control and treatment groups is crossed with variants of two other design features: time in sample and data collection mode. This enables analysis of the effects of the targeted letter in different survey contexts. The targeted version of the letter improved response rates for two operationally important low-response propensity groups, but only in certain survey design contexts.

Background: Surveys of immigrants face challenges of coverage, representativeness, and response rates. Longitudinal studies of immigrants and ethnic minorities, which have potential to address pressing issues in demographic research, are rare or partial. In the absence of register data, the highest quality approach is argued to be probability sampling using household screening. Objective: To describe the design and implementation of a nationally representative probability sample of immigrants and ethnic minorities in the United Kingdom. Methods: We boosted a nationally representative sample by using small-area census data to identify areas that covered the majority of immigrant and target ethnic minority populations and oversampled addresses from those areas using varying sampling fractions. Households were screened for eligibility based on whether they included a target immigrant/ethnic minority member. If so, all adult members were interviewed. Results: We anticipated the main challenges would be: fewer eligible households than predicted in sampled areas due to geographical mobility; refusal of those screened to provide information on household eligibility; nonparticipation of eligible households. All these issues were found to some degree. We describe how we addressed them and with what success. Conclusions: A careful design and robust fieldwork practices can enable a two-stage probability sampling to achieve good coverage and a much more representative sample of immigrants and ethnic minorities than with more ad hoc methods. The potential research payoffs are substantial. Contribution: We demonstrate the potential for careful two-stage sampling on the back of an existing study for creating a high-quality multi-purpose survey of immigrants.

This special issue is devoted to discussion of probability-based survey panels that collect data either solely or partly through online questionnaires. Panels of this kind have been around for a long time, though they have been few in number, but recent years have seen several new panels start up in Europe. This has led to renewed interest in the methodology of such panels and also to deeper questioning of the role of these panels. On one hand, the probability-based panels are to some extent competing against cheaper non-probability access panels. On the other hand, the probability-based panels are increasingly being seen as possible alternatives to more expensive probability-based survey methods. In both cases, clients and data users want to better understand the relative advantages and disadvantages of the probability-based panels.

Over time, some respondents have been recruited to participate in longitudinal research may stop responding to survey requests. This phenomenon is known as panel attrition and it occurs when a longitudinal panel fails to retain all recruited members. While there are many potential causes of this attrition, there are a number of concrete steps that researchers that manage panels can take to tackle the problem of panel attrition. This chapter provides an overview of the causes and consequences of panel attrition. It also highlights strategies that may be effective in addressing this problem by reviewing innovations and best practices for minimizing panel attrition and also reducing the statistical impacts of attrition when it does occur.

This article describes the application of the total survey error paradigm to longitudinal surveys. Several aspects of survey error, and of the interactions between different types of error, are distinct in the longitudinal survey context. Furthermore, error trade-off decisions in survey design and implementation are subject to some unique considerations. Previous literature on total survey error mostly fails to explicitly consider uniquely longitudinal issues. We aim to show how uniquely longitudinal sources of error in surveys should be understood within the Total Survey Error framework and we provide examples of studies of some of the unique interactions between errors.

Summary This paper discusses the challenges in assessing the effects of mixed modes on measurement. We briefly review theories about why different modes of data collection can lead to differences in survey responses and statistical methods typically used to assess mode effects. We then discuss the challenges, including which mode effects are identified, how to test for mode effects, and whether these would affect substantive conclusions. The issues raised are illustrated with examples from the European Social Survey, which is conducting a programme of experimental research to inform decisions about whether to use mixed modes of data collection. The paper concludes with general implications for mixed modes research. Resume Cet article passe en revue les defis relatifs a l'evaluation de l'effet sur l'erreur mesure, de l'utilisation de plusieurs modes de collecte des donnees. Nous debutons par une revue succincte des theories sur la facon dont differents modes peuvent produire des reponses divergentes pour ensuite traiter des methodes statistiques generalement utilisees afin d'evaluer l'effet du mode de collecte. Ensuite nous discutons les defis, y compris quels effets du mode de collecte sur l'erreur mesure sont identifies, comment tester l'effet du mode et si les differences entre modes impactent sur les conclusions de fond. Les considerations sont illustrees avec des examples de l'Enquete Sociale Europeenne ou un programme de recherche est presentement en cours afin d'evaluer la faisabilite d'utiliser plusieurs modes de collecte des donnees. L'article se termine par des recommandations generales pertinentes pour la poursuite de la recherche sur les modes de collecte multiples.

In this journal, Elliott and Ellingworth (1997)reported their attempts to assess the impact of certain sources of survey error on the British Crime Survey. This article attempts to correct some flaws in their article, to place their results in a wider context, and to provide some further - arguably more robust - estimates of the impact of non-response bias.