Search

In mixed-mode surveys, mode differences in measurement bias, also called measurement effects or mode effects, continue to pose a problem to survey practitioners. In this paper, we discuss statistical adjustment of measurement bias to the level of a measurement benchmark mode in the context of inference from mixed-mode data. Doing so requires auxiliary information, which we suggest collecting in a re-interview administered to a sub-set of respondents to the first stage of a sequential mixed-mode survey. In the re-interview, relevant questions from the main survey are repeated. After introducing the design and presenting relevant statistical theory, this paper evaluates by Monte Carlo simulation the performance of six candidate estimators that exploit re-interview information. In the simulation parameters are systematically varied that define the size and type of measurement and selection effects between modes in the mixedmode design. Our results indicate that the performance of the estimators strongly depends on the true measurement error model. However, one estimator, called the inverse regression estimator, performs particularly well under all considered scenarios. Our results suggest that the re-interview method is a useful approach to adjust measurement effects in the presence of non-ignorable selectivity between modes in mixed-mode data.

This study applies ordinal confirmatory factor analysis for multiple groups to assess equivalence of scale, random errors and systematic (nonrandom) errors of attitudinal questions surveyed on rating scales under different survey modes (Face-to-Face [F2F], Telephone, Paper, and Web). Empirical findings from a large-scale experiment are presented. Consistent with theoretical expectations, interviewer- and self-administered surveys measured all assessed questions on systematically different scales, with different systematic bias, and with differing extents of random error. These measurement effects were absent when comparing Paper with Web or F2F with Telephone. It is concluded that modes impact primarily systematic measurement effects affecting multiple items equally. Interviewer- and self-administered modes should only be combined with great care in mixed-mode surveys that focus on attitudinal constructs. Combining Paper and Web or Telephone and F2F are the viable options. Thereby choosing the self-administered modes appears more efficient, because these modes exhibited higher indicator reliabilities (smaller random error) than the interviewer modes.