Search

It is well known among researchers and practitioners that election polls suffer from a variety of sampling and nonsampling errors, often collectively referred to as total survey error. Reported margins of error typically only capture sampling variability, and in particular, generally ignore nonsampling errors in defining the target population (e.g., errors due to uncertainty in who will vote). Here, we empirically analyze 4221 polls for 608 state-level presidential, senatorial, and gubernatorial elections between 1998 and 2014, all of which were conducted during the final three weeks of the campaigns. Comparing to the actual election outcomes, we find that average survey error as measured by root mean square error is approximately 3.5 percentage points, about twice as large as that implied by most reported margins of error. We decompose survey error into election-level bias and variance terms. We find that average absolute election-level bias is about 2 percentage points, indicating that polls for a given election often share a common component of error. This shared error may stem from the fact that polling organizations often face similar difficulties in reaching various subgroups of the population, and that they rely on similar screening rules when estimating who will vote. We also find that average election-level variance is higher than implied by simple random sampling, in part because polling organizations often use complex sampling designs and adjustment procedures. We conclude by discussing how these results help explain polling failures in the 2016 U.S. presidential election, and offer recommendations to improve polling practice. [ABSTRACT FROM AUTHOR]

This paper presents a total survey error model that simultaneously treats sampling error, nonresponse error and measurement error. The main aim for developing the model is to determine the optimal allocation of the available resources for the total survey error reduction. More precisely, the paper is concerned with obtaining the best possible accuracy in survey estimate through an overall economic balance between sampling and nonsampling error.

To estimate the coverage error for web surveys in Europe over time, we analyzed data from the Eurobarometer. The Eurobarometer collects data for the European Community across member and applicant states. Since 2005, the Eurobarometer has contained a straightforward question on Internet access. We compared respondents with and without Internet access and estimated coverage bias for demographic variables (sex, age, length of education) and sociopolitical variables (left-right position on a political scale, life satisfaction). Countries in Europe do differ in Internet penetration and resulting coverage bias. Over time, Internet penetration dramatically increases and coverage bias decreases, but the rate of change differs across countries. In addition, the countries' development significantly affects the pace of these changes. [ABSTRACT FROM AUTHOR]

For more than three decades the National Crime Victimization Survey (NCVS)—and its predecessor the National Crime Survey (NCS)—have been used to calculate estimates of nonfatal crime in the United States. Though the survey has contributed much to our understanding of criminal victimization, some aspects of the survey’s methodology continue to be analyzed (e.g., repeat victimizations, proxy interviews, and bounding). Surprisingly, one important aspect of NCVS methodology has escaped this scrutiny: respondent fatigue. A potential source of nonsampling error, fatigue bias is thought to manifest as respondents become “test wise” after repeated exposure to NCVS survey instruments. Using a special longitudinal NCVS data file, we revisit the presence and influence of respondent fatigue in the NCVS. Specifically, we test the theory that respondents exposed to longer interviews during their first interview are more likely to refuse to participate in the survey 6 months later. Contrary to expectations based on the literature, results show that prior reporting of victimization and exposure to a longer interview is not a significant predictor of a noninterview during the following time-in-sample once relevant individual characteristics are accounted for. Findings do demonstrate significant effects of survey mode and several respondent characteristics on subsequent survey non-participation. [ABSTRACT FROM AUTHOR]

We show how conditional logistic regression can be used to estimate the probability of being enumerated in a census and apply the model to the 1990 Post-Enumeration Survey (PES) in the United States. The estimates can be used in the estimation of population size and the estimation of correlation bias, for example. Unlike the classical stratification approach, the logistic approach permits the use of continuous explanatory variables. Model choice can be based on the standard techniques of the generalized linear models. We discuss some special problems caused by the fact that the PES sample area is open to migration between the captures. We also consider the effect of data errors in estimation. We characterize hard-to-enumerate populations and give some tentative estimates of correlation bias. [ABSTRACT FROM AUTHOR]

In July 1991 the Census Bureau recommended to its parent agency, the Department of Commerce, that the 1990 census be adjusted for undercount. The Secretary of Commerce decided not to adjust, however. Those decisions relied at least partly on the Census Bureau's analyses of the accuracy of the census and of the proposed undercount adjustments based on the Post-Enumeration Survey (PES). Error distributions for the nation, states, and smaller geographic units were estimated with extensions of methods applied to test censuses. To summarize and assess the relative importance of errors in different units, the Census Bureau used aggregate loss functions. This article describes the total error analysis and loss function analysis of the Census Bureau. In its decision not to adjust the census, the Department of Commerce cited different criteria than aggregate loss functions. Those criteria are identified and discussed. [ABSTRACT FROM AUTHOR]

We describe a methodology for estimating the accuracy of dual systems estimates (DSE's) of population, census estimates of population, and estimates of undercount in the census. The DSE's are based on the census and a post-enumeration survey (PES). We apply the methodology to the 1988 dress rehearsal census of St Louis and east-central Missouri and we discuss its applicability to the 1990 census and PES The methodology is based on decompositions of the total (or net) error into components, such as sampling error, matching error, and other nonsampling errors. Limited information about the accuracy of certain components of error, notably failure of assumptions in the "capture--recapture" model, but others as well, lead us to offer tentative estimates of the errors of the census, DSE, and undercount estimates for 1988. Improved estimates are anticipated for 1990. [ABSTRACT FROM AUTHOR]

The loss of precision in estimates of means due to variability among interviewers can be substantial for some questionnaire items and survey designs. The most commonly used methods for estimating the magnitude of this loss are inappropriate for binary items in complex surveys. This article shows how parameters from a model for variance components in binary variables (Anderson and Aitkin 1985) are related to the increased variance of population estimates. It is suggested that one of these parameters, a measure of correlation between interviewer observations on a latent variable, may be more appropriate than the intrainterviewer correlation p for measuring the magnitude of interviewer effects. This is because it is unaffected by the level of the attribute in the population, which is not true of p. Interviewer effects for a household respondent's recorded labor-force status in a random digit dial telephone survey are examined using the new model and estimation process. Small but positive effects are present for at least one of the labor-force categories. [ABSTRACT FROM AUTHOR]

An error model for dual frame survey designs is developed. It includes components of error for sampling variance, interviewer variance, and bias in each frame. A cost model that attempts to capture the complexity of a full scale dual frame survey is presented. The error and cost models are applied to a large national survey, the National Crime Survey, and the effect that alternative levels of bias in both frames have on the optimal allocation of sample to the two frames is examined for two types of crime. [ABSTRACT FROM AUTHOR]