Search

When and how can researchers synthesize survey data with analyses of social media content to study public opinion, and when and how can social media data complement surveys to better inform researchers and policymakers? This paper explores how public opinions might differ between survey and social media platforms in terms of content and audience, focusing on the test case of opinions about autonomous vehicles. The paper first extends previous overviews comparing surveys and social media as measurement tools to include a broader range of survey types, including surveys that result from public deliberation, considering the dialogic characteristics of different social media, and the range of issue publics and marginalized voices that different surveys and social media forums can attract. It then compares findings and implications from analyses of public opinion about autonomous vehicles from traditional surveys, results of public deliberation, and analyses of Reddit posts, applying a newly developed computational text analysis tool. Findings demonstrate that social media analyses can both help researchers learn more about issues that are uncovered by surveys and also uncover opinions from subpopulations with specialized knowledge and unique orientations toward a subject. In light of these findings, we point to future directions on how researchers and policymakers can synthesize survey and social media data, and the corresponding data integration techniques, to study public opinion. [ABSTRACT FROM AUTHOR]

When and how does state indoctrination work? Building upon research on motivated reasoning and family socialization, I argue that only those individuals whose parents have connections to political patronage are subject to state indoctrination because their pro-regime biases transmitted from parents induce higher receptivity prior to government messages. Focusing on political education in China, I conduct a quasi-experimental analysis exploiting the sharp variation in textbook content generated by China's most recent curriculum reform. Results based on a national survey show that the new politics textbooks successfully affected only those individuals whose parents had worked for the government. The finding survives extensive robustness checks and falsification tests. I also consider several alternative explanations of the effects: preference falsification, selective attention, parental indoctrination, and educational quality. This paper not only highlights the role of intergenerational transmission in moderating the effectiveness of state indoctrination but also casts doubt on the actual degree to which regimes can change minds by changing educational content. [ABSTRACT FROM AUTHOR]

Many in the survey research community have expressed concern at the growing popularity of nonprobability surveys. The absence of random selection prompts justified concerns about self-selection producing biased results and means that traditional, design-based estimation is inappropriate. This paper seeks to provide insight into the conditions under which nonprobability surveys can be expected to provide estimates free of selection bias. In fields such as epidemiology and economics that routinely work with observational data, researchers have identified the necessary conditions for unbiased estimation of causal effects when treatments are not assigned randomly. Similar conditions apply to survey estimates when respondents are not randomly selected. Drawing on this body of research, we propose a framework composed of three elements that determine the level of selection bias in survey estimates. In this paper, we first provide a general overview of these components and demonstrate the link between causal inference and survey inference in the probability-based setting. Second, we give simplified examples to demonstrate how each of the components can contribute to bias in survey estimates. Finally, we review current practices in the area of nonprobability data collection and estimation, and specify how these methods relate to the elements identified here. [ABSTRACT FROM AUTHOR]

This paper addresses speeding, that is, "too fast" responses, in web surveys. Relying on the response process model, we argue that very short response times indicate low data quality, stemming from a lack of attention on the part of respondents. To identify speeding, prior research employed case-wise procedures. Using data from nine online surveys, we demonstrate that response behavior of individual respondents varies considerably during a survey. Thus, we use case- and page-wise procedures to capture speeding behavior that taps different, although related, phenomena. Moreover, page-specific speeding measures capture aspects of data quality that traditional quality measures do not cover. Employing both page-specific and case-wise speeding measures, we examine whether removing speeders makes a difference in substantive findings. The evidence indicates that removing "too fast" responses does not alter marginal distributions, irrespective of which speeder-correction technique is employed. Moreover, explanatory models yield, by and large, negligible coefficient differences (on average about one standard error). Only in exceptional cases differences exceed two standard errors. Our findings suggest that speeding primarily adds some random noise to the data and attenuate correlations, if it makes a difference at all. The paper concludes by discussing implications and limitations. [ABSTRACT FROM AUTHOR]

For decades, sample surveys have provided information of value to sponsors in government, academia, business, and the public. That value proposition is threatened now by declining survey participation and the advent of competition from alternative data sources. In this environment, some developments in survey practice include new thinking about how to recruit respondents, new methods for applying communication technology, and new approaches to blending survey and non-survey data. Going forward, survey data may increasingly be one component of information products, formed from various sources, including administrative records and unstructured ("big") data. The papers in this special issue of Public Opinion Quarterly will contribute to discussions on how future surveys should be conducted and their place in a new information order. [ABSTRACT FROM AUTHOR]

In 2012, the US Census Bureau posed a challenge under the America COMPETES Act, an act designed to improve the competitiveness of the United States by investing in innovation through research and development. The Census Bureau contracted Kaggle.com to host and manage a worldwide competition to develop the best statistical model to predict 2010 Census mail return rates. The Census Bureau provided competitors with a block group-level database consisting of housing, demographic, and socioeconomic variables derived from the 2010 Census, five-year American Community Survey estimates, and 2010 Census operational data. The Census Bureau then challenged teams to use these data (and other publicly available data) to construct the models. One goal of the challenge was to leverage winning models as inputs to a new model-based hard-to-count (HTC) score, a metric to stratify and target geographic areas according to propensity to self-respond in sample surveys and censuses. All contest winners employed data-mining and machine-learning techniques to predict mail-return rates. This made the models relatively hard to interpret (when compared with the Census Bureau’s original HTC score) and impossible to directly translate to a new HTC score. Nonetheless, the winning models contained insights toward building a new model-based score using variables from the database. This paper describes the original algorithm-based HTC score, insights gained from the Census Return Rate Challenge, and the model underlying a new HTC score. [ABSTRACT FROM AUTHOR]

Who do legislators best represent? This paper addresses this question by investigating the degree of ideological congruence between senators and constituents on a unified scale. Specifically, I measure congruence between legislators and four constituent subsets--donors, co-partisans, supporters, and registered voters. To estimate the preferences of these groups, I use a large survey of voters and an original survey of campaign contributors that samples both in- and out-of-state contributors in the 2012 election cycle. I find that senators' preferences reflect the preferences of the average donor better than any other group. Senators from both parties are slightly more ideologically extreme than the average co-partisan in their state and those who voted for them in 2012. Finally, senators' preferences diverge dramatically from the preference of the average voter in their state. The degree of divergence is nearly as large as if voters were randomly assigned to a senator. These results show that in the case of the Senate, there is a dearth of congruence between constituents and senators--unless these constituents are those who write checks and attend fund-raisers. [ABSTRACT FROM AUTHOR]

In survey research, a consensus has grown regarding the effectiveness of incentives encouraging survey participation across different survey modes and target populations. Most of this research has been based on surveys from the United States, whereas few studies have provided evidence that these results can be generalized to other contexts. This paper is the first to present comprehensive information concerning the effects of incentives on response rates and nonresponse bias across large-scale surveys in Germany. The context could be viewed as a critical test for incentive effects because Germany's population is among the most survey-critical in the world, with very low response rates. Our results suggest positive incentive effects on response rates and patterns of effects that are similar to those in previous research: The effect increased with the monetary value of the incentive; cash incentives affected response propensity more strongly than lottery tickets do; and prepaid incentives could be more cost effective than conditional incentives. We found mixed results for the effects of incentives on nonresponse bias. Regarding large-scale panel surveys, we could not unequivocally confirm that incentives increased response rates in later panel waves. [ABSTRACT FROM AUTHOR]