Search

The 2000 presidential election focused attention on the problem of unrecorded votes, in which a person casts a ballot but fails to record a valid vote for a particular contest. Although much recent research has evaluated voting technologies and their effects on unrecorded votes, there has been little research on the effects of ballot design. We argue that the same theories used to design arid evaluate self-administered surveys can be used to analyze ballot features. We collect and code paper-based ballots used in the 2002 general election from 250 counties in 5 states. We code the ballots in terms of several graphic design elements, including the content and location of ballot instructions and the layout of candidate names and office titles. Our analysis suggests that several ballot features are associated with unrecorded votes (both overvotes and undervotes) in the gubernatorial contests. We also find that ballot design features exacerbate the racial disparity in unrecorded votes. Ballot design can be an important factor in determining whether voters are able to cast a ballot accurately, which can influence the legitimacy of elections. [ABSTRACT FROM AUTHOR]

This study compares estimates of self-reported substance use and psychological well-being obtained in computer-assisted and paper-and-pencil self-administered surveys. We examine the extent to which mode effects on survey responses are moderated by respondent characteristics and respondent attitudes, including attitudes toward computers, attitudes about confidentiality and privacy, and their general mistrust of others. Respondents age 12-34 were selected through a multistage area probability sample of urban and suburban areas nationally and were randomly assigned to interview mode. There were few main effects of mode on self-reported substance use and well-being. Significant mode-by-age interaction terms revealed that adolescents were more sensitive to mode of administration than older respondents. Adolescents reported significantly higher levels of alcohol use, illicit drug use, and psychological distress in the computer mode than on paper self-administered questionnaires. Significant mode-by-mistrust interactions were also found. Respondents with higher levels of mistrust in others were less likely to report substance use in the computer mode than in the paper-and-pencil format. [ABSTRACT FROM AUTHOR]

When and how can researchers synthesize survey data with analyses of social media content to study public opinion, and when and how can social media data complement surveys to better inform researchers and policymakers? This paper explores how public opinions might differ between survey and social media platforms in terms of content and audience, focusing on the test case of opinions about autonomous vehicles. The paper first extends previous overviews comparing surveys and social media as measurement tools to include a broader range of survey types, including surveys that result from public deliberation, considering the dialogic characteristics of different social media, and the range of issue publics and marginalized voices that different surveys and social media forums can attract. It then compares findings and implications from analyses of public opinion about autonomous vehicles from traditional surveys, results of public deliberation, and analyses of Reddit posts, applying a newly developed computational text analysis tool. Findings demonstrate that social media analyses can both help researchers learn more about issues that are uncovered by surveys and also uncover opinions from subpopulations with specialized knowledge and unique orientations toward a subject. In light of these findings, we point to future directions on how researchers and policymakers can synthesize survey and social media data, and the corresponding data integration techniques, to study public opinion. [ABSTRACT FROM AUTHOR]

When and how does state indoctrination work? Building upon research on motivated reasoning and family socialization, I argue that only those individuals whose parents have connections to political patronage are subject to state indoctrination because their pro-regime biases transmitted from parents induce higher receptivity prior to government messages. Focusing on political education in China, I conduct a quasi-experimental analysis exploiting the sharp variation in textbook content generated by China's most recent curriculum reform. Results based on a national survey show that the new politics textbooks successfully affected only those individuals whose parents had worked for the government. The finding survives extensive robustness checks and falsification tests. I also consider several alternative explanations of the effects: preference falsification, selective attention, parental indoctrination, and educational quality. This paper not only highlights the role of intergenerational transmission in moderating the effectiveness of state indoctrination but also casts doubt on the actual degree to which regimes can change minds by changing educational content. [ABSTRACT FROM AUTHOR]

Many in the survey research community have expressed concern at the growing popularity of nonprobability surveys. The absence of random selection prompts justified concerns about self-selection producing biased results and means that traditional, design-based estimation is inappropriate. This paper seeks to provide insight into the conditions under which nonprobability surveys can be expected to provide estimates free of selection bias. In fields such as epidemiology and economics that routinely work with observational data, researchers have identified the necessary conditions for unbiased estimation of causal effects when treatments are not assigned randomly. Similar conditions apply to survey estimates when respondents are not randomly selected. Drawing on this body of research, we propose a framework composed of three elements that determine the level of selection bias in survey estimates. In this paper, we first provide a general overview of these components and demonstrate the link between causal inference and survey inference in the probability-based setting. Second, we give simplified examples to demonstrate how each of the components can contribute to bias in survey estimates. Finally, we review current practices in the area of nonprobability data collection and estimation, and specify how these methods relate to the elements identified here. [ABSTRACT FROM AUTHOR]

This paper addresses speeding, that is, "too fast" responses, in web surveys. Relying on the response process model, we argue that very short response times indicate low data quality, stemming from a lack of attention on the part of respondents. To identify speeding, prior research employed case-wise procedures. Using data from nine online surveys, we demonstrate that response behavior of individual respondents varies considerably during a survey. Thus, we use case- and page-wise procedures to capture speeding behavior that taps different, although related, phenomena. Moreover, page-specific speeding measures capture aspects of data quality that traditional quality measures do not cover. Employing both page-specific and case-wise speeding measures, we examine whether removing speeders makes a difference in substantive findings. The evidence indicates that removing "too fast" responses does not alter marginal distributions, irrespective of which speeder-correction technique is employed. Moreover, explanatory models yield, by and large, negligible coefficient differences (on average about one standard error). Only in exceptional cases differences exceed two standard errors. Our findings suggest that speeding primarily adds some random noise to the data and attenuate correlations, if it makes a difference at all. The paper concludes by discussing implications and limitations. [ABSTRACT FROM AUTHOR]

One of the benefits of web-based research is the capacity to deliver video content directly to individual survey participants. However, yet to be investigated is whether the participant demographic and attitudinal characteristics are distorted when respondents are asked to view and respond to videos on their personal computers. To address this question, this research paper focuses on two potential types of selection bias introduced into web-based studies with video content: (1) that resulting from respondents’ differential ability to view videos; and (2) that resulting from demographic and motivational influences of adequate exposure to video content. Through a secondary analysis of two studies focusing on adult cigarette smokers living in the United States, this research paper investigates whether the ability to view videos differs between demographic groups, whether respondents who do not view videos for their full duration differ from those who view videos for an appropriate amount of time, and whether video characteristics influenced this relationship. Results show that age, education, sex, household internet access, and work status influence the likelihood of a respondent reporting an inability to view videos. In addition, age, employment status, and congeniality of and interest in issue topic affect adequate video viewing times. These outcomes suggest that although web-based studies offer many advantages, the inclusion of videos within these surveys has the potential to undermine the accuracy of study findings and distort the representative nature of the study sample. Recommendations for minimizing potential distortions are presented. [ABSTRACT FROM PUBLISHER]

For decades, sample surveys have provided information of value to sponsors in government, academia, business, and the public. That value proposition is threatened now by declining survey participation and the advent of competition from alternative data sources. In this environment, some developments in survey practice include new thinking about how to recruit respondents, new methods for applying communication technology, and new approaches to blending survey and non-survey data. Going forward, survey data may increasingly be one component of information products, formed from various sources, including administrative records and unstructured ("big") data. The papers in this special issue of Public Opinion Quarterly will contribute to discussions on how future surveys should be conducted and their place in a new information order. [ABSTRACT FROM AUTHOR]

In 2012, the US Census Bureau posed a challenge under the America COMPETES Act, an act designed to improve the competitiveness of the United States by investing in innovation through research and development. The Census Bureau contracted Kaggle.com to host and manage a worldwide competition to develop the best statistical model to predict 2010 Census mail return rates. The Census Bureau provided competitors with a block group-level database consisting of housing, demographic, and socioeconomic variables derived from the 2010 Census, five-year American Community Survey estimates, and 2010 Census operational data. The Census Bureau then challenged teams to use these data (and other publicly available data) to construct the models. One goal of the challenge was to leverage winning models as inputs to a new model-based hard-to-count (HTC) score, a metric to stratify and target geographic areas according to propensity to self-respond in sample surveys and censuses. All contest winners employed data-mining and machine-learning techniques to predict mail-return rates. This made the models relatively hard to interpret (when compared with the Census Bureau’s original HTC score) and impossible to directly translate to a new HTC score. Nonetheless, the winning models contained insights toward building a new model-based score using variables from the database. This paper describes the original algorithm-based HTC score, insights gained from the Census Return Rate Challenge, and the model underlying a new HTC score. [ABSTRACT FROM AUTHOR]

Due to the inherent sensitivity of many survey questions, a number of researchers have adopted an indirect questioning technique known as the list experiment (or the item-count technique) in order to reduce dishonest or evasive responses. However, standard practice with the list experiment requires a large sample size, utilizes only a difference-in-means estimator, and does not provide a measure of the sensitive item for each respondent. This paper addresses all of these issues. First, the paper presents design principles for the standard list experiment (and the double list experiment) for the reduction of bias and variance as well as providing sample-size formulas for the planning of studies. Second, this paper proves that a respondent-level probabilistic measure for the sensitive item can be derived. This provides a basis for diagnostics, improved estimation, and regression analysis. The techniques in this paper are illustrated with a list experiment from the 2008–2009 American National Election Studies (ANES) Panel Study and an adaptation of this experiment. [ABSTRACT FROM PUBLISHER]

In POQ’s early years, survey data appeared only occasionally in articles and a focus on methodology was even less common. Today, by contrast, it is rare to find a POQ article that is not based on survey data, and many social scientists think of the journal as the leading forum for research on survey methodology. This sea change would have surprised the journal's founders and seems far from inevitable. An account of how and why the change occurred must answer at least two questions. First, what led a journal whose mandate was the study of public opinion in its widest sense to publish analyses based almost exclusively on sample surveys? Second, given the concentration on survey research, what led to an emphasis on methodology? This article focuses on the second question by examining the balance between substance and method over the journal's three-quarters of a century. It considers four kinds of evidence: the papers published in POQ; the POQ papers that have been the most influential; the POQ editors; and the authors who have appeared most often in POQ. [ABSTRACT FROM PUBLISHER]

Previous research has revealed techniques to improve response quality in open-ended questions in both paper and interviewer-administered survey modes. The purpose of this paper is to test the effectiveness of similar techniques in web surveys. Using data from a series of three random sample web surveys of Washington State University undergraduates, we examine the effects of visual and verbal answer-box manipulations (i.e., altering the size of the answer box and including an explanation that answers could exceed the size of the box) and the inclusion of clarifying and motivating introductions in the question stem. We gauge response quality by the amount and type of information contained in responses as well as response time and item nonresponse. The results indicate that increasing the size of the answer box has little effect on early responders to the survey but substantially improved response quality among late responders. Including any sort of explanation or introduction that made response quality and length salient also improved response quality for both early and late responders. In addition to discussing these techniques, we also address the potential of the web survey mode to revitalize the use of open-ended questions in self-administered surveys. [ABSTRACT FROM AUTHOR]

This paper investigates the impact of paper and email contacts on web survey response rates. We use six combinations of paper and email prenotifications and reminders to test the impact of mixed-mode contacts. In addition, we use two survey samples that differ in their relationship with the sponsoring institution to test if the impact of contact mode is conditional on relationship between respondents and the survey researchers. Contrary to previous research, we find little differences in response rates across experimental groups. [ABSTRACT FROM AUTHOR]

Abstract: This study examined whether the prevalence of self-reported health risk behaviors among high school students varied by survey setting (school versus home) and mode of administration (paper and pencil versus computer). Students in grades 9 and 11 were assigned randomly to one of four conditions--school paper-and-pencil instrument (PAPI), school computer-assisted self-interview (CASI), home PAPI, and home CASI. During the spring of 2004, 4,506 students completed identically worded questionnaires based on the Youth Risk Behavior Survey questionnaire. Logistic regression analyses controlling for sex, grade, and race/ethnicity revealed that setting was associated significantly with the reporting of 30 of the 55 risk behaviors examined, and mode was associated significantly with the reporting of 7 of the 55 behaviors. For all behaviors with a significant setting main effect, the odds of reporting the behavior were greater among students who completed questionnaires at school than among students who completed questionnaires at home. For all behaviors with a significant mode main effect, PAPI mode students had lower odds of reporting the behavior than CASI mode students. Because social measurement research assumes that higher prevalence estimates are more valid than lower estimates, methodological factors shown to increase estimates, such as setting and mode, should be considered when planning surveys. [ABSTRACT FROM AUTHOR]

Who do legislators best represent? This paper addresses this question by investigating the degree of ideological congruence between senators and constituents on a unified scale. Specifically, I measure congruence between legislators and four constituent subsets--donors, co-partisans, supporters, and registered voters. To estimate the preferences of these groups, I use a large survey of voters and an original survey of campaign contributors that samples both in- and out-of-state contributors in the 2012 election cycle. I find that senators' preferences reflect the preferences of the average donor better than any other group. Senators from both parties are slightly more ideologically extreme than the average co-partisan in their state and those who voted for them in 2012. Finally, senators' preferences diverge dramatically from the preference of the average voter in their state. The degree of divergence is nearly as large as if voters were randomly assigned to a senator. These results show that in the case of the Senate, there is a dearth of congruence between constituents and senators--unless these constituents are those who write checks and attend fund-raisers. [ABSTRACT FROM AUTHOR]

In survey research, a consensus has grown regarding the effectiveness of incentives encouraging survey participation across different survey modes and target populations. Most of this research has been based on surveys from the United States, whereas few studies have provided evidence that these results can be generalized to other contexts. This paper is the first to present comprehensive information concerning the effects of incentives on response rates and nonresponse bias across large-scale surveys in Germany. The context could be viewed as a critical test for incentive effects because Germany's population is among the most survey-critical in the world, with very low response rates. Our results suggest positive incentive effects on response rates and patterns of effects that are similar to those in previous research: The effect increased with the monetary value of the incentive; cash incentives affected response propensity more strongly than lottery tickets do; and prepaid incentives could be more cost effective than conditional incentives. We found mixed results for the effects of incentives on nonresponse bias. Regarding large-scale panel surveys, we could not unequivocally confirm that incentives increased response rates in later panel waves. [ABSTRACT FROM AUTHOR]

The importance of format, graphic layout, and question routing instructions on the quality of survey data has been acknowledged for self-administered survey instruments, but the significance of these factors for questionnaires administered by interviewers has not been recognized equally. This paper examines two studies in which interviewers used different questionnaire designs to administer the same set of survey questions to randomly selected adult respondents in housing units that made up area probability samples of the Detroit metropolitan area. The paper presents empirical evidence of specific effects (questions skipped in error, unprobed answers) directly attributable to the choice of questionnaire design. The analysis shows that questionnaire design choices can either help or hurt the quality of data collected by interviewers. Furthermore, the behaviors of experienced and inexperienced interviewers are affected in similar ways. In other words, interviewing experience does not compensate for format deficits in the design of survey instruments. [ABSTRACT FROM AUTHOR]

This paper describes a five-step procedure for meta- analysis. Especially important was the contacting of authors of prior papers. This was done primarily to improve the accuracy of the coding; it also helped to identify unpublished research and to supply missing information. Application of the five-step procedure to the issue of return postage in mail surveys yielded significantly more papers and produced more definitive conclusions than those derived from traditional reviews. This meta- analysis indicated that business reply postage is seldom cost- effective because first class postage yields an additional 9% return. Business reply rates were lower than for other first class postage in each of the 20 comparisons. [ABSTRACT FROM AUTHOR]

Most research examining partisan media effects uses individual differences in exposure to news sources to predict attitude change. In this paper, we improve upon this approach by using variations in cable news coverage to predict subsequent changes in viewer impressions of the candidates. This approach allows us to examine the distinct effects of in-party and out-party candidate coverage. Content analyses and survey data show that partisan media effects result from coverage of the opposition candidate, and not from coverage of the like-minded candidate. Specifically, during the 2008 presidential election, increased coverage of Obama (McCain) on Fox News (MSNBC) made viewers less favorable toward Obama (McCain). Meanwhile, coverage of McCain (Obama) on Fox News (MSNBC) had minimal effects on viewer impressions. These results suggest that media effects persist even during an era dominated by selective exposure. [ABSTRACT FROM PUBLISHER]

In 2010 and 2011, highly visible national surveys documented frequent failure among the public, especially among Republicans, to acknowledge that Barack Obama was born in the United States. However, different questions yielded strikingly different results. The highest rate of partisan division was generated by a CBS/New York Times closed-ended question that included potentially leading introductory sentences. The smallest partisan gap in apparent misinformation was generated by an ABC News/Washington Post open-ended question that did not ask a follow-up that was needed to gauge public beliefs about whether Mr. Obama was born in the United States. This paper reviews the polls on birtherism and describes an experiment embedded in a nationally representative sample survey testing whether methodological features of these two questions might have distorted their results and caused the apparent discrepancy between them. A version of the closed-ended question including the leading introductory sentences yielded a much larger degree of apparent partisan division than did a version of the question without the introductory sentences. Following the open question with another question that clarified people’s beliefs (asking about whether Hawaii was part of the United States at the time of Barack Obama’s birth) did not alter the conclusions supported by that item about the accuracy of public understanding. This study therefore illustrates how different question wording caused different polls to produce notably different results and provides a caution about the use of potentially leading wording. [ABSTRACT FROM PUBLISHER]

Interviewer observations of household characteristics can be useful variables for constructing post-survey nonresponse adjustments, particularly if the observations are good proxies of key variables collected later in the survey. The U.S. Census Bureau and other survey data-collection organizations now systematically design interviewer observations with this purpose in mind. However, because these observations are typically estimates or judgments made by the interviewers, they may be incorrect. The effectiveness of nonresponse adjustments using these observations is hampered by such errors. Thus, knowledge about both household- and interviewer-level factors that affect the accuracy of interviewer observations is important for survey researchers in their efforts to design interviewer observations. To date, only one published study of a face-to-face household survey has attempted to identify these factors. This paper attempts to help fill this gap in knowledge by presenting multilevel models of the accuracy of a housing-unit observation recorded by interviewers in the National Survey of Family Growth (NSFG): whether or not young children under the age of 15 are present in the household. The presence of children in the household is an important correlate of key variables in the NSFG and many other socio-economic surveys, and is related to survey participation in general. The accuracy of this observation was found to be a function of selected household-level factors and interactions between household- and interviewer-level factors. Implications of these results for practice and directions for future research are discussed. [ABSTRACT FROM PUBLISHER]

Fair evaluation of interviewers based on their cooperation rates is complicated in most surveys. Depending on the mode(s) of data collection and the methods used to assign sampled cases to interviewers, different interviewers often work cases that are more or less difficult to interview. Further, in centralized telephone facilities, interviewers are given cases affected by the results of earlier attempts by other interviewers. This paper proposes and evaluates an interviewer performance indicator that attempts to repair these weaknesses and can be used in all data collection modes involving interviewers. For each contact made by an interviewer, the deviation between the outcome of that contact (1 = successful interview, 0 = other) and the predicted probability of a successful interview for that contact (according to auxiliary information and survey paradata) is recorded. Each interviewer then receives a score that is the mean of these deviations across all contacts. This performance indicator gives larger credit to the interviewer who obtains success on difficult cases versus relatively easy cases. The indicator also gives smaller penalties to failures on very difficult cases. We analyze face-to-face and telephone contacts from three different surveys to illustrate the computation of this indicator and its properties. We find that calling-history paradata are the strongest predictors of obtaining interviews in both modes (especially for face-to-face contacts), and produce propensity-adjusted performance indicators that more effectively distinguish among interviewers. [ABSTRACT FROM PUBLISHER]

This article introduces an inexpensive, low-tech Audio-Self-Administered Questionnaire that uses a basic MP3 player (MP3/ASAQ) and compares its performance in collecting data about sensitive attitudes with a number of alternatives, including a face-to-face survey. The paper compares five administration procedures in an experiment conducted in a survey on sensitive caste-related attitudes in rural India. Respondents in the MP3/ASAQ group listened to a prerecorded instrument that presented them with a number of first-person statements made by “respondents like [them],” entered their responses on an answer sheet using simple shapes and logos, and finally placed their form in a bolted ballot box. Like previous studies evaluating self-administration techniques, our study indicates that the MP3/ASAQ significantly increased socially undesirable answers, as compared with an equivalent face-to-face interview. Comparisons with additional administration procedures suggest that when self-administration is combined with the use of earphones the threat of bystander disapproval (as opposed to interviewer disapproval) is reduced by effectively isolating respondents from their social environment. [ABSTRACT FROM PUBLISHER]

As survey researchers have begun exploiting paradata—for example, for the correction of nonresponse bias—the quality of these data has come into question. Inaccurate information is likely to affect the resulting statistics and conclusions drawn from such data. This paper focuses on one type of paradata, observations made by interviewers during the data-collection process, and assesses the quality of these observations by examining their measurement error properties. The analysis uses the UK Census Nonresponse Link Study, which links interviewers’ observations collected on six major UK surveys with Census data. Comparing five interviewer observations with self-reports from the Census, the accuracy of the observations for both respondents and nonrespondents to the surveys is evaluated. A multilevel modeling approach is used to explore under which conditions the interviewers’ observations match the reports on the Census forms, accounting for the clustering of sample members within interviewers and areas. The analysis finds that the overall percent agreement between the observations and the Census is generally high, ranging from 87 to 98 percent. The type of housing structure and the final result code are significantly associated with measurement error. For four of the five observations, there is evidence that the interviewer significantly influences the level of measurement error, even after controlling for household, interviewer, and area characteristics. The results presented here will inform future analyses assessing the quality of interviewers’ observations. [ABSTRACT FROM PUBLISHER]

This paper presents results from six experiments that examine the effect of the position of an item on the screen on the evaluative ratings it receives. The experiments are based on the idea that respondents expect “good” things—those they view positively—to be higher up on the screen than “bad” things. The experiments use items on different topics (Congress and HMOs, a variety of foods, and six physician specialties) and different methods for varying their vertical position on the screen. A meta-analysis of all six experiments demonstrates a small but reliable effect of the item’s screen position on mean ratings of the item; the ratings are significantly more positive when the item appears in a higher position on the screen than when it appears farther down. These results are consistent with the hypothesis that respondents follow the “Up means good” heuristic, using the vertical position of the item as a cue in evaluating it. Respondents seem to rely on heuristics both in interpreting response scales and in forming judgments. [ABSTRACT FROM PUBLISHER]

The measurement of household relationships has a long history at the U.S. Census Bureau, but recent challenges include accurately measuring the number of same-sex couples. In 2010, the American Community Survey estimated that there were 593,000 same-sex households. In the same year, the census estimate for the number of same-sex couples was 902,000 (O’Connell and Feliz 2011). Same-sex couples are classified through two questionnaire items: relationship and sex. In an effort to understand the measurement error that might be causing the estimates to differ, we compared questionnaire design formats and modes and conducted a sex-by-name analysis, focus groups, and cognitive interviews. The paper demonstrates that misreporting of sex rather than relationship was responsible for the overreporting of same-sex couples. [ABSTRACT FROM PUBLISHER]

Yu, Tian, and Tang (2008) proposed two new techniques for asking questions on sensitive topics in population surveys: the triangular model (TM) and the crosswise model (CM). The two models can be used as alternatives to the well-known randomized response technique (RRT) and are meant to overcome some of the drawbacks of the RRT. Although Yu, Tian, and Tang provide a promising theoretical analysis of the proposed models, they did not test them. We therefore provide results from an experimental survey in which the crosswise model was implemented and compared to direct questioning. To our knowledge, this is the first empirical evaluation of the crosswise model. We focused on the crosswise model because it seems better suited than the triangular model to overcome the self-protective “no” bias observed for the RRT. This paper-and-pencil survey on plagiarism was administered to Swiss and German students in university classrooms. Results suggest that the CM is a promising data-collection instrument eliciting more socially undesirable answers than direct questioning. [ABSTRACT FROM PUBLISHER]

For decades, numerous surveys have asked Americans the “Most Important Problem” (MIP) question: “What do you think is the most important problem facing this country today?” Global warming and the environment have rarely been cited by more than a small number of respondents in these surveys in recent years, which might seem to suggest that these have not been the most important issues to Americans. This paper explores the possibility that an additional method of assessing the public's priorities might support a different conclusion. Three experiments embedded in national surveys (two done via the Internet, the other done by telephone) show that when asked the traditional MIP question, respondents rarely mentioned global warming or the environment, but when other respondents were asked to identify the most serious problem that will face the world in the future if nothing is done to stop it, global warming and the environment were the most frequently mentioned problems. Furthermore, a large majority of Americans indicated that they wanted the federal government to devote substantial effort to combating problems that the world will face in the future if nothing is done to stop them. Thus, future surveys might include both versions of the MIP question to more fully document Americans’ priorities. [ABSTRACT FROM PUBLISHER]

Interviewers play a crucial role in gaining cooperation from a sample unit. This paper aims to identify the interviewer characteristics that influence survey cooperation. Of principal interest to survey practitioners are interviewer attributes associated with higher cooperation rates, particularly among sample members whose characteristics are traditionally associated with a lower probability of response. Our data source is unusually rich, in that it contains extensive information on interviewers, including their attitudes and behaviors, which are linked to detailed information on both responding and nonresponding sample units. An important value of the data is that they permit examining a host of as yet unanswered questions about whether some interviewer attributes stimulate cooperation among some respondents but not others. In short, we investigate whether some sample units react favorably to certain interviewer characteristics. A multilevel cross-classified logistic model with random interviewer effects is used to account for clustering of households within interviewers, due to unmeasured interviewer attributes, and for the cross-classification of interviewers within areas. The model allows for statistical interactions between interviewer and household characteristics.We find that interviewer confidence and attitudes toward persuading reluctant respondents play an important role in explaining between-interviewer variation in refusal rates. We also find evidence of interaction effects between the interviewer and householder, for example with respect to gender and educational level, supporting the notion of similarity between interviewers and respondents generating higher cooperation. The results are discussed with respect to potential implications for survey practice and design. [ABSTRACT FROM PUBLISHER]

Survey response rates have been declining over the past several decades, particularly for random-digit-dialing (RDD) telephone surveys (see de Leeuw and de Heer 2002; Steeh 1981). This trend affects research panels such as the Gallup Panel, which uses RDD methodology to recruit its members. If significant improvements in panel recruitment response rates are to be achieved, new approaches must be considered. This paper presents the findings of a mail and telephone mode experiment conducted by the Gallup Panel to analyze the individual and combined effects of incentives, advance letters, and follow-up telephone calls on the panel recruitment response rate. Study results indicate that the mail recruitment approach nets a higher panel response rate, and that the cost-effectiveness of the mail recruitment approach is significantly greater than the telephone recruitment approach. Study results also suggest that the advance letter, incentive, and telephone follow-up conditions all have independent, positive influences on the response rate; and that the groups that receive an advance letter, that receive incentives, and that receive a follow-up telephone call have higher panel recruitment response rates than the control group. [ABSTRACT FROM PUBLISHER]

This paper uses mixed Markov latent class models and data from multiwave national panel surveys to investigate the stability of individual-level party identification in three Anglo-American democracies—the United States, Britain, and Canada. Analyses reveal that partisan attachments exhibit substantial dynamism at the latent variable level in the American, British, and Canadian electorates. Large-scale partisan dynamics are not a recent development; rather, they are present in all of the national panel surveys conducted since the 1950s. In all three countries, a generalized “mover–stayer” model outperforms rival models including a partisan stability model and a “black–white” nonattitudes model that specifies random partisan dynamics. The superiority of generalized mover–stayer models of individual-level party identification comports well with American and British studies that document nonstationary, long memory in macropartisanship. The theoretical perspective provided by party identification updating models is consistent with the mix of durable and flexible partisans found in the United States and elsewhere. [ABSTRACT FROM PUBLISHER]

Audio computer-assisted self-interviewing (audio-CASI or ACASI) has been widely adopted around the world as a method for eliciting more candid responses to sensitive questions in surveys. While few studies have explored the added advantages audio may bring over text-CASI, those that have (e.g., Tourangeau and Smith, 1996, Public Opinion Quarterly 60(2):275-304) found large effects for self administration over interviewer administration, but only modest additional gains from the audio enhancement. In this paper, we explore the use of audio-CASI versus text-CASI in a national survey of fertility-related issues in the United States (the National Survey of Family Growth). In the pretest, male and female respondents were randomly assigned to audio-CASI (n = 299) or text-CASI (n = 312). We compare the distributions of substantive responses between modes and examine a variety of paradata (e.g., keystroke files, time stamps) to examine the use of the CASI instruments. The main study, which interviewed 7,643 women and 4,928 men aged 15-44 in 2002-03, used audio-CASI only, but again we have a variety of paradata and interviewer debriefing items to examine the extent to which subjects made use of the audio enhancements to CASI. Our results indicate that most respondents make Iimited use of the audio features of audio-CASI and accordingly the gains produced by this technology are modest relative to text-CASI. [ABSTRACT FROM AUTHOR]

Strack and Martin (1987) have proposed that although probability samples of general populations are needed for studying attitude content, psychological processes such as context and other response effects can be investigated quite well or even better with homogeneous convenience samples. The present paper shows that, in contrast, when social change is included as an essential variable in a study of a well- known context effect, the process itself is weakened and may even disappear. The distinction between content and process is not always tenable in surveys once we include long-term social change as a variable, as we must as the horizon for survey research extends further and further into the future. [ABSTRACT FROM AUTHOR]

Previous research has shown that mass perceptions about the sizes of minority populations are influenced by sociodemographic, threat, and context variables. This paper extends the analysis to a population group that has thus far received only limited attention, gays and lesbians. Our analysis of a statewide survey of Florida residents in 2002 shows that people in sociodemographic groups associated with low levels of political knowledge were more likely to report higher estimates of the gay population. Threat variables and objective context had relatively little impact, but estimates also were higher among individuals who reported personal contact with gays and lesbians. [ABSTRACT FROM AUTHOR]

This paper examines the effects of question ordering on support for affirmative action (AA) for women and racial minorities. Public opinion surveys show that the public expresses greater support for gender-targeted AA than for race-targeted AA, but no research has addressed the extent to which expressed support for one group influences expressed support toward the other. We examine this question by testing the effects of question order on reported support for race and gender targets. Using a split-ballot experimental design, national data show that presenting gender-targeted AA before race-targeted AA increases reported support for the racial target, and conversely, presenting race targeted AA first decreases reported support for gender-targeted AA. The order effects imply that race still has a significant impact on these considerations. The study finds exceptions to this rule among blacks and liberals, whose support for AA programs is unaffected by the order in which the programs are considered. [ABSTRACT FROM AUTHOR]

Traditional statistical analyses of interviewer effects on survey data do not examine whether these effects change over a field period. However. the nature of the survey interview is dynamic. Interviewers' behaviors and perceptions may evolve as they gain experience, thus potentially affecting data quality. This paper looks at how interview length and interviewer evaluations of respondents change over interviewers' workloads. Multilevel models with random interviewer effects are used to account for the clustering of cases within interviewers and individual interviewer characteristics in the 1984, 1988, and 2000 National Election Studies. The 1984 and 1988 NES released sample in four replicates, minimizing the confound between order in an interviewers' workload and sample composition. We find that over the course of the studies, both measures change significantly. Interviewer prior survey experience also was significantly negatively related to the length of the interview. These findings have implications for interviewer training prior to and during studies, as well as suggesting future research to reveal why these behaviors and perceptions change. [ABSTRACT FROM AUTHOR]

For survey researchers, it is common practice to use the check-all question format in Web and mail surveys but to convert to the forced-choice question format in telephone surveys. The assumption underlying this practice is that respondents will answer the two formats similarly. In this research note we report results from 16 experimental comparisons in two Web surveys and a paper survey conducted in 2002 and 2003 that test whether the check-all and forced-choice formats produce similar results. In all 16 comparisons, we find that the two question formats do not perform similarly; respondents endorse more options and take longer to answer in the forced-choice format than in the check-all format. These findings suggest that the forced-choice question format encourages deeper processing of response options and, as such, is preferable to the check-all format, which may encourage a weak satisficing response strategy. Additional analyses show that neither acquiescence bias nor item nonresponse seem to pose substantial problems for use of the forced-choice question format in Web surveys. [ABSTRACT FROM AUTHOR]

A probability sample of households frequently must be translated into a sample of persons representing characteristics of the adult population. Researchers have developed at least 14 within-household respondent selection procedures or their variants to accomplish this task. Although probability methods are preferable, they often increase nonresponse. Quasi-probability and nonprobability techniques have been devised to increase cooperation and decrease costs, although they sacrifice the advantages of randomness. The purpose of this paper is to analyze the results of 16 studies that have compared within-household respondent selection procedures, with respect to demographic representativeness, cooperation or response rates, and costs. Although there is an increasing need to hold down costs yet maintain quality and accuracy as response rates are falling, little systematic, accessible evidence exists to guide choice of respondent selection method. [ABSTRACT FROM AUTHOR]

This article presents information on sampling in random digit dial surveys. The study proposes a new and simple method of within-household sampling that is minimally intrusive and, thus, should not have a deleterious effect on response rates. The method results in a probability sample of eligible persons within households. The method is especially simple to implement in computer-assisted telephone interview random digit dial surveys. The study focus on sampling one adult from each household as this is the primary application, but we also describe how the method can be extended to sample more than one person per household. This paper provides an introduction to the method and results from one study. Experimental comparisons of this approach with other methods of within-household sampling would be a useful next step. The author has presented the method as a way of sampling one adult from all adults in the household, but it can be applied in any situation in which the screener respondent is eligible by first determining if the screener respondent is sampled. If the screener respondent is not eligible, a variant of the method can be used.

Web surveys are becoming increasingly popular, as reflected in the growing research on web survey methodology (e.g., Couper 2000; Couper, Traugott, and Lamias 2001; Crawford, Couper, and Lamias 2001; Dillman and Bowker 2001; Dillman et al. 1998; Tourangeau, Couper, and Steiger 2001). Despite this spate of recent research, it is still not entirely clear if the techniques used to increase response rates in paper and telephone surveys will directly translate to web surveys. Yet understanding which techniques increase response rates in web surveys is increasingly important. Researchers are faced with decreasing response rates in surveys (Smith 1995; Steeh 1981), as well as increased competition with marketers and spammers on the Internet, for the cooperation of respondents (one recent study predicts that by 2006 e-mail users will receive over 1,400 spam messages per year [Tynan 2002]). Without such knowledge, web surveys may become less useful as a tool for survey research. Previous research on web surveys has focused on such areas as coverage and sampling error, effects of multiple follow-ups, controlling survey access, and survey appearance. Very little research has been conducted on how the process by which members of the sample are contacted affects the probability of response. In our review of the literature, we did not discover any studies examining aspects of the e-mail contact, with the exception of a study by Cook, Heath, and Thompson (2000). The meta-analysis of 68 web surveys by those researchers indicates that personalized contacts have a positive impact on response rates... [ABSTRACT FROM AUTHOR]

This article focuses on the person-level/household-level component of the 1999 Questionnaire Design Experimental Research Survey (QDERS) experiment. The QDERS was a split-sample controlled experiment, using paper-and-pencil questionnaires in a telephone interview with a single household respondent. Due in great measure to its significantly higher yield of employer or union-based health plan coverage, the person-level approach produced more insured persons overall than the household-level approach and, thus, lower estimates of the uninsured. The absence of information about the true characteristics of QDERS household prevents definitive conclusions about data quality differences. The QDERS experiment included a response variance reinterview or approximately five hundred households in each treatment to permit a comparison of the reliability of the data produced by the person-level and household screener interview designs. Item nonresponse in general was trivial, and differences in item nonresponse are for the most part nonexistent. Timing data from a small sample of tape-recorded interviews confirm the increased efficiency of the household-level interview. Interviews evaluated both questionnaire treatments via a debriefing questionnaire.

This paper develops and describes latent class models which can be used to analyze dichotomous questions that include don't know (DK) responses. The models allow the researcher to account for different origins of the DK response; in particular, to account for DKs originating from nonattitudes, from equivocation, from an item whose cut point lies near a respondent's ideal point on the attitude, and from item misunderstanding. The models are illustrated on data involving the trade-off between environmental and economic concerns obtained by mail survey in the Pittsburgh, Pennsylvania, area. A model which includes a modified Guttman scale structure for substantive attitude holders and a separate DK class fits these data very well. In order to show the usefulness of the models, the respondents are assigned to latent classes according to this model and the characteristics of the latent classes are contrasted on attitudinal, behavioral, and demographic variables. [ABSTRACT FROM AUTHOR]

Findings from analyses presented in this paper, using data from a Roper survey, suggest that the role of attitudes toward homosexuals should be at the center of future explorations of the relationship between the media coverage of AIDS and public opinion. While the available data are limited, our analyses raise the possibility that anti-gay attitudes constrain the ability of the media to effectively communicate information about risk factors and how the disease is transmitted. Researchers need to explore the possibility that anti-gay attitudes stand between media information and public knowledge and public opinion. [ABSTRACT FROM AUTHOR]

In this brief paper we contend that the number and composition of individuals identifying with no political party may change substantially when the "Independent" option is omitted from the classic National Election Study (NES) party identification question. Data from two surveys of Vermont residents reveal that when the independent choice is not presented the percentage of self-identified independents drops significantly. Moreover, the relationship between age and partisanship varied depending on whether or not the nonpartisan choice was given. When the independent option was announced, a much higher percentage of younger than older respondents chose the option, but with the nonpartisan choice omitted, more older individuals actually opted to identify as independents. A similar analysis using nationwide survey data from the Roper Organization and NES supports our Vermont conclusions. [ABSTRACT FROM AUTHOR]

Previous research has documented effects of the order in which response choices are offered to respondents using closed-ended survey items, but no theory of the psychological sources of these effects has yet been proposed. This paper offers such a theory drawn from a variety of psychological research. Using data from a split-ballot experiment in the 1984 General Social Survey involving a variant of Kohn's parental values measure, we test some predictions made by the theory about what kind of response order effect would be expected (a primacy effect) and among which respondents it should be strongest (those low in cognitive sophistication). These predictions are confirmed. We also test the "form-resistant correlation" hypothesis. Although correlations between items are altered by changes in response order, the presence and nature of the latent value dimension underlying these responses is essentially unaffected. [ABSTRACT FROM AUTHOR]

Traditionally, as reflected in methodology textbook reviews of the survey nonresponse literature, interviews have been thought to achieve substantially higher response rates than mailed questionnaires. The paper describes a correlational design for assessing the typical response difference between these two forms of data collection, after controls for other factors known to affect survey response. Attention is given to change over time in the response difference between interviews and questionnaires. In the primary data set, the net response due to data collection amounts to some 7.5 percentage points, in the middle (30-70) range of survey response. The analysis quantifies the decline over time in interview response, and shows mailed questionnaires to be free from decline. A second data set reproduces the essential findings. [ABSTRACT FROM AUTHOR]

This article explores how interviewer gender influences responses to marriage-related questions in face-to-face surveys utilizing the 2006 China General Social Survey (CGSS). The specific marriage items examined concern respondent’s views of the happiness of married people and normative views of marriage and ask: (a) whether married men are happier than unmarried men; (b) whether married women are happier than unmarried women; (c) whether getting married is better than being single; and (d) whether staying married is better than getting divorced. Drawing from social desirability theory, this article examines two specific research questions. First, does the gender of the interviewer influence respondents’ normative views of marriage? Second, do the effects of interviewer’s gender on these questions differ for male and female respondents? The results provide mixed support for gender-of-interviewer effects. Consistent with social desirability theory, the findings indicate that female interviewers elicit more pro-marriage and neutral responses than negative responses to the first two items than male interviewers. For the most part, social desirability theory was not supported for the last two items. Gender differences in the susceptibility to gender of interviewer effects were inconsistent across models. [ABSTRACT FROM PUBLISHER]

Recently, scholars have claimed that implicit prejudice, as measured by the Affect Misattribution Procedure (AMP), eroded support for Obama in the 2008 presidential election. In the present research, we reexamine the case for the contention that implicit racial prejudice as measured by the AMP has political consequences. Using three nationally representative data sets and extending the analyses of previous work, we examine associations between AMP scores and electoral behavior, candidate evaluations, and racial policy attitudes in 2008 and 2009. In contrast to previous research, we find little evidence of implicit prejudice effects. Our findings have important implications for those concerned with the political impact of implicit racial prejudice against Blacks and the AMP as a measure of implicit prejudice. [ABSTRACT FROM PUBLISHER]

Recent studies of the U.S. Congress have demonstrated a substantial difference between the level of partisan polarization displayed by legislators’ votes and that shown in citizens’ survey responses about those votes. Perhaps public polarization would increase if citizens were more attentive to political debates in Congress. Using natural variation in citizens’ levels of political information, I show that citizens who are informed about the partisan alignment of issues have a preference distribution similar to that of Congress, even after the sample is reweighted to resemble the entire public in their political, social, and demographic characteristics via matching. In contrast, using a survey experiment, I show that cue and argument treatments only partially reduce the discrepancy between the views expressed by the public and the voting behavior of Congress on the same issues. Both experimental and observational studies have significant limitations for measuring counterfactuals involving public opinion, and so our understanding of the polarization gap remains unfortunately limited. [ABSTRACT FROM PUBLISHER]

Much has been learned about why some citizens are tolerant and others are not. However, the subfield has not reached agreement as to how these attitudes ought to be measured, and at least three competing measurement approaches currently exist: (1) the fixed-group approach, championed by the General Social Survey; (2) the “least-liked” group approach, invented by Sullivan, Piereson, and Marcus (1982); and (3) an approach focusing not on groups but on support for restrictive public policies (e.g., Davis 2007). Although a few measurement studies have been published, generally research in this subfield has shown little concern about how different dependent variables overlap (or do not overlap). Measurement issues thus stand as an important impediment to advancing our understanding of the causes and consequences of political intolerance. This article considers the connections between political intolerance and general support for civil liberties. Most generally, I discover that different measures seem to be measuring somewhat different constructs, and that researchers therefore should not assume that different aspects of tolerance and support for civil liberties are interchangeable. These findings have important substantive implications for democratic theory: general support for civil liberties seems to be a policy preference legitimately contested within democratic politics, whereas political intolerance is a form of exceptionalism in which the political freedoms of particular groups and ideologies are targeted. [ABSTRACT FROM PUBLISHER]