Search

Mail communications remain a prominent aspect of survey research, especially due to the increasing adoption of web-push surveys throughout the world. However, there is a lack of research on the effectiveness of comprehensive communication strategies for increasing response rates. This paper tests the influence of two communication theories—social exchange theory and pre-suasion theory—on response rates in a survey of residents of southern West Virginia. Social exchange theory has long been used as a strategy for encouraging response, but the theory of pre-suasion offers a new approach of tailoring and sequencing written and visual communications. We operationalized pre-suasion and social exchange communications techniques in the cover letter and questionnaire, then conducted a 2-by-2 experimental design to identify differences in response and data quality. The results show significant differences in response based on the communications in the letter, but no differences based on the questionnaire. The versions with social exchange letters received a response rate 5.5 percent higher than pre-suasion letters. Further, communication differences had no statistically significant effect on measures of data quality. These findings suggest future avenues for research on broad communication strategies in mail survey materials. [ABSTRACT FROM AUTHOR]

In recent years, web-push strategies have been developed in cross-sectional mixed-mode surveys to improve response rates and reduce the costs of data collection. However, pushing respondents into the more cost-efficient web mode has rarely been examined in the context of panel surveys. This study evaluates how a web-push intervention affects the willingness of panel members to switch survey modes from mail to web. We tested three web-push strategies in a German probability-based mixed-mode panel by randomly assigning 1,895 panelists of the mail mode to one of three conditions: (1) the web option was offered to panelists concurrently with the paper questionnaire including a promised €10 incentive for completing the survey on the web, (2) the web option was presented sequentially two weeks before sending the paper questionnaire and respondents were also promised an incentive of €10, or (3) same sequential web-first approach as for condition 2, but with a prepaid €10 incentive instead of a promised incentive. The study found that a sequential presentation of the web option significantly increases the web response in a single survey but may not motivate more panelists to switch to the web mode permanently. Contrary to our expectation, offering prepaid incentives neither improves the web response nor the proportion of mode switchers. Overall, all three web-push strategies show the potential to effectively reduce survey costs without causing differences in panel attrition after five consecutive waves. Condition 2, the sequential web-first design combined with a promised incentive was most effective in pushing respondents to switch to the web mode and in reducing costs. [ABSTRACT FROM AUTHOR]

Address-based sampling(ABS) involves drawing probability samples from address frames; in the United States, ABS frames are usually based, in part, on U.S. Postal Service mail delivery databases. Its use has grown rapidly over the past 10 years to the point where it is considered a viable alternative to random-digit-dial dual frame telephone sample designs. Although telephone surveys are well established in the U.S., they have seen declining response rates and the need to sample from both the landline telephone number sampling frame and the cell phone sampling frame. ABS typically involves sampling from a single address frame and opens up the possibility of using various contact and data collection modes including mail, Internet, telephone, and in-person. Mixed modes are often employed in ABS with anywhere from one to multiple phases of contact and/or data collection. The American Association for Public Opinion Research (AAPOR) Council, operating under the auspices of the AAPOR Standards Committee, convened a task force to prepare a comprehensive report on Address-based Sampling. This paper is drawn from the AAPOR report and focuses on sampling, approaches to data collection, and weighting procedures. We also address an important and somewhat challenging aspect of ABS designs for population surveys, the random selection of an adult from the sample household. [ABSTRACT FROM AUTHOR]

Survey researchers have typically assumed that university sponsorship consistently increases response rates and reduces nonresponse error across different populations, but they have not tested the effects of utilizing different university sponsors to collect data from the same population. In addition, scholars have not examined how these effects differ for mixed-mode (web and mail) or mail-only data collection. To explore these questions, we conducted an experiment in spring 2012 with an address-based sample of residents from two states (Washington and Nebraska), using two university sponsors (Washington State University and the University of Nebraska- Lincoln) and two modes (a sequential "web-push" design versus a mail-only design). We found that within-state-sponsored surveys tended to obtain higher response rates than out-of-state-sponsored surveys for both "web-push" and mail-only designs. Our study also investigates the impacts of mode and sponsor on the representativeness of survey estimates. [ABSTRACT FROM AUTHOR]

We report results from two statewide experiments in Washington designed to test potential methods for using postal mail to obtain survey responses over the Internet from address-based samples of general public households. The five methods we test are: 1) sending Web and mail modes of response sequentially; 2) providing a prepaid $5 incentive; 3) offering an instruction card for responding over the Web; 4) sending the follow-up request by Priority Mail; and 5) providing an additional $5 incentive with this follow-up request. Results are evaluated from the standpoint of response rates, demographic representativeness of respondents, and survey costs and data collection times for Web and mail modes. We find that a “Web-plus-mail” design—mailing an initial Web request followed by a mail request—with the prepaid incentive offers much potential for obtaining Internet responses. In addition, the mail follow-up to the initial Web request significantly increases overall response rates and improves respondent representativeness. However, the results also show that a mail-only design consistently obtains higher response rates and a demographically similar sample compared to Web-plus-mail. We also find significant challenges in obtaining a representative sample of respondents with either Web or mail modes and in realizing the potential cost and speed benefits of the Internet in using postal mail contacts. [ABSTRACT FROM PUBLISHER]

We conducted two experiments designed to evaluate several strategies for improving response to Web and Web/mail mixed-mode surveys. Our goal was to determine the best ways to maximize Web response rates in a highly Internet-literate population with full Internet access. We find that providing a simultaneous choice of response modes does not improve response rates (compared to only providing a mail response option). However, offering the different response modes sequentially, in which Web is offered first and a mail follow-up option is used in the final contact, improves Web response rates and is overall equivalent to using only mail. We also show that utilizing a combination of both postal and email contacts and delivering a token cash incentive in advance are both useful methods for improving Web response rates. These experiments illustrate that although different implementation strategies are viable, the most effective strategy is the combined use of multiple response-inducing techniques. [ABSTRACT FROM PUBLISHER]

The article presents a report published by the American Association for Public Opinion Research (AAPOR) which reviews empirical information related to the opt-in online discussion panels used for data collection on public opinion. The Opt-In Online Panel Task Force established by the AAPOR executive council in September 2008 is mentioned. Methods used to recruit online panelists are reviewed. The author comments on the difficulties related to gathering data online in spite of the number of adults in the U.S. that do not use the Internet on a regular basis. Methods used to reduce bias in public opinion research are also discussed.

Previous research has revealed techniques to improve response quality in open-ended questions in both paper and interviewer-administered survey modes. The purpose of this paper is to test the effectiveness of similar techniques in web surveys. Using data from a series of three random sample web surveys of Washington State University undergraduates, we examine the effects of visual and verbal answer-box manipulations (i.e., altering the size of the answer box and including an explanation that answers could exceed the size of the box) and the inclusion of clarifying and motivating introductions in the question stem. We gauge response quality by the amount and type of information contained in responses as well as response time and item nonresponse. The results indicate that increasing the size of the answer box has little effect on early responders to the survey but substantially improved response quality among late responders. Including any sort of explanation or introduction that made response quality and length salient also improved response quality for both early and late responders. In addition to discussing these techniques, we also address the potential of the web survey mode to revitalize the use of open-ended questions in self-administered surveys. [ABSTRACT FROM AUTHOR]

A potential limitation of web-only panels of the general public, even when households are selected using probability methods, is that only about 70 percent of U.S. households have members with Internet access. In addition, some members of Internet-connected households may be unable or unwilling to participate over the web. The Gallup Panel uses both web and mail modes to survey respondents and in 2006 included approximately 50,000 households selected by random-digit dialing. Frequent Internet users were assigned to respond by the web, while others were assigned to participate by mail using a paper questionnaire with a similar visual layout to the web. We use several approaches to determine whether or not the mail option adds value to the results in an otherwise Internet panel and organize our analyses around answering a series of questions. First, does the use of mail allow different types of people to be included? Second, do mail and web respondents give different answers to the same questions? Third, does weighting on and controlling for demographics eliminate any differences in responses from mail and web respondents and indicate that mail is not needed? Finally, do differences exist when responses are collected using an independent mode? In general, the answers to these questions suggest that use of mail adds value to the panel results and improves the overall accuracy of survey results. [ABSTRACT FROM AUTHOR]

Recent experimental research has shown that respondents to forced-choice questions endorse significantly more options than respondents to check-all questions. This research has challenged the common assumption that these two question formats can be used interchangeably but has been limited to comparisons within a single survey mode. In this paper we use data from a 2004 random sample survey of university students to compare the forced-choice and check-all question formats across web self-administered and telephone interviewer-administered surveys as they are commonly used in survey practice. We find that the within-mode question format effects revealed by previous research and reaffirmed in the current study appear to persist across modes as well; the telephone forced-choice format produces higher endorsement than the web check-all format. These results provide further support for the argument that the check-all and forced-choice question formats do not produce comparable results and are not interchangeable formats. Additional comparisons show that the forced-choice format performs similarly across telephone and web modes. [ABSTRACT FROM AUTHOR]

We utilize and apply visual design theory to experimentally test ways to improve the likelihood that web respondents report date answers in a particular format desired by the researcher, thus reducing possible deleterious effects of error messages or requests for corrections. These experiments were embedded in a series of web surveys of random samples of university students. We seek to examine the sequential and cumulative effects of visually manipulating the size and proximity of the answer spaces, the use of symbols instead of words, the verbal language of the question stem, and the graphical location of the symbolic instruction. Our results show that the successive series of visual language manipulations improve respondents' use of the desired format (two digits for the month and four digits for the year) from 45 percent to 96 percent. These results suggest that writing effective questions for web surveys may depend as much or more on the presentation of the answer categories/spaces as the question wording itself. [ABSTRACT FROM AUTHOR]

For survey researchers, it is common practice to use the check-all question format in Web and mail surveys but to convert to the forced-choice question format in telephone surveys. The assumption underlying this practice is that respondents will answer the two formats similarly. In this research note we report results from 16 experimental comparisons in two Web surveys and a paper survey conducted in 2002 and 2003 that test whether the check-all and forced-choice formats produce similar results. In all 16 comparisons, we find that the two question formats do not perform similarly; respondents endorse more options and take longer to answer in the forced-choice format than in the check-all format. These findings suggest that the forced-choice question format encourages deeper processing of response options and, as such, is preferable to the check-all format, which may encourage a weak satisficing response strategy. Additional analyses show that neither acquiescence bias nor item nonresponse seem to pose substantial problems for use of the forced-choice question format in Web surveys. [ABSTRACT FROM AUTHOR]

This article reports results from 14 experimental comparisons designed to test 7 hypotheses about the effects of two types of nonverbal languages (symbols and graphics) on responses to self- administered questionnaires. The experiments were included in a survey of 1,042 university students. Significant differences were observed for most comparisons, providing support for all seven hypotheses. These results support the view that respondents' answers to questions in self- administered surveys are influenced by more than words. Thus, the visual presentation of questions must be taken into consideration when designing such surveys and, especially, when comparing results across surveys in which the visual presentation of questions is varied. [ABSTRACT FROM AUTHOR]

This article presents observations on survey methodology in the early twenty-first century. The century was challenging for survey methodologists. The telephone, which once seemed the heir apparent to face-to-face interview, is no longer an obvious choice for conducting many surveys. It is affected by change in connectedness of telephone instruments from households to individuals and cultural shift whereby people are able to control the telephone rather than vice, versa. The Internet has not served as an immediate source of salvation. It is limited by restrictions on access, an inability to develop sample frames, and response rates that are lower than those achievable by telephone. Other data collection methodologies, from Interactive Voice Response to classroom and home administration on computers, and mixed-mode designs are being proposed as unique solutions for specific survey topics and populations. Random sampling from defined populations is being forgone by some, replaced by modeling of data collected from volunteer panels. Data collection procedures that seemed curiosities have achieved use and permanency in ways that had not anticipated, creating a survey landscape that is quite different.

Review of past E-mail surveys indicates that a methodology to achieve consistently high response rates similar to those that can be obtained by traditional mail has not been developed. In addition, researchers have tended to use E-mail surveys only for populations with universal E-mail access. This study utilizes knowledge from past mail-survey research to develop an E-mail procedure. Further, an experiment is conducted to assess the potential for using a multimode strategy to obtain responses from individuals unreachable through E-mail. The multimode approach proved to be successful and techniques shown to be effective in standard mail surveys were also found to be appropriate for an E-mail survey. [ABSTRACT FROM AUTHOR]

This study examines the effects of two different prominently displayed appeals in combination with two different prominently displayed confidentiality assurances on mail-back completion rates for census questionnaires. The experiment was carried out on a national probability sample of 30,000 household units, using occupant-addressed questionnaires that contained the con- tent of the 1990 decennial census short form. Neither the benefits appeal nor the variation between two confidentiality assurances had a significant effect on completion rates. The mandatory appeal, however, significantly improved completion rates, even when it appeared on the envelope only. [ABSTRACT FROM AUTHOR]

An experimental test was made of whether completion rates for a census questionnaire could be improved by offering the option of calling a toll-free number and providing the requested information to an interviewer as an alternative to mailing it back. Conducted on a national probability sample of households, five treatment panels were created to examine the effect of delivering, in different ways, the invitation to respond by telephone or mail. Offering such invitations did not improve completion rates except when included with a follow-up letter that did not also include a replacement questionnaire. [ABSTRACT FROM AUTHOR]

An experimental study of alternatives to the current U.S. decennial census questionnaire shows that shortening the questionnaire and respondent-friendly questionnaire design improve response, whereas asking a potentially difficult and/or objectionable question, that is, social security number, lowers response. This national study of 17,000 household addresses also demonstrates that relatively high mail survey response can be achieved without addressing correspondence to individual names of residents. [ABSTRACT FROM AUTHOR]

In two experiments, refusal rates to telephone interviews were not affected by substantial changes in the introductory remarks of the interviewer. A prior letter significantly lowered refusal rates in a third experiment. In all three, interviewer sex had no effect. [ABSTRACT FROM AUTHOR]

Efforts to measure trends in citizen preferences have been curtailed by the tendency of public opinion analysts to study ephemeral issues, to depend upon single questions, and to analyze issues individually rather than in relation to competing concerns. An illustrative analysis uses conceptualization of opinions in a value framework and factor analysis to overcome these limitations. [ABSTRACT FROM AUTHOR]

The article presents studies which explored various procedures for increasing mail questionnaire response in large samples of the general public. All mailings in the 1970 study bore postage meter marks rather than stamps. It was reasoned that this was one more sign of depersonalization, and new stamps used while still a novelty might increase response. A two-step experiment was performed in 1971 in which the newly issued multicolored ecology stamps were applied to half the initial mailing. The remainder were metered. Another variation introduced in the 1971 study was inclusion of a replacement questionnaire in the third mailing. The effect was significant. Thirty-two percent responded with completed questionnaires versus 22.4 percent to the third mailing of the 1970 study. The studies exhibited virtually identical results. The percentage dropped because they were deceased, in poor health (usually because of the infirmities of old age), had moved out of state, or had moved with no forwarding address, was 8.0 in 1970 and 7.2 percent in 1971. A most attractive feature of these studies was the low cost, virtually the same in both years. Direct costs of postage, paper, and printing of questionnaires, envelopes, and cover letters amounted to a fraction over 47 cents per potential respondent (or about 68 cents per completed questionnaire). The results hold two important implications. First, large samples of the general public can be stimulated to respond well to rather lengthy mail questionnaires, and the technique should not be categorically dismissed as a data collection device, especially in light of the small costs. Second, efforts to stimulate response may be more productive if attention is focused on the many manipulable aspects of the mail questionnaire process, rather than on single techniques.