Search

This study compares estimates of self-reported substance use and psychological well-being obtained in computer-assisted and paper-and-pencil self-administered surveys. We examine the extent to which mode effects on survey responses are moderated by respondent characteristics and respondent attitudes, including attitudes toward computers, attitudes about confidentiality and privacy, and their general mistrust of others. Respondents age 12-34 were selected through a multistage area probability sample of urban and suburban areas nationally and were randomly assigned to interview mode. There were few main effects of mode on self-reported substance use and well-being. Significant mode-by-age interaction terms revealed that adolescents were more sensitive to mode of administration than older respondents. Adolescents reported significantly higher levels of alcohol use, illicit drug use, and psychological distress in the computer mode than on paper self-administered questionnaires. Significant mode-by-mistrust interactions were also found. Respondents with higher levels of mistrust in others were less likely to report substance use in the computer mode than in the paper-and-pencil format. [ABSTRACT FROM AUTHOR]

Abstract: This study examined whether the prevalence of self-reported health risk behaviors among high school students varied by survey setting (school versus home) and mode of administration (paper and pencil versus computer). Students in grades 9 and 11 were assigned randomly to one of four conditions--school paper-and-pencil instrument (PAPI), school computer-assisted self-interview (CASI), home PAPI, and home CASI. During the spring of 2004, 4,506 students completed identically worded questionnaires based on the Youth Risk Behavior Survey questionnaire. Logistic regression analyses controlling for sex, grade, and race/ethnicity revealed that setting was associated significantly with the reporting of 30 of the 55 risk behaviors examined, and mode was associated significantly with the reporting of 7 of the 55 behaviors. For all behaviors with a significant setting main effect, the odds of reporting the behavior were greater among students who completed questionnaires at school than among students who completed questionnaires at home. For all behaviors with a significant mode main effect, PAPI mode students had lower odds of reporting the behavior than CASI mode students. Because social measurement research assumes that higher prevalence estimates are more valid than lower estimates, methodological factors shown to increase estimates, such as setting and mode, should be considered when planning surveys. [ABSTRACT FROM AUTHOR]

The importance of format, graphic layout, and question routing instructions on the quality of survey data has been acknowledged for self-administered survey instruments, but the significance of these factors for questionnaires administered by interviewers has not been recognized equally. This paper examines two studies in which interviewers used different questionnaire designs to administer the same set of survey questions to randomly selected adult respondents in housing units that made up area probability samples of the Detroit metropolitan area. The paper presents empirical evidence of specific effects (questions skipped in error, unprobed answers) directly attributable to the choice of questionnaire design. The analysis shows that questionnaire design choices can either help or hurt the quality of data collected by interviewers. Furthermore, the behaviors of experienced and inexperienced interviewers are affected in similar ways. In other words, interviewing experience does not compensate for format deficits in the design of survey instruments. [ABSTRACT FROM AUTHOR]

This paper describes a five-step procedure for meta- analysis. Especially important was the contacting of authors of prior papers. This was done primarily to improve the accuracy of the coding; it also helped to identify unpublished research and to supply missing information. Application of the five-step procedure to the issue of return postage in mail surveys yielded significantly more papers and produced more definitive conclusions than those derived from traditional reviews. This meta- analysis indicated that business reply postage is seldom cost- effective because first class postage yields an additional 9% return. Business reply rates were lower than for other first class postage in each of the 20 comparisons. [ABSTRACT FROM AUTHOR]

The measurement of household relationships has a long history at the U.S. Census Bureau, but recent challenges include accurately measuring the number of same-sex couples. In 2010, the American Community Survey estimated that there were 593,000 same-sex households. In the same year, the census estimate for the number of same-sex couples was 902,000 (O’Connell and Feliz 2011). Same-sex couples are classified through two questionnaire items: relationship and sex. In an effort to understand the measurement error that might be causing the estimates to differ, we compared questionnaire design formats and modes and conducted a sex-by-name analysis, focus groups, and cognitive interviews. The paper demonstrates that misreporting of sex rather than relationship was responsible for the overreporting of same-sex couples. [ABSTRACT FROM PUBLISHER]

For survey researchers, it is common practice to use the check-all question format in Web and mail surveys but to convert to the forced-choice question format in telephone surveys. The assumption underlying this practice is that respondents will answer the two formats similarly. In this research note we report results from 16 experimental comparisons in two Web surveys and a paper survey conducted in 2002 and 2003 that test whether the check-all and forced-choice formats produce similar results. In all 16 comparisons, we find that the two question formats do not perform similarly; respondents endorse more options and take longer to answer in the forced-choice format than in the check-all format. These findings suggest that the forced-choice question format encourages deeper processing of response options and, as such, is preferable to the check-all format, which may encourage a weak satisficing response strategy. Additional analyses show that neither acquiescence bias nor item nonresponse seem to pose substantial problems for use of the forced-choice question format in Web surveys. [ABSTRACT FROM AUTHOR]

This article focuses on the person-level/household-level component of the 1999 Questionnaire Design Experimental Research Survey (QDERS) experiment. The QDERS was a split-sample controlled experiment, using paper-and-pencil questionnaires in a telephone interview with a single household respondent. Due in great measure to its significantly higher yield of employer or union-based health plan coverage, the person-level approach produced more insured persons overall than the household-level approach and, thus, lower estimates of the uninsured. The absence of information about the true characteristics of QDERS household prevents definitive conclusions about data quality differences. The QDERS experiment included a response variance reinterview or approximately five hundred households in each treatment to permit a comparison of the reliability of the data produced by the person-level and household screener interview designs. Item nonresponse in general was trivial, and differences in item nonresponse are for the most part nonexistent. Timing data from a small sample of tape-recorded interviews confirm the increased efficiency of the household-level interview. Interviews evaluated both questionnaire treatments via a debriefing questionnaire.

Traditionally, as reflected in methodology textbook reviews of the survey nonresponse literature, interviews have been thought to achieve substantially higher response rates than mailed questionnaires. The paper describes a correlational design for assessing the typical response difference between these two forms of data collection, after controls for other factors known to affect survey response. Attention is given to change over time in the response difference between interviews and questionnaires. In the primary data set, the net response due to data collection amounts to some 7.5 percentage points, in the middle (30-70) range of survey response. The analysis quantifies the decline over time in interview response, and shows mailed questionnaires to be free from decline. A second data set reproduces the essential findings. [ABSTRACT FROM AUTHOR]