Search

In complex probability sample surveys, numerous adjustments are customarily made to the survey weights to reduce potential bias in survey estimates. These adjustments include sampling design (SD) weight adjustments, which account for features of the sampling plan, and non-sampling design (NSD) weight adjustments, which account for non-sampling errors and other effects. Variance estimates prepared from complex survey data customarily account for SD weight adjustments, but rarely account for all NSD weight adjustments. As a result, variance estimates may be biased and standard confidence intervals may not achieve their nominal coverage levels. We describe the implementation of the bootstrap method to account for the SD and NSD weight adjustments for complex survey data. Using data from the National Immunization Survey (NIS), we illustrate the use of the bootstrap (i). for evaluating the use of standard confidence intervals that use Taylor series approximations to variance estimators that do not account for NSD weight adjustments, (ii). for obtaining confidence intervals for ranks estimated from weighted survey data, and (iii). for evaluating the predictive power of logistic regressions using receiver operating characteristic curve analyses that account for the SD and NSD adjustments made to the survey weights., (Copyright 2003 John Wiley & Sons, Ltd.)

Telephone surveys are widely used in the U.S.A. for the study of health-related topics. They are subject to 'coverage bias' because they cannot sample households that do not have telephones. Although only around 5 per cent of households do not have a telephone, rates of telephone coverage show substantial variation by geography, demographic factors and socio-economic factors. In particular, lack of telephone service is more common among households that contain ethnic and racial minorities or that have lower socio-economic status with fewer opportunities for access to medical care and poorer health outcomes. Thus, failure to adequately account for households without telephones in health surveys may yield estimates of health outcomes that are misleading, particularly in states with at least moderate telephone non-coverage. The dynamic nature of the population of households without telephones offers a way of accounting for such households in telephone surveys. At any given time the population of telephone households includes households that have had a break or interruption in telephone service. Empirical results strongly suggest that these households are very similar to households that have never had telephone service. Thus, sampled households that report having had an interruption in telephone service may be used also to represent the portion of the population that has never had telephone service. This strategy can lead to a reduction in non-coverage bias in random-digit-dialling surveys. This paper presents two methods of adjusting for non-coverage of non-telephone households. The effectiveness of these methods is examined using data from the National Health Interview Survey. The interruption-in-telephone-service methods reduce non-coverage bias and can also result in a lower mean squared error. The application of the interruption-in-telephone-service methods to the National Immunization Survey is also discussed. This survey produces estimates for the 50 states and 28 urban areas. The interruption-in-telephone-service estimates tend be slightly lower than estimates resulting from poststratification and from another non-coverage adjustment method. The results suggest that the reduction in bias is greatest for variables that are highly correlated with the presence or absence of telephone service.