When data are not missing at random: implications for measuring health conditions in the Behavioral Risk Factor Surveillance System.

Objectives: To examine the effect on estimated levels of health conditions produced from large-scale surveys, when either list-wise respondent deletion or standard demographic item-level imputation is employed. To assess the degree to which further bias reduction results from the inclusion of correlated ancillary variables in the item imputation process.
Design: Large cross-sectional (US level) household survey.
Participants: 218 726 US adults (18 years and older) in the 2006 Behavioral Risk Factor Surveillance System Survey. This survey is the largest US telephone survey conducted by the Centers for Disease Control and Prevention.
Primary and Secondary Outcome Measures: Estimated rates of severe depression among US adults.
Results: The use of list-wise respondent deletion and/or demographic imputation results in the underestimation of severe depression among adults in the USA. List-wise deletion produces underestimates of 9% (8.7% vs 9.5%). Demographic imputation produces underestimates of 7% (8.9% vs 9.5%). Both of these differences are significant at the 0.05 level.
Conclusion: The use of list-wise deletion and/or demographic-only imputation may produce significant distortion in estimating national levels of certain health conditions.