How Valid Are the Responses to Nursing Home Survey Questions? Some Issues and Concerns

Purpose: Although surveys are usually piloted before fielding, cognitive-based testing of surveys is not standard practice in nursing home (NH) research. Many terms used in the literature do not have standard definitions and may be interpreted differently by researchers, respondents, and policy makers. The purpose of this study was to ensure that survey respondents understood questions as intended, determine whether the Nursing Home Administrator (NHA) or the Director of Nursing (DON) was better able to answer questions on certain topics, and to inform the answer choices provided for questions. Methods: Using existing survey questions and input from experts, we developed surveys to be administered to DONs and NHAs. Cognitive-based interviews were conducted with 45 participants. We took detailed notes during all interviews, and 2 researchers independently coded these notes for key themes. Results: Many terms and concepts routinely used by NH researchers and policy makers, such as "direct-care workers" and "palliative care," were not uniformly interpreted by those managing NHs. For example, respondents' definitions of direct-care workers ranged from nursing assistants to broader categories of clinical and other staff members, including nurses, activities staff, and social workers. We also found NHAs and DONs, at times, did not possess or have access to information the researchers expected them to. Implications: Our results may help explain discrepant findings across NH studies. They also underscore the necessity of cognitive-based testing for survey development and have important implications for policy decisions.