Search

INTRODUCTION: Face-to-face surveys are applied frequently when conducting research in older populations. Interviewers play a decisive role in data quality, may affect measurement and influence results. This study uses survey data about pain in nursing home residents and analyses, whether affiliation-of-interviewer (internal vs. external to nursing home) and gender-of-interviewer affect residents' responses in terms of interviewer variance and systematically varying pain reports. METHODS: Overall, 258 nursing home residents with up to moderate cognitive impairment were examined by 61 interviewers about pain intensity and interference applying the Brief Pain Inventory. Interviewer variance was measured using intra-interviewer correlation coefficients (ρ). Two-factorial covariance analysis was applied to analyse whether pain intensity and interference scores differ by interviewer characteristics. RESULTS: Interviewer heterogeneity accounts for almost one quarter of total variance on average. Interviewer variance is higher for internal and male interviewers than for external and female interviewers. Covariance analyses show significant effects of interviewer characteristics on pain reports. Average pain intensity and interference scores vary considerably by interviewer gender and affiliation. Highest pain intensity was reported towards female internal and male external interviewers; highest pain interference was reported towards male external interviewers. CONCLUSION: Residents' answers substantially differ in relation to who is assessing pain. There is a risk of imprecise and biased survey estimates on sensitive topics like pain in nursing homes. Interviewer gender and affiliation seem to evoke gender-specific and status-related expectations and attributions which influence residents' response process. Interviewer effects pose a considerable threat to survey data quality in institutionalised older populations.

The belief that survey research instruments mediating communication between interviewer and respondent influence the quality of data obtained in the interview has become a conventional methodological wisdom long ago. However, the impact of the cognitive load experienced by the interviewer has not been systematically examined. When a questionnaire is filled up by an interviewer it is the latter who has to allocate limited individual resources of attention, memory, visual and motor control, active listening and interpretation in order to minimize the respondent’s misunderstanding of the questions and one’s own errors of the answers’ fixation. However, among various methods of pre-testing or evaluating survey mode effects and assessing instruments’ quality, the methods of multimodal quantitative estimation of instrument-related cognitive load experienced by interviewers during the interview are still rare or lacking. Thus, the article presents a brief review of subjective, behavioral and physiological measures of the cognitive load, which are used in such disciplinary fields as cognitive science, ergonomics, etc., and a discussion of preliminary findings of the field quasi-experiment conducted among the interviewers of the Russian Longitudinal Monitoring Survey on the first stage of transition to the CAPI mode. The quasi-experiment findings prove some possibilities and limitations of the parallel use of a version of the cognitive load rating scale developed by F. Paas and a simple physiological measure (heart rate) supplemented with a background screen video capture from Android-based tablets used for CAPI interviews for the multimodal quantitative evaluation and optimization of the interviewer’s cognitive load.

For a given survey data file we define a non-unique record, NUR, as a sequence of all values in a given case (record), which is identical to that of another case in the same dataset. We analyzed 1,721 national surveys in 22 international projects, covering 142 countries and 2.3 million respondents, and found a total of 5,893 NURs concentrated in 162 national surveys, in 17 projects and 80 countries. We show that the probability of the occurrence of any NUR in an average survey sample is exceedingly small, and although NURs constitute a minor fraction of all records, it is unlikely that they are solely the result of random chance. We describe how NURs are distributed across projects, countries, time, modes of data collection, and sampling methods. We demonstrate that NURs diminish data quality and potentially have undesirable effects on the results of statistical analyses. Identifying NURs allows researchers to examine the consequences of their existence in data files. We argue that such records should be flagged in all publically available data archives. We provide a complete list of NURs for all analyzed national surveys., Survey Research Methods, Vol 11, No 1 (2017)

There is an ongoing debate in the survey research literature about whether and when probability and nonprobability sample surveys produce accurate estimates of a larger population. Statistical theory provides a justification for confidence in probability sampling as a function of the survey design, whereas inferences based on nonprobability sampling are entirely dependent on models for validity. This article reviews the current debate about probability and nonprobability sample surveys. We describe the conditions under which nonprobability sample surveys may provide accurate results in theory and discuss empirical evidence on which types of samples produce the highest accuracy in practice. From these theoretical and empirical considerations, we derive best-practice recommendations and outline paths for future research.

There is an ongoing debate in the survey research literature about whether and when probability and nonprobability sample surveys produce accurate estimates of a larger population. Statistical theory provides a justification for confidence in probability sampling as a function of the survey design, whereas inferences based on nonprobability sampling are entirely dependent on models for validity. This article reviews the current debate about probability and nonprobability sample surveys. We describe the conditions under which nonprobability sample surveys may provide accurate results in theory and discuss empirical evidence on which types of samples produce the highest accuracy in practice. From these theoretical and empirical considerations, we derive best-practice recommendations and outline paths for future research.

There is an ongoing debate in the survey research literature about whether and when probability and nonprobability sample surveys produce accurate estimates of a larger population. Statistical theory provides a justification for confidence in probability sampling as a function of the survey design, whereas inferences based on nonprobability sampling are entirely dependent on models for validity. This article reviews the current debate about probability and nonprobability sample surveys. We describe the conditions under which nonprobability sample surveys may provide accurate results in theory and discuss empirical evidence on which types of samples produce the highest accuracy in practice. From these theoretical and empirical considerations, we derive best-practice recommendations and outline paths for future research.

Effets synchrones et longitudinaux sur la qualité des panels CATI des intervieweurs et des répondants : C’est en particulier dans les enquête par panel que les effets de la perte de répondants, de l’apprentissage par les répondants, et de l’expérience des intervieweurs jouent un rôle crucial en ce qui concerne la qualité des données. Nous examinons de manière synchrone et longitudinale pour la même enquête trois indicateurs de qualité. Dans l’analyse synchrone, nous comparons la qualité des données dans l’échantillon d’origine non-renouvellé avec celle d’un échantillon qui l’a été, les deux étant sondé dans la même vague. Pour la même vague, une enquête des intervieweurs a été faite, fournissant des données sur leur démographie sociale, leurs attitudes envers des enquêtes, et leur estimation de la charge de travail, nous permettant ainsi d’estimer l’effet des intervieweurs. L’analyse longitudinale permet l’analyse des effets de l’apprentissage par les répondants par rapport aux indicateurs de qualité en considérant les mêmes répondants dans plusieurs vagues. Le Panel Suisse des Ménages – une enquête CATI représentative de la population résidentielle suisse – fournit une base de données idéale pour modéliser puisque l’affectation des intervieweurs est au hasard pour toutes les vagues et dans chacune. Ce plan évite la confusion possible avec d’autres effets dus aux affectations non aléatoire telles que les affectations par quartiers où les meilleurs enquêteurs traitant les cas les plus difficiles. Pour séparer les effets des intervieweurs, des répondants et des vagues, nous utilisons des models multi-niveaux croisés. Especially in panel surveys, respondent attrition, respondent learning, and interviewer experience effects play a crucial role with respect to data quality. We examine three interview survey quality indicators in the same survey in a cross sectional as well as in a longitudinal way. In the cross sectional analysis we compare data quality in the mature original sample with that in a refreshment sample, surveyed in the same wave. Because in the same wave an interviewer survey was conducted, collecting attitudes on their socio demography, survey attitudes and burden measures, we are able to consider interviewer fixed effects as well. The longitudinal analysis gives more insight in the respondent learning effects with respect to the quality indicators considered by considering the very same respondents across waves. The Swiss Household Panel, a CATI survey representative of the Swiss residential population, forms an ideal modelling database: the interviewer – respondent assignment is random, both within and across waves. This design avoids possible confusion with other effects stemming from a non-random assignment of interviewers, e.g. area effects or effects from assigning the best interviewers to the hard cases. In order to separate interviewer, respondent and wave effects, we build cross-classified multilevel models.

Cet article est l’introduction d’une thèse avec le même titre (fait au Départment de Statistique de l’université de Stockholm) et qui s’adresse à des problèmes associés avec la qualité des enquêtes par entretiens. L’intervieweur a plusieurs taches à accomplir et ils peuvent souvent commetre des erreurs. L’auteur présente quelques thèmes généraux ainsi que leur traitement habituel. Elle suggère ensuite des approches nouvelles et des idées pour des recherches futures sur la question de coopération dans les enquêtes et d’accomplir correctement l’entretien avec un répondant. Elle examine aussi certaines procédures pour la réduction des non-réponses en termes de coûts et de réduction d’erreurs. This article is the introduction to a thesis with the same title (done at the Department of Statistics of the University of Stockholm) which deals with some of the issues associated with the quality in interview surveys. The interviewer has many tasks, and some of them are extremely error-prone. The author describes some general themes and how these are usually handled. She also suggests some new approaches and ideas for further work, both when it comes to gaining cooperation and when it comes to doing a good job in administering the question-answer process. She also evaluates a specific procedure to reduce nonresponse errors in interview surveys in terms of costs and error reduction.

A survey with validation question was delivered to 3475 respondents to study the impact of using validation question to improve data quality. Results show that The respondents who passed the validation question had significantly older ages, higher median incomes and higher education levels. The WTP estimates with and without the respondents who fails the validation question differ significantly.