Search

The 2020 Decennial Census garnered unprecedented attention due to the proposed reintroduction of the citizenship question by the Trump Administration. The legal imperative to respond to the Census and the sensitive nature of the citizenship question led to tensions and public outrage. This paper discusses the controversy surrounding the addition of the citizenship question and its implications on vulnerable communities, particularly undocumented immigrants and communities of color. Although the citizenship question was ultimately not added to the 2020 Decennial Census, it continued to be asked on the American Community Survey (ACS). We examine how negative public sentiment during this time, in tandem with historical mistrust of government data collection, may have manifested as item non-response within the ACS, specifically on questions relating to citizenship status and, more broadly, the Origin and Language subgroup of questions. The analyses in this paper explore trends in non-response rates from the American Community Survey from 2014 to 2022. Item allocation rates are used as a proxy for non-response to questions relating to citizenship. A variety of data visualization tools, supported by statistical methods, are employed to weave connections between allocation trends and the proposed addition of the citizenship question to the Decennial Census. We aim to bring transparency to this issue to educate and encourage response within vulnerable communities by emphasizing the importance of accurate data collection and positive perceptions of government-issued surveys, which are used to make key legislative decisions and create a more democratic nation. [ABSTRACT FROM AUTHOR]

In Estonia, the Generations and Gender Survey 2020 (GGS-II) is the third large-scale demographic survey that collects data on family and fertility dynamics. As the country participates in the Survey of Health, Ageing and Retirement in Europe, the GGS-II opted for a shorter age range of the sample (18–59). The questionnaire in the GGS-II in Estonia follows the GGS-II wave 1 baseline questionnaire. The questionnaire also includes the Global Uncertainties’ module developed by the Nordic countries, a battery of questions on the perceived impact of COVID-19, and several country-specific items. The GGS-II in Estonia was implemented using only computer-assisted web interviewing (CAWI). In this article, we present a concise overview of the sampling and data collection process, analyse representativeness and response rates, and briefly assess the data quality. We conclude that despite low response rates, the GGS-II provides a good basis for the analysis of fertility and family dynamics.

In recent years, the increase in general interest in methods for measuring cognitive load and subjectively perceived mental effort when solving various tasks and in the interpersonal communication was accompanied by an increase in the specific interest of social researchers in the multimodal assessment of the cognitive load of interviewers and respondents based on objective and subjective indicators, including paradata and webcam data, in order to control this loads impact on the quality of survey data. The authors argue that the possibilities of relatively new approaches to measuring cognitive load with neurophysiological methods (such as the use of wearable devices for oculography - eye tracking and pupillometry - which do not disrupt the natural course of respondents and interviewers activity) are still underestimated, although they allow an accurate time linkage of measured parameters dynamics (primarily the size of the pupil) to the question format, mode and phase of survey completion, external influences localized in time, etc. As a rule, quantitative studies of surveys cognitive load and its possible impact on the quality of survey data focus on computer-assisted (CAPI) or paper-based (PAPI) interviewing, while the specificity of the cognitive load in the self-completed computerized (CASI) and paper (PPSI) surveys was not studied. The article presents the results of the methodological experiment based on a modified version of the multimodal approach to the comparative assessment of the cognitive load of interviewers working with paper and computerized questionnaire. We expanded the range of methods for assessing cognitive load by using a wearable oculographic device (eye tracker) to measure the dynamics of pupil size when answering different survey questions. The results of the experiment confirmed the hypothesis about the approximate equivalence of the two modes of survey completion in terms of their cognitive load for younger respondents with a high level of functional computer literacy, and allowed an initial assessment of the technical and metrological capabilities and limitations of the use of pupil dynamics indicators, measured with a wearable oculographic device, to study the respondents cognitive load.

This article proposes a new approach to analyze protest participation measured in surveys of uneven quality. Because single international survey projects cover only a fraction of the world's nations in specific periods, researchers increasingly turn to ex-post harmonization of different survey data sets not a priori designed as comparable. However, very few scholars systematically examine the impact of the survey data quality on substantive results. We argue that the variation in source data, especially deviations from standards of survey documentation, data processing, and computer files—proposed by methodologists of Total Survey Error, Survey Quality Monitoring, and Fitness for Intended Use—is important for analyzing protest behavior. In particular, we apply the Survey Data Recycling framework to investigate the extent to which indicators of attending demonstrations and signing petitions in 1,184 national survey projects are associated with measures of data quality, controlling for variability in the questionnaire items. We demonstrate that the null hypothesis of no impact of measures of survey quality on indicators of protest participation must be rejected. Measures of survey documentation, data processing, and computer records, taken together, explain over 5% of the intersurvey variance in the proportions of the populations attending demonstrations or signing petitions. [ABSTRACT FROM AUTHOR]

Introduction Face-to-face surveys are applied frequently when conducting research in older populations. Interviewers play a decisive role in data quality, may affect measurement and influence results. This study uses survey data about pain in nursing home residents and analyses, whether affiliation-of-interviewer (internal vs. external to nursing home) and gender-of-interviewer affect residents' responses in terms of interviewer variance and systematically varying pain reports. Methods Overall, 258 nursing home residents with up to moderate cognitive impairment were examined by 61 interviewers about pain intensity and interference applying the Brief Pain Inventory. Interviewer variance was measured using intra-interviewer correlation coefficients (ρ). Two-factorial covariance analysis was applied to analyse whether pain intensity and interference scores differ by interviewer characteristics. Results Interviewer heterogeneity accounts for almost one quarter of total variance on average. Interviewer variance is higher for internal and male interviewers than for external and female interviewers. Covariance analyses show significant effects of interviewer characteristics on pain reports. Average pain intensity and interference scores vary considerably by interviewer gender and affiliation. Highest pain intensity was reported towards female internal and male external interviewers; highest pain interference was reported towards male external interviewers. Conclusion Residents' answers substantially differ in relation to who is assessing pain. There is a risk of imprecise and biased survey estimates on sensitive topics like pain in nursing homes. Interviewer gender and affiliation seem to evoke gender-specific and status-related expectations and attributions which influence residents' response process. Interviewer effects pose a considerable threat to survey data quality in institutionalised older populations. [ABSTRACT FROM AUTHOR]

In recent decades, factorial survey experiments (FSEs) have become increasingly widespread and successful for analyzing attitudes and behavioral intentions. FSEs measure the ratings of multidimensional treatments embedded in textual scenarios, which are called vignettes. Analyses of FSEs often assume that these ratings are interval scaled. Past research indicates that this assumption is problematic. Therefore, the following article develops a design for interval scaling tests in FSEs and a method for analyzing FSEs which is sensitive to the scaling level of ratings. An exemplary application of scaling sensitive factorial survey analysis in comparison to standard methods yields effect sizes, which are about a sixth larger with regard to the treatments used in the FSE and about a third larger with respect to influences of between respondent differences. The new method also enables the evaluation of interval in contrast to noninterval scaled rating behavior. [ABSTRACT FROM AUTHOR]

The belief that survey research instruments mediating communication between interviewer and respondent influence the quality of data obtained in the interview has become a conventional methodological wisdom long ago. However, the impact of the cognitive load experienced by the interviewer has not been systematically examined. When a questionnaire is filled up by an interviewer it is the latter who has to allocate limited individual resources of attention, memory, visual and motor control, active listening and interpretation in order to minimize the respondent’s misunderstanding of the questions and one’s own errors of the answers’ fixation. However, among various methods of pre-testing or evaluating survey mode effects and assessing instruments’ quality, the methods of multimodal quantitative estimation of instrument-related cognitive load experienced by interviewers during the interview are still rare or lacking. Thus, the article presents a brief review of subjective, behavioral and physiological measures of the cognitive load, which are used in such disciplinary fields as cognitive science, ergonomics, etc., and a discussion of preliminary findings of the field quasi-experiment conducted among the interviewers of the Russian Longitudinal Monitoring Survey on the first stage of transition to the CAPI mode. The quasi-experiment findings prove some possibilities and limitations of the parallel use of a version of the cognitive load rating scale developed by F. Paas and a simple physiological measure (heart rate) supplemented with a background screen video capture from Android-based tablets used for CAPI interviews for the multimodal quantitative evaluation and optimization of the interviewer’s cognitive load.

There is an ongoing debate in the survey research literature about whether and when probability and nonprobability sample surveys produce accurate estimates of a larger population. Statistical theory provides a justification for confidence in probability sampling as a function of the survey design, whereas inferences based on nonprobability sampling are entirely dependent on models for validity. This article reviews the current debate about probability and nonprobability sample surveys. We describe the conditions under which nonprobability sample surveys may provide accurate results in theory and discuss empirical evidence on which types of samples produce the highest accuracy in practice. From these theoretical and empirical considerations, we derive best-practice recommendations and outline paths for future research. [ABSTRACT FROM AUTHOR]

The current paper proposes a novel method for detecting careless respondents, namely, floodlight detection of careless respondents. This novel method consists of two steps: (1) creating a nonsense regression model and (2) testing a moderator role of response time on the nonsense regression model. An illustration of the floodlight detection of careless respondents method was performed with online survey data collected from 341 Turkish participants. The floodlight detection of careless respondents method is the first systematic approach to calculate a cut-off value for response time, which distinguishes careless respondents from careful respondents. According to the results of floodlight detection of careless respondents, the percentage of careless respondents was 59.8%, which is little higher than the percentage of careless respondents calculated through Instructional Manipulation Check (40.7%) and bogus item (44.2%). The floodlight detection of careless respondents method is described herein, and implications are provided for future research. [ABSTRACT FROM AUTHOR]

Surveys are one of the most popular ways to collect employee information. Because of their widespread use, data quality is an increasingly important concern. The purpose of this paper is to (1) introduce the intra-individual response variability (IRV) index as an easily calculated and flexible way to detect insufficient effort responding (IER); (2) examine the extent to which various IER indices detect the same or different respondents engaging in IER behavior; and (3) investigate relationships between individual differences and commonly used IER indices to better understand systematic and theoretically relevant IER behavior. In a two-part study, 199 undergraduates responded to questionnaires online, and various IER indices were calculated. The IRV index identifies different respondents than other IER indices. Values on the IRV index (as well as other IER indices) are related to scores on theoretically meaningful individual differences in conscientiousness, agreeableness, and boredom proneness. This study provides researchers with a robust, easily calculated, and flexible means for screening questionnaire data for IER behavior. Practical recommendations for finding and making decisions about IER behavior patterns are provided. This study introduces the IRV index, an extension of the long string, used to identify survey research participants who likely engaged in one type of IER behavior. It is also one of the first studies to evaluate the extent to which IER indices identify different respondents as having engaged in IER and provides additional evidence that values on these indices are related to individual differences. [ABSTRACT FROM AUTHOR]

For a given survey data file we define a non-unique record, NUR, as a sequence of all values in a given case (record), which is identical to that of another case in the same dataset. We analyzed 1,721 national surveys in 22 international projects, covering 142 countries and 2.3 million respondents, and found a total of 5,893 NURs concentrated in 162 national surveys, in 17 projects and 80 countries. We show that the probability of the occurrence of any NUR in an average survey sample is exceedingly small, and although NURs constitute a minor fraction of all records, it is unlikely that they are solely the result of random chance. We describe how NURs are distributed across projects, countries, time, modes of data collection, and sampling methods. We demonstrate that NURs diminish data quality and potentially have undesirable effects on the results of statistical analyses. Identifying NURs allows researchers to examine the consequences of their existence in data files. We argue that such records should be flagged in all publically available data archives. We provide a complete list of NURs for all analyzed national surveys.

Background: Little is known about the public health professionals engaged in educating and training new or future researchers in public health. Research in this direction identifies their issues, concerns, challenges, and needs. This study focused on the professional development challenges of Chinese public health professionals., Methods: Snowball sampling was utilized. A total of 265 public health professionals participated. An instrument of 6 dimensions (burnout, sleep issue, mood issue, friends' support, exercise, and challenges) was developed, revised, and administered online. Two different approaches, the conventional and data screening approaches, were applied. The former started with item quality analyses, whereas the latter began with data quality checks. The chi-square tests of associations and logistic regressions were performed on both approaches., Results and Discussion: 19.25% of the participants were detected and deleted as careless respondents. Using both approaches, six professional development challenges except one ("Multidisciplinary learning") were significantly associated with various demographic features. The two approaches produced different models though they converged sometimes. The latent variables of exercise predicted professional development challenges more frequently than other latent variables. Regarding correct classification rates, results from the data screening approach were comparable to those from the conventional approach., Conclusion: The latent variables of exercise, such as "Exercise effects," "Expectations of exercise," and "Belief in exercise," might be understudied. More research is necessary for professional development challenges using exercise as a multidimensional construct. Based on the current study, screening and deleting careless responses in survey research is necessary., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2023 Wang, Kong, Li and Zhao.)

Conducting online surveys through consumer panels has become increasingly popular for researchers to assess consumers’ preferences and attitudes for the purpose of obtaining welfare estimates because they are inexpensive, flexible, and allow for fast feedback. Interestingly, few studies have examined the behaviors of online survey panelists, particularly their satisficing behaviors on welfare measures. This study demonstrates the use of validation questions (trap questions) to detect survey respondents’ satisficing behavior and its impact on consumer choice, willingness to pay (WTP), and consumer surplus (CS) estimates. We find that respondents who fail a validation question (VQ) are more likely to violate the weak axiom of revealed preferences ( WARP ) in the choice experiment. The estimates for preference parameters, WTP, and CS are statistically different between those who pass and those who fail the VQ. In addition, the WTP and CS from respondents passing the VQ in general have smaller variances than those from respondents failing the VQ. These results indicate that without controlling for potential satisficing behaviors, online surveys may produce less efficient estimates (estimates with larger variance) of welfare measures. [ABSTRACT FROM AUTHOR]

•Ensuring data quality in large scale surveys is challenging.•The trend and pattern of declining fertility and declining contraceptive use in India is puzzling.•Interview privacy setting and interviewer effect can partially explain the anomaly.•Large scale surveys impose severe demands on survey supervision and ability to ensure privacy.•Innovative ways of data collection for sensitive issues can be explored for proper reporting., Competing Interests: None., (© 2022 Published by Elsevier Ltd.)

The use of chatbots as an online survey tool is becoming increasingly popular owing to their convenience, particularly when face-to-face interactions are difficult. However, with longer surveys, interaction experience and data quality can decrease due to several factors, such as increased fatigue. In this study, we compared how applying humanization techniques to survey chatbots can affect survey-taking experience in three aspects: respondents' perceptions of chatbots, interaction experience, and data quality. To address our research goal, two different versions of survey chatbots were compared: a humanization applied survey chatbot (HASbot) and a baseline chatbot (baselinebot). The HASbot simultaneously incorporates four humanization techniques: use of self-introduction, addressing by name, using adaptive response speed, and echoing respondents' answers. Our experimental study with 59 middle school-aged adolescents showed that compared to the baselinebot, respondents' perceptions of the HASbot were more positive, with higher levels of anthropomorphism and social presence. In terms of interaction experience, the respondents spent more time interacting with the HASbot and showed a higher level of satisfaction. For data quality, the HASbot outperformed the baselinebot in terms of self-disclosure; however, the HASbot also elicited a higher social desirability bias. No difference was observed in the response differentiation between the two chatbots. [Display omitted] • We compared humanization applied survey chatbot (HASbot) to baseline chatbot. • HASbot increases anthropomorphism and social presence perceptions. • HASbot elicits higher satisfaction & longer interaction time. • HASbot yields higher level of self-disclosure. • HASbot has a drawback of causing social desirability bias. [ABSTRACT FROM AUTHOR]