Search

In this non-empirical article, which is intended as a decision-making resource for researchers, we identify issues surrounding data integrity that commonly arise in online survey research and we propose remediation strategies based on challenges we encountered during a particular pilot study as well as our collective experience with conducting online survey research. Using the metaphor of an onion, we peel off the layers of this complex problem, synthesize the various available strategies used across disciplines, and propose some novel ones based on our perspective as psychologists. Corresponding to this multi-layered problem, we propose multi-layered solutions to prevent illegitimate responding—by both humans and non-humans (robots or “bots” for short)—from compromising the quality of data collected via online survey research. The first layer entails strategic item selection and protective programming in survey development. The second layer involves astute advertising and recruitment tactics to minimize illegitimate responses during survey dissemination. The third layer includes algorithms and other mechanisms to identify suspicious responses for possible exclusion during data verification. When we peel off the layers and reach the core problem of illegitimate responses to online surveys—financial incentives—we will propose ways of navigating respondent reimbursement to mitigate their inadvertent harmful impacts on the research process. By proposing these solutions, we aim to protect the integrity of scientific inquiry in psychology, especially given how often this method is used in the discipline.

Background: Scientific evidence indicates that HIV viral suppression to an undetectable level eliminates sexual transmission risk ("Undetectable=Untransmittable" or "U=U"). However, U=U messaging has been met with skepticism among sexual minority men (SMM) and others. In this survey-based experiment, we manipulated messaging about HIV risk and examined reactions and perceived message accuracy among US SMM., Methods: SMM living with HIV (n = 106) and HIV-negative/status-unknown SMM (n = 351) participated in an online survey (2019-2020). Participants were randomly assigned to 1 of 3 messaging conditions, which varied by level of HIV sexual transmission risk associated with an undetectable viral load (No Risk [U=U]/Low Risk/Control). Participants reported reactions, message accuracy, and reasons for perceiving inaccuracy. We coded open-response data (reactions and reasons) into conceptual categories (eg, "Enthusiasm"). We compared reactions, accuracy ratings, and reasons by condition and serostatus., Results: In the No Risk Condition, common reactions were Enthusiasm (40.0%), Skepticism/Disagreement (20.0%), and Agreement (19.4%), reactions common to comparison conditions. A higher percentage of HIV-negative/status-unknown participants (24.1%) expressed Skepticism/Disagreement in the No Risk Condition compared with other conditions (3.2%-9.7%). Participants living with HIV were more likely than HIV-negative/status-unknown participants to perceive the message as accurate in all conditions. In the No Risk Condition, common reasons for perceiving inaccuracy were Risk Misstated (46.1%), Oversimplified/Caveats Needed (17.1%), and Personal Unfamiliarity/Uncertainty (14.5%), reasons common to comparison conditions. Across conditions, 10.3% of participants attributed message inaccuracy to undetectable being misdefined., Conclusion: Most SMM reacted favorably to U=U messaging. However, many-especially HIV-negative/status-unknown SMM-expressed skepticism. Interventions are needed to enhance U=U understanding and acceptance., Competing Interests: The authors have no funding or conflicts of interest to disclose., (Copyright © 2024 Wolters Kluwer Health, Inc. All rights reserved.)