Search

Amazon Mechanical Turk (MTurk) is one of the most popular crowdsourcing platforms for online research. Although the recent MTurk data quality crisis has raised serious concerns regarding its utility, this issue can be mitigated by including quality control methods. To date, only a few studies have compared the effectiveness of the methods in preserving an experimental effect. To provide empirical guidance on which quality control methods to use for MTurk research, this study compared different methods on their effectiveness in preserving the magnitude effect in delay discounting, a robust finding in the behavioral economics literature. Two groups of MTurkers, whose HIT approval rate (%) was between 95 and 98 and 99 and 100, were recruited in October 2021. They completed a delay discounting task evaluating the magnitude effect and questions from different quality control methods. The results showed that the group with higher HIT approval rates (i.e., 99%--100%) showed a greater magnitude effect and were more likely to pass the quality control methods than those with lower rates. In addition, the majority of the quality control methods were able to preserve the magnitude effect at the expense of sample sizes. The best combinations of quality control methods that optimally maintained the effect were explored with a discussion of their implications. Although improving data quality is imperative, caution should be exercised as the quality control methods may potentially bias the sample in an undesirable manner (e.g., excluding individuals who show steep delay discounting). Keywords MTurk * Data quality * Online research * Delay discounting * Magnitude effect, Introduction The use of online crowdsourcing platforms to collect behavioral data has become popular in social science research in recent decades. These platforms provide evident scientific benefits, including efficiency and [...]

Episodic future thinking (EFT), an intervention involving mental simulation of future events, has been shown to reduce both delay discounting and cigarette self-administration. In the present study, we extended these findings by showing that EFT in a web-based sample of smokers reduces delay discounting and intensity of demand for cigarettes (ad libitum consumption) in a hypothetical purchase task. No effect was observed on elasticity of demand (sensitivity to price) or cigarette craving. We also explored whether demand characteristics (specifically, the 'good-subject' effect) might be responsible for observed effects. EFT participants were significantly better able than control participants to discern the experimental hypothesis. However, EFT participants were not better than controls at identifying whether they had been assigned to the experimental group and, likewise, showed no differences in attitudes about the experiment and experimenter. Importantly, effects of EFT on delay discounting and demand remained significant even when controlling for measures of demand characteristics, indicating that EFT's effects are independent of participants' perceptions about the experiment., Author(s): Jeffrey S. Stein [sup.1] , Allison N. Tegge [sup.1] [sup.2] [sup.3] , Jamie K. Turner [sup.1] , Warren K. Bickel [sup.1] [sup.4] [sup.5] [sup.6] [sup.7] Author Affiliations: (Aff1) 0000 [...]

Objective: 'Sludge' refers to administrative burdens or frictions that preclude people from getting what they want or need (eg, duplicative forms, complicated instructions, long waiting times). This mixed methods study evaluated patients' perceptions of sludge in the colorectal cancer (CRC) screening process and some impacts of this sludge., Design: We employed an exploratory sequential mixed methods study design that comprised patient interviews and a patient survey. The interviews informed final survey revisions and captured contextual data about patients' experiences with sludge. Interview transcripts were inductively and deductively analysed to identify overarching themes. The survey quantified sludge, delayed or forgone screenings, screening experience (Net Promoter Score) and health system distrust (Health System Distrust Scale). We used χ 2 or t-tests for univariable comparisons and logistic or linear regressions to evaluate the association between cumulative sludge score and delayed or forgone screenings, screening experience and health system distrust. Results were integrated for interpretation., Setting: Southeastern United States., Participants: Patients who were 45-75 years of age, at average risk for CRC and had either completed or been referred for CRC screening (colonoscopy or stool-based test) within the previous 12 months., Results: 22 interview participants and 255 survey participants completed the study. 38 (15%) survey participants rated their screening experience as poor (Net Promoter Score=0-7 out of 10). The mean (SD) Health System Distrust Scale score was 22.4 (6.3) out of 45 possible points (higher score=greater distrust). Perceptions of sludge in the CRC screening process varied, with long waiting times and burdensome communication being the most common sources (58% and 35% of participants, respectively). Sludge was positively associated with delayed or forgone screenings (OR=1.42, 95% CI 1.28, 1.57, p<0.001), poor screening experience (OR=1.15, 95% CI 1.04, 1.28, p=0.009) and health system distrust (β=0.47, p<0.001). Qualitative findings add descriptive detail about sludge encountered, context to impacts experienced, and illustrate the heavy emotional impact of sludge: ' it just isn't worth it' ., Conclusion: Efforts to reduce sludge in the CRC screening process may improve timely completion of CRC screening, enhance patient experience and restore trust in the health system., Competing Interests: Competing interests: None declared., (© Author(s) (or their employer(s)) 2024. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.)