Search

There has been growing recognition of the popularity of medical crowdfunding and research documenting how crowdfunding arises from, and contributes to, social and health inequities. While many researchers have surmised that racism could well play a role in medical crowdfunding campaign outcomes, research on these dynamics has been limited. No research to date has examined these dynamics among the most successful medical crowdfunding campaigns, focusing instead on average users' experiences or specific patient subpopulations. This paper analyzes key characteristics and demographics of the 827 most successful medical crowdfunding campaigns captured at a point in time in 2020 on the popular site GoFundMe, creating the first demographic archetype of "viral" or highly successful campaigns. We hypothesized that this sample would skew towards whiter, younger populations, more heavily represent men, and reflect critical illnesses and accidents affecting these populations, in addition to having visually appealing, well-crafted storytelling. Analysis supported these hypotheses, showing significant levels of racial and gender disparities among campaigners. While white men had the greatest representation, Black and Asian users, and black women in particular, were highly underrepresented. Like other studies, we find evidence that racial and gender disparities persist in terms of campaign outcomes as well. Alongside this quantitative analysis, a targeted discourse analysis revealed campaign narratives and comments reinforced racist and sexist tropes of selective deservingness. These findings add to growing calls for more health research into the ways that social media technologies shape health inequities for historically marginalized and disenfranchised populations. In particular, we underscore how successful crowdfunding campaigns, as a both a means of raising funds for health and a broader site of public engagement, may deepen and normalize gendered and racialized inequities. In this way, crowdfunding can be seen as a significant technological amplifier of the fundamental social causes of health disparities., (Copyright © 2023 Elsevier Ltd. All rights reserved.)

Objectives. To assess whether medical crowdfunding use and outcomes are aligned with health financing needs in the United States. Methods. We collected data on 437 596 US medical GoFundMe campaigns between 2016 and 2020. In addition to summarizing trends in campaign initiation and earnings, we used state- and county-level data to assess whether crowdfunding usage and earnings were higher in areas with greater medical debt, uninsured populations, and poverty. Results. Campaigns raised more than $2 billion from 21.7 million donations between 2016 and 2020. Returns were highly unequal, and success was low, especially in 2020: only 12% of campaigns met their goals, and 16% received no donations at all. Campaigns in 2020 raised substantially less money in areas with more medical debt, higher uninsurance rates, and lower incomes. Conclusions. Despite its popularity and portrayals as an ad-hoc safety net, medical crowdfunding is misaligned with key indicators of health financing needs in the United States. It is best positioned to help in populations that need it the least. ( Am J Public Health . 2022;112(3):491-498. https://doi.org/10.2105/AJPH.2021.306617).

Objectives: Crisis pregnancy centers (CPCs) are nonprofit antiabortion organizations that claim provision of pregnancy resources. With the Reproduction Freedom, Accountability, Comprehensive Care, and Transparency Act repealed, CPCs are no longer mandated to share information on state-funded family planning and abortion services. As patients increasingly seek healthcare guidance online, we evaluated crowd-sourced reviews of CPCs using the social networking site Yelp., Methods: CPCs were identified with the CPC Map, a geo-based location resource. Of California's 145 CPCs, 84% had Yelp pages, and 619 reviews (2010-2019) were extracted. Thematic codes were individually applied to 220 excerpts and then analyzed in detail using thematic analysis to capture emergent themes related to motivations for and experiences of CPCs. To ensure thematic saturation, we applied a natural language-processing technique called the meaning extraction method to computationally derive themes of discussion from all of the extracted posts., Results: Motivations to seek care from CPCs included pregnancy confirmation, gaps in healthcare coverage, parenting and emotional support, and abortion care. A review of experiences reveal that CPC faith-based practice garnered both positive- and negative-based experiences. Reviewers also articulated inaccurate medical information, lack of transparency, and reduced options at CPCs., Conclusions: This is the first study to analyze California CPCs using a social media platform. Pregnant patients turn to social media to share experiences about pregnancy resources, to find healthcare providers, and to increase transparency of services. This content provides valuable insight into the concerns of pregnant patients and offers an intimate view of California CPCs at a time when no federal regulations are in place.

Burgeoning cases of tick-borne disease present a significant public health problem in the United States. Passive tick surveillance gained traction as an effective way to collect epidemiologic data, and in particular, photograph-based tick surveillance can complement in-hand tick specimen identification to amass distribution data and related encounter demographics. We compared the Federal Information Processing Standards (FIPS) code of tick photos submitted to a free public identification service (TickSpotters) from 2014 to 2019 to published nationwide county reports for three tick species of medical concern: Ixodes scapularis Say (Ixodida: Ixodidae), Ixodes pacificus Cooley and Kohls (Ixodida: Ixodidae), and Amblyomma americanum Linneaus (Ixodida: Ixodidae). We tallied the number of TickSpotters submissions for each tick species according to "Reported" or "Established" criteria per county, and found that TickSpotters submissions represented more than half of the reported counties of documented occurrence, and potentially identified hundreds of new counties with the occurrence of these species. We detected the largest number of new county reports of I. scapularis presence in Michigan, North Carolina, and Texas. Tick image submissions revealed potentially nine new counties of occurrence for I. pacificus, and we documented the largest increase in new county reports of A. americanum in Kentucky, Illinois, Indiana, and Ohio. These findings demonstrate the utility of crowdsourced photograph-based tick surveillance as a complement to other tick surveillance strategies in documenting tick distributions on a nationwide scale, its potential for identifying new foci, and its ability to highlight at-risk localities that might benefit from tick-bite prevention education., (© The Author(s) 2021. Published by Oxford University Press on behalf of Entomological Society of America. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.)

Importance: Mortality across US counties varies considerably, from 252 to 1847 deaths per 100 000 people in 2018. Although patient satisfaction with health care is associated with patient- and facility-level health outcomes, the association between health care satisfaction and community-level health outcomes is not known., Objective: To examine the association between online ratings of health care facilities and mortality across US counties and to identify language specific to 1-star (lowest rating) and 5-star (highest rating) reviews in counties with high vs low mortality., Design, Setting, and Participants: This retrospective population-based cross-sectional study examined reviews and ratings of 95 120 essential health care facilities across 1301 US counties. Counties that had at least 1 essential health care facility with reviews available on Yelp, an online review platform, were included. Essential health care was defined according to the 10 essential health benefits covered by Affordable Care Act insurance plans., Main Outcomes and Measures: The mean rating of essential health care facilities was calculated by county from January 1, 2015, to December 31, 2019. Ratings were on a scale of 1 to 5 stars, with 1 being the worst rating and 5 the best. County-level composite measures of health behaviors, clinical care, social and economic factors, and physical environment were obtained from the University of Wisconsin School of Medicine and Public Health County Health Rankings database. The 2018 age-adjusted mortality by county was obtained from the Centers for Disease Control and Prevention Wide-ranging Online Data for Epidemiological Research database. Multiple linear regression analysis was used to estimate the association between mean facility rating and mortality, adjusting for county health ranking variables. Words with frequencies of use that were significantly different across 1-star and 5-star reviews in counties with high vs low mortality were identified., Results: The 95 120 facilities meeting inclusion criteria were distributed across 1301 of 3142 US counties (41.4%). At the county level, a 1-point increase in mean rating was associated with a mean (SE) age-adjusted decrease of 18.05 (3.68) deaths per 100 000 people (P < .001). Words specific to 1-star reviews in high-mortality counties included told, rude, and wait, and words specific to 5-star reviews in low-mortality counties included Dr, pain, and professional., Conclusions and Relevance: This study found that, at the county level, higher online ratings of essential health care facilities were associated with lower mortality. Equivalent online ratings did not necessarily reflect equivalent experiences of care across counties with different mortality levels, as evidenced by variations in the frequency of use of key words in reviews. These findings suggest that online ratings and reviews may provide insight into unequal experiences of essential health care.

Background: Hospital reviews posted online by patients are unsolicited and less structured than Hospital Consumer Assessment of Healthcare Providers and Systems surveys. The differences between online review platforms and their degrees of correlation with validated satisfaction and safety measures are unknown., Methods: We identified 515 large acute care teaching hospitals in the United States. We collected patient satisfaction results and postsurgical patient safety indicators from Hospital Compare. We also collected hospital star ratings (1-5) from Facebook, Google, and Yelp. Mean ratings were compared with paired t tests. Concordance between ratings websites, Hospital Consumer Assessment of Healthcare Providers and Systems scores, and surgical safety indicators were assessed with Pearson's correlation coefficient., Results: Mean Facebook ratings (3.81, interquartile range 3.5-4.3) were more favorable than Google (3.26, interquartile range 2.8-3.6) or Yelp (2.59, interquartile range 2.3-2.9). Facebook ratings were least strongly correlated with the Hospital Consumer Assessment of Healthcare Providers and Systems recommended hospital score (ρ = 0.356). Google was modestly correlated (ρ = 0.479), and Yelp was most strongly correlated (ρ = 0.500). The negative correlation between crowdsourced rating and composite safety indicator was too small to be meaningful on any platform., Conclusion: There is variation between platforms in consumer ratings of hospitals. Ratings on Facebook are more favorable than Google or Yelp. These are independently correlated with Hospital Consumer Assessment of Healthcare Providers and Systems scores. These findings suggest that unstructured consumer reviews generally reflect similar directionality as Hospital Consumer Assessment of Healthcare Providers and Systems satisfaction scores. Users should be aware of the significant difference between platforms. Consumer ratings platforms are not consistently correlated with postsurgical patient safety indicators, so online ratings may not reflect the safety of surgical care received., (Copyright © 2021 Elsevier Inc. All rights reserved.)

A growing number of studies within the field of gerontology have included samples recruited from Amazon's Mechanical Turk (MTurk), an online crowdsourcing portal. While some research has examined how younger adult participants recruited through other means may differ from those recruited using MTurk, little work has addressed this question with older adults specifically. In the present study, we examined how older adults recruited via MTurk might differ from those recruited via a national probability sample, the Health and Retirement Study (HRS), on a battery of outcomes related to health and cognition. Using a Latin-square design, we examined the relationship between recruitment time, remuneration amount, and measures of cognitive functioning. We found substantial differences between our MTurk sample and the participants within the HRS, most notably within measures of verbal fluency and analogical reasoning. Additionally, remuneration amount was related to differences in time to complete recruitment, particularly at the lowest remuneration level, where recruitment completion required between 138 and 485 additional hours. While the general consensus has been that MTurk samples are a reasonable proxy for the larger population, this work suggests that researchers should be wary of overgeneralizing research conducted with older adults recruited through this portal.

Every internet search query made out of curiosity by anyone who observed something in nature, as well as every photo uploaded to the internet, constitutes a data point of potential use to scientists. Researchers have now begun to exploit the vast online data accumulated through passive crowdsourcing for studies in ecology and epidemiology. Here, we demonstrate the usefulness of iParasitology, i.e. the use of internet data for tests of parasitological hypotheses, using hairworms (phylum Nematomorpha) as examples. These large worms are easily noticeable by people in general, and thus likely to generate interest on the internet. First, we show that internet search queries (collated with Google Trends) and photos uploaded to the internet (specifically, to the iNaturalist platform) point to parts of North America with many sightings of hairworms by the public, but few to no records in the scientific literature. Second, we demonstrate that internet searches predict seasonal peaks in hairworm abundance that accurately match scientific data. Finally, photos uploaded to the internet by non-scientists can provide reliable data on the host taxa that hairworms most frequently parasitize, and also identify hosts that appear to have been neglected by scientific studies. Our findings suggest that for any parasite group likely to be noticeable by non-scientists, information accumulating through internet search activity, photo uploads, social media or any other format available online, represents a valuable source of data that can complement traditional scientific data sources in parasitology.

Background/aim: The purpose of this study was to assess patients' use of a crowdfunding platform to raise funds for radiation treatment and to better understand the direct and indirect costs associated with treatments., Materials and Methods: The GoFundMe crowdfunding database was queried for four unique categories related to radiation treatment campaigns. Covariates identified included clinical and demographic variables, and associations between amount raised and these predictors were analyzed using a generalized linear model., Results: While 56% percent of campaigns cited direct costs associated with treatment, 73.4% of campaigns cited indirect costs related to treatment. Indirect expenses related to travel (31.7%) as well as living expenses (29.2%) were cited most often across all four treatment categories., Conclusion: This study enhances understanding regarding patients use of crowdfunding for radiation treatment. Increased focus should be placed on discussing the indirect costs of care with patients and their families., (Copyright © 2021 International Institute of Anticancer Research (Dr. George J. Delinasios), All rights reserved.)

This study shows that while status seeking motivates people to participate in crowdsourcing platforms, it also negatively impacts the bedrock of crowdsourcing-wisdom of crowds. Using Yelp restaurant reviews in 6 cities, we found that motivations of status seeking lead people to review a greater variety of restaurants, and achieving status further encourages this variety seeking as well as the targeting of more expensive restaurants for review. The impact of this individual-level tendency is confirmed by our aggregate-level analysis which shows that restaurants with higher price levels, higher uniqueness levels, and a larger percentage of elite reviews tend to obtain enough reviews to generate wisdom of crowds sooner than other restaurants. This leads to a different kind of distortion to crowd wisdom: an over-representation of status-conferring products and an under-representation of products that are not status-worthy. The findings suggest the importance of studying sources of distortion that are endemic to crowdsourcing itself., Competing Interests: The authors have declared that no competing interests exist.

The volume of labeled data is often the primary determinant of success in developing machine learning algorithms. This has increased interest in methods for leveraging crowds to scale data labeling efforts, and methods to learn from noisy crowd-sourced labels. The need to scale labeling is acute but particularly challenging in medical applications like pathology, due to the expertise required to generate quality labels and the limited availability of qualified experts. In this paper we investigate the application of Scalable Variational Gaussian Processes for Crowdsourcing (SVGPCR) in digital pathology. We compare SVGPCR with other crowdsourcing methods using a large multi-rater dataset where pathologists, pathology residents, and medical students annotated tissue regions breast cancer. Our study shows that SVGPCR is competitive with equivalent methods trained using gold-standard pathologist generated labels, and that SVGPCR meets or exceeds the performance of other crowdsourcing methods based on deep learning. We also show how SVGPCR can effectively learn the class-conditional reliabilities of individual annotators and demonstrate that Gaussian-process classifiers have comparable performance to similar deep learning methods. These results suggest that SVGPCR can meaningfully engage non-experts in pathology labeling tasks, and that the class-conditional reliabilities estimated by SVGPCR may assist in matching annotators to tasks where they perform well.

Objective: To utilize a novel crowdsourcing method to measure financial toxicity and its effects among a national cohort of gynecologic cancer patients., Methods: Crowdsourcing methods were used to administer an online survey to women in the United States with gynecologic cancers. We used the Comprehensive Score for Financial Toxicity (COST) tool to measure financial toxicity and the EQ-5D-3L to measure quality of life (QOL). Based on prior work, we defined high financial toxicity as a COST score ≤ 23. We assessed correlation of COST scores with QOL. We used log-binomial regression to examine associations between high financial toxicity and cost-coping strategies., Results: Among the final study sample of 334 respondents, 87% were white, median age at diagnosis was 55 (interquartile range 47-63), 52% had stage III or IV disease and 90% had private insurance or Medicare. Median COST score was 24 (interquartile range 15-32) and 49% of respondents reported high financial toxicity. Greater financial toxicity was correlated with worse QOL (p < 0.001). Participants reporting high financial toxicity were more likely to use cost-coping strategies, including spending less on basic goods (RR: 3.3; 95% CI: 2.1-5.1), borrowing money or applying for financial assistance (RR: 4.0; 95% CI: 2.4-6.9), and delaying or avoiding care (RR: 5.6; 95% CI: 2.6-12.1)., Conclusions: Crowdsourcing is an effective tool to measure financial toxicity. Nearly half of respondents reported high financial toxicity, which was significantly associated with worse QOL, utilization of cost-coping strategies and delays or avoidance of care., Competing Interests: Declaration of Competing Interest The authors report no conflict of interest., (Copyright © 2021. Published by Elsevier Inc.)

Background and Objectives: Filtering the deluge of new research to facilitate evidence synthesis has proven to be unmanageable using current paradigms of search and retrieval. Crowdsourcing, a way of harnessing the collective effort of a "crowd" of people, has the potential to support evidence synthesis by addressing this information overload created by the exponential growth in primary research outputs. Cochrane Crowd, Cochrane's citizen science platform, offers a range of tasks aimed at identifying studies related to health care. Accompanying each task are brief, interactive training modules, and agreement algorithms that help ensure accurate collective decision-making.The aims of the study were to evaluate the performance of Cochrane Crowd in terms of its accuracy, capacity, and autonomy and to examine contributor engagement across three tasks aimed at identifying randomized trials., Study Design and Setting: Crowd accuracy was evaluated by measuring the sensitivity and specificity of crowd screening decisions on a sample of titles and abstracts, compared with "quasi gold-standard" decisions about the same records using the conventional methods of dual screening. Crowd capacity, in the form of output volume, was evaluated by measuring the number of records processed by the crowd, compared with baseline. Crowd autonomy, the capability of the crowd to produce accurate collectively derived decisions without the need for expert resolution, was measured by the proportion of records that needed resolving by an expert., Results: The Cochrane Crowd community currently has 18,897 contributors from 163 countries. Collectively, the Crowd has processed 1,021,227 records, helping to identify 178,437 reports of randomized controlled trials (RCTs) for Cochrane's Central Register of Controlled Trials. The sensitivity for each task was 99.1% for the RCT identification task (RCT ID), 99.7% for the RCT identification task of trials from ClinicalTrials.gov (CT ID), and 97.7% for the identification of RCTs from the International Clinical Trials Registry Platform (ICTRP ID). The specificity for each task was 99% for RCT ID, 98.6% for CT ID, and 99.1% for CT ICTRP ID. The capacity of the combined Crowd and machine learning workflow has increased fivefold in 6 years, compared with baseline. The proportion of records requiring expert resolution across the tasks ranged from 16.6% to 19.7%., Conclusion: Cochrane Crowd is sufficiently accurate and scalable to keep pace with the current rate of publication (and registration) of new primary studies. It has also proved to be a popular, efficient, and accurate way for a large number of people to play an important voluntary role in health evidence production. Cochrane Crowd is now an established part of Cochrane's effort to manage the deluge of primary research being produced., (Copyright © 2021 The Authors. Published by Elsevier Inc. All rights reserved.)

Measuring airways in chest computed tomography (CT) scans is important for characterizing diseases such as cystic fibrosis, yet very time-consuming to perform manually. Machine learning algorithms offer an alternative, but need large sets of annotated scans for good performance. We investigate whether crowdsourcing can be used to gather airway annotations. We generate image slices at known locations of airways in 24 subjects and request the crowd workers to outline the airway lumen and airway wall. After combining multiple crowd workers, we compare the measurements to those made by the experts in the original scans. Similar to our preliminary study, a large portion of the annotations were excluded, possibly due to workers misunderstanding the instructions. After excluding such annotations, moderate to strong correlations with the expert can be observed, although these correlations are slightly lower than inter-expert correlations. Furthermore, the results across subjects in this study are quite variable. Although the crowd has potential in annotating airways, further development is needed for it to be robust enough for gathering annotations in practice. For reproducibility, data and code are available online: http://github.com/adriapr/crowdairway.git., Competing Interests: The authors have declared that no competing interests exist.

Background: As the number of COVID-19 cases increased precipitously in the United States, policy makers and health officials marshalled their pandemic responses. As the economic impacts multiplied, anecdotal reports noted the increased use of web-based crowdfunding to defray these costs., Objective: We examined the web-based crowdfunding response in the early stage of the COVID-19 pandemic in the United States to understand the incidence of initiation of COVID-19-related campaigns and compare them to non-COVID-19-related campaigns., Methods: On May 16, 2020, we extracted all available data available on US campaigns that contained narratives and were created between January 1 and May 10, 2020, on GoFundMe. We identified the subset of COVID-19-related campaigns using keywords relevant to the COVID-19 pandemic. We explored the incidence of COVID-19-related campaigns by geography, by category, and over time, and we compared the characteristics of the campaigns to those of non-COVID-19-related campaigns after March 11, when the pandemic was declared. We then used a natural language processing algorithm to cluster campaigns by narrative content using overlapping keywords., Results: We found that there was a substantial increase in overall GoFundMe web-based crowdfunding campaigns in March, largely attributable to COVID-19-related campaigns. However, as the COVID-19 pandemic persisted and progressed, the number of campaigns per COVID-19 case declined more than tenfold across all states. The states with the earliest disease burden had the fewest campaigns per case, indicating a lack of a case-dependent response. COVID-19-related campaigns raised more money, had a longer narrative description, and were more likely to be shared on Facebook than other campaigns in the study period., Conclusions: Web-based crowdfunding appears to be a stopgap for only a minority of campaigners. The novelty of an emergency likely impacts both campaign initiation and crowdfunding success, as it reflects the affective response of a community. Crowdfunding activity likely serves as an early signal for emerging needs and societal sentiment for communities in acute distress that could be used by governments and aid organizations to guide disaster relief and policy., (©Sameh Nagui Saleh, Christoph U Lehmann, Richard J Medford. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 09.02.2021.)

Background and Objectives: The Cochrane Central Register of Controlled Trials (CENTRAL) is compiled from a number of sources, including PubMed and Embase. Since 2017, we have increased the number of sources feeding into CENTRAL and improved the efficiency of our processes through the use of application programming interfaces, machine learning, and crowdsourcing.Our objectives were twofold: (1) Assess the effectiveness of Cochrane's centralized search and screening processes to correctly identify references to published reports which are eligible for inclusion in Cochrane systematic reviews of randomized controlled trials (RCTs). (2) Identify opportunities to improve the performance of Cochrane's centralized search and screening processes to identify references to eligible trials., Methods: We identified all references to RCTs (either published journal articles or trial registration records) with a publication or registration date between 1st January 2017 and 31st December 2018 that had been included in a Cochrane intervention review. We then viewed an audit trail for each included reference to determine if it had been identified by our centralized search process and subsequently added to CENTRAL., Results: We identified 650 references to included studies with a publication year of 2017 or 2018. Of those, 634 (97.5%) had been captured by Cochrane's Centralised Search Service. Sixteen references had been missed by the Cochrane's Centralised Search Service: six had PubMed-not-MEDLINE status, four were missed by the centralized Embase search, three had been misclassified by Cochrane Crowd, one was from a journal not indexed in MEDLINE or Embase, one had only been added to Embase in 2019, and one reference had been rejected by the automated RCT machine learning classifier. Of the sixteen missed references, eight were the main or only publication to the trial in the review in which it had been included., Conclusion: This analysis has shown that Cochrane's centralized search and screening processes are highly sensitive. It has also helped us to understand better why some references to eligible RCTs have been missed. The CSS is playing a critical role in helping to populate CENTRAL and is moving us toward making CENTRAL a comprehensive repository of RCTs., (Copyright © 2020 The Authors. Published by Elsevier Inc. All rights reserved.)

Background: Crowdfunding has become a unique response to the challenge of health care expenses, yet it has been rarely studied by the medical community. We looked to describe the scope of crowdfunding in thyroid surgery and analyze the factors that contribute toward a successful campaign., Methods: In November 2018, active campaigns were retrieved from a popular crowdfunding Web site using search terms thyroidectomy and thyroid surgery and filtered to include only campaigns that originated in the United States., Results: About 1052 thyroid surgery-related campaigns were analyzed. About 836 (79.5%) involved female patients and 43 (4.1%) pediatric patients. About 792 campaigns (75.3%) referred to thyroid cancer as a primary condition, 163 (15.5%) benign thyroid disease, and 97 (9.2%) other conditions. The average amount raised per campaign was $2514.54 (range, $0-$53,160). About 338 (32.1%) campaigns were self-posted, 317 (30.1%) posted by family, and 397 (37.7%) posted by friends. Median campaign duration was 20 mo, with a median number of 16 donors, 17 hearts, and 136 social media shares. Campaigns related to thyroid cancer raised more funds ($2729.97) than benign ($1669.84) or other ($2175.03) conditions (P < 0.001). Campaigns submitted by friends ($3524.78) received more funding than those by self ($1672.48) or family ($2147.19) (P < 0.001). Campaign duration, donor number, share number, and hearts were also significant predictors of amount raised., Conclusions: For thyroid surgery-related crowdfunding, campaigns referring to thyroid cancer had the highest amount of funds raised. Campaigns created by friends and other factors related to increased community engagement such as social media shares were also related to increased funds., (Copyright © 2020 Elsevier Inc. All rights reserved.)

Background: As the outbreak of coronavirus disease 2019 (COVID-19) progresses, epidemiological data are needed to guide situational awareness and intervention strategies. Here we describe efforts to compile and disseminate epidemiological information on COVID-19 from news media and social networks., Methods: In this population-level observational study, we searched DXY.cn, a health-care-oriented social network that is currently streaming news reports on COVID-19 from local and national Chinese health agencies. We compiled a list of individual patients with COVID-19 and daily province-level case counts between Jan 13 and Jan 31, 2020, in China. We also compiled a list of internationally exported cases of COVID-19 from global news media sources (Kyodo News, The Straits Times, and CNN), national governments, and health authorities. We assessed trends in the epidemiology of COVID-19 and studied the outbreak progression across China, assessing delays between symptom onset, seeking care at a hospital or clinic, and reporting, before and after Jan 18, 2020, as awareness of the outbreak increased. All data were made publicly available in real time., Findings: We collected data for 507 patients with COVID-19 reported between Jan 13 and Jan 31, 2020, including 364 from mainland China and 143 from outside of China. 281 (55%) patients were male and the median age was 46 years (IQR 35-60). Few patients (13 [3%]) were younger than 15 years and the age profile of Chinese patients adjusted for baseline demographics confirmed a deficit of infections among children. Across the analysed period, delays between symptom onset and seeking care at a hospital or clinic were longer in Hubei province than in other provinces in mainland China and internationally. In mainland China, these delays decreased from 5 days before Jan 18, 2020, to 2 days thereafter until Jan 31, 2020 (p=0·0009). Although our sample captures only 507 (5·2%) of 9826 patients with COVID-19 reported by official sources during the analysed period, our data align with an official report published by Chinese authorities on Jan 28, 2020., Interpretation: News reports and social media can help reconstruct the progression of an outbreak and provide detailed patient-level data in the context of a health emergency. The availability of a central physician-oriented social network facilitated the compilation of publicly available COVID-19 data in China. As the outbreak progresses, social media and news reports will probably capture a diminishing fraction of COVID-19 cases globally due to reporting fatigue and overwhelmed health-care systems. In the early stages of an outbreak, availability of public datasets is important to encourage analytical efforts by independent teams and provide robust evidence to guide interventions., Funding: Fogarty International Center, US National Institutes of Health., (© 2020 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY 4.0 license.)

Introduction: The majority of Thailand's road traffic deaths occur on motorised two-wheeled or three-wheeled vehicles. Accurately measuring helmet use is important for the evaluation of new legislation and enforcement. Current methods for estimating helmet use involve roadside observation or surveillance of police and hospital records, both of which are time-consuming and costly. Our objective was to develop a novel method of estimating motorcycle helmet use., Methods: Using Google Maps, 3000 intersections in Bangkok were selected at random. At each intersection, hyperlinks of four images 90° apart were extracted. These 12 000 images were processed in Amazon Mechanical Turk using crowdsourcing to identify images containing motorcycles. The remaining images were sorted manually to determine helmet use., Results: After processing, 462 unique motorcycle drivers were analysed. The overall helmet wearing rate was 66.7 % (95% CI 62.6 % to 71.0 %). Taxi drivers had higher helmet use, 88.4% (95% CI 78.4% to 94.9%), compared with non-taxi drivers, 62.8% (95% CI 57.9% to 67.6%). Helmet use on non-residential roads, 85.2% (95% CI 78.1 % to 90.7%), was higher compared with residential roads, 58.5% (95% CI 52.8% to 64.1%). Using logistic regression, the odds of a taxi driver wearing a helmet compared with a non-taxi driver was significantly increased 1.490 (p<0.01). The odds of helmet use on non-residential roads as compared with residential roads was also increased at 1.389 (p<0.01)., Conclusion: This novel method of estimating helmet use has produced results similar to traditional methods. Applying this technology can reduce time and monetary costs and could be used anywhere street imagery is used. Future directions include automating this process through machine learning., Competing Interests: Competing interests: None declared., (© Author(s) (or their employer(s)) 2020. No commercial re-use. See rights and permissions. Published by BMJ.)

Background: Crowdsourcing is used increasingly in health and medical research. Crowdsourcing is the process of aggregating crowd wisdom to solve a problem. The purpose of this systematic review is to summarize quantitative evidence on crowdsourcing to improve health., Methods: We followed Cochrane systematic review guidance and systematically searched seven databases up to September 4th 2019. Studies were included if they reported on crowdsourcing and related to health or medicine. Studies were excluded if recruitment was the only use of crowdsourcing. We determined the level of evidence associated with review findings using the GRADE approach., Results: We screened 3508 citations, accessed 362 articles, and included 188 studies. Ninety-six studies examined effectiveness, 127 examined feasibility, and 37 examined cost. The most common purposes were to evaluate surgical skills (17 studies), to create sexual health messages (seven studies), and to provide layperson cardio-pulmonary resuscitation (CPR) out-of-hospital (six studies). Seventeen observational studies used crowdsourcing to evaluate surgical skills, finding that crowdsourcing evaluation was as effective as expert evaluation (low quality). Four studies used a challenge contest to solicit human immunodeficiency virus (HIV) testing promotion materials and increase HIV testing rates (moderate quality), and two of the four studies found this approach saved money. Three studies suggested that an interactive technology system increased rates of layperson initiated CPR out-of-hospital (moderate quality). However, studies analyzing crowdsourcing to evaluate surgical skills and layperson-initiated CPR were only from high-income countries. Five studies examined crowdsourcing to inform artificial intelligence projects, most often related to annotation of medical data. Crowdsourcing was evaluated using different outcomes, limiting the extent to which studies could be pooled., Conclusions: Crowdsourcing has been used to improve health in many settings. Although crowdsourcing is effective at improving behavioral outcomes, more research is needed to understand effects on clinical outcomes and costs. More research is needed on crowdsourcing as a tool to develop artificial intelligence systems in medicine., Trial Registration: PROSPERO: CRD42017052835. December 27, 2016.

Background: Social media has become an indispensable tool for patients to learn about aesthetic surgery. Currently, procedure-specific patient preferences for social media platforms and content are unknown., Objectives: The authors sought to evaluate social media preferences of patients seeking aesthetic surgery., Methods: We utilized a choice-based conjoint analysis survey to analyze the preferences of patients seeking 3 common aesthetic procedures: breast augmentation (BA), facial rejuvenation (FR), and combined breast/abdominal surgery (BAB). Participants were asked to choose among social media platforms (Facebook, Twitter, Instagram, Snapchat, Pinterest, Tumblr, YouTube), information extent (basic, moderate, comprehensive), delivery mechanism (prerecorded video, live video, photographs, text description), messenger (surgeon, nurse/clinic staff, patient), and option for interactivity (yes/no). The survey was administered using an Internet crowdsourcing service (Amazon Mechanical Turk)., Results: A total of 647 participants were recruited: 201 in BA, 255 in FR, and 191 in BAB. Among attributes surveyed, participants in all 3 groups (BA, FR, BAB) valued social media platform as the most important (30.9%, 33.1%, 31.4%), followed by information extent (23.1%, 22.9%, 21.6%), delivery mechanism (18.9%, 17.4%, 18%), messenger (16%, 17%, 17.2%), and interactivity (11.1%, 9.8%, 11.8%). Within these attributes, Facebook ranked as the preferred platform, with comprehensive information extent, live video as the delivery mechanism, and surgeon as the messenger as most preferred., Conclusions: The choice of social media platform is the most important factor for patients, and they indicated a preference for comprehensive information delivered by the surgeon via live video on Facebook. Our study elucidates social media usage in common aesthetic populations, which can help improve aesthetic patient outreach., (© 2018 The American Society for Aesthetic Plastic Surgery, Inc. Reprints and permission: journals.permissions@oup.com.)

Background: Breast reconstruction outcomes have traditionally been measured by evaluating the opinions of patients and surgeons., Objectives: Our goal was to assess the views of the general public., Methods: A survey was designed and distributed through a crowdsourcing website called Amazon Mechanical Turk. Questions assessed participant demographics, personal experience with breast reconstruction, perceptions on breast reconstruction, and opinions regarding aesthetics results. Responses were analyzed using chi-square test., Results: A total of 992 responses were collected. Most participants were female (56.1%), white (32.1%), aged 30 to 39 years (40.4%), and had a bachelor's degree (42.0%). A total of 44.2% had personal experience with breast reconstruction and 25.8% with nipple reconstruction. Several aesthetic and reconstructive factors were significantly favored over others across sex, ethnicity, age group, education level, and personal experience with breast reconstruction. For instance, women were more likely to prefer reconstructed nipples (P < 0.0001), view a breast without a nipple as complete (P = 0.024) and place less importance on nipple shape (P = 0.002). Additionally, those who personally experienced nipple reconstruction were willing to undergo more procedures for a complete nipple-areola complex (P < 0.0001), to increase aesthetic results (P = 0.018), and to increase chances of nipple survival (P = 0.002)., Conclusions: Crowdsourcing can be useful in plastic surgery and has helped identify several key findings. The importance of the nipple in reconstruction has been validated; almost three-quarters of respondents did not view a breast without a nipple as complete. The aesthetic preferences seem to support bilateral nipple-sparing reconstruction when possible. Most importantly, the respondents helped elucidate key differences in perception of aesthetic outcomes., (Published by Oxford University Press on behalf of The American Society for Aesthetic Plastic Surgery 2018.)

Background: Two randomized controlled trials (RCTs) were conducted to explore the utility of the Mechanical Turk (MTurk) crowdsourcing platform to conduct rapid trials evaluating online interventions for unhealthy alcohol use., Methods: Both trials employed a staged recruitment procedure where participants who drank in an unhealthy fashion were identified using a baseline survey and then invited to take part in a 6-month follow-up. Participants in both trials were randomized to receive one of several different online interventions or to a no intervention control condition. In study 1, the online interventions were password protected and only those who accessed the study portal were randomized to condition. In study 2, participants were directed to free-of charge interventions and asked to send a screenshot of the intervention to demonstrate that they had complied., Results: Participants reporting unhealthy alcohol use were recruited fairly rapidly. Large numbers of screeners were completed (Study 1: n = 4910; Study 2: n = 5812), found eligible (Study 1: n = 3741; Study 2: n = 4095), and randomized to condition (Study 1: n = 511; Study 2: n = 878). Fair follow-up rates were observed at 6 months for each study (Study 1: 82%; Study 2: 66%). Neither trial was able to clearly demonstrate that providing access to the online interventions lead to increased reductions in alcohol use as compared to the control group., Conclusions: While recruitment through a crowdsourcing platform is rapid and relatively low cost, it is possible that the lack of impact of the online websites employed in these trials could be due to the source of participants rather than the lack of efficacy of the interventions., Trial Registration: ClinicalTrials.gov # NCT02977026 and NCT03060135 .

Background: Prophylactic mesh placement (PMP) at the time of open abdominal surgery has gained momentum over the last decade. However, there remains an identifiable gap in the literature regarding patient-reported outcomes and qualitative metrics. In effort to gauge the population's understanding or familiarity with PMP, this study provides an educational framework and uses crowdsourcing as a novel means to assess perception among the general population., Methods: A cross-sectional survey study was conducted among the general public to elicit perspectives on PMP. An online crowdsourcing platform was used to capture responses to a questionnaire. Pearson's correlation coefficients, paired t-test, chi-square test, and Fisher's exact tests were performed., Results: Of 433 respondents, 338 (78.1%) were included. Individuals who had previously undergone surgery and those who had prior hernia repair were more likely to choose PMP than surgically naïve patients (P = 0.06)., Conclusions: The majority of respondents support the use of PMP. This study contributes to the existing body of literature on PMP and serves as the first qualitative description to gauge the population's perception and understanding of this surgical technique. Within the evolving health care landscape, understanding quality-of-life measures have become increasingly important in defining successful surgical outcomes. Although the data-driven level-I evidence supports the clinical use of PMP, this study intends to establish a framework for future patient-reported outcome studies., (Copyright © 2017 Elsevier Inc. All rights reserved.)

Introduction: Adherence to a gluten-free (GF) diet is the mainstay of therapy for celiac disease. Until now, those wishing to avoid gluten in restaurants had to rely on menu labels, word of mouth, intuition, and restaurant workers' advice, with a relative dearth of supporting data. We used crowd-sourced data from users of a portable gluten detection device to estimate rates of, and identify risk factors for, gluten contamination of supposed GF restaurant foods., Methods: We analyzed data from a portable gluten detection device (Nima), collected across the United States during an 18-month period by users who opted to share the results of their point-of-care tests. Data were sorted by region, time of day, median household income in the restaurant's vicinity, restaurant genre, and food items. We used the χ test for bivariate analysis and multiple logistic regression for multivariate analysis to identify predictors of gluten detection in restaurant food., Results: There were 5,624 tests, performed by 804 users, in the examined period. Gluten was detected in 32% of GF labeled foods. Rates of gluten detection differed by meal, with 27.2% at breakfast and 34.0% at dinner (P = 0.0008). GF labeled pizza and pasta were most likely to test positive for gluten, with gluten detected in 53.2% of pizza and 50.8% of pasta samples. On multivariate analysis, GF labeled food was less likely to test positive for gluten in the West than in the Northeast United States (odds ratio 0.80; 95% confidence interval 0.67-0.95)., Conclusions: This study of crowd-sourced data suggests that a substantial fraction of GF labeled restaurant foods contain detectable gluten. Although the highly sensitive Nima device may detect gluten at levels <20 parts per million (ppm), leading to gluten exposure of unknown clinical significance, our findings raise a potential concern. In addition, our findings of higher rates of gluten detection in pizza and pasta provide practical data when providing dining strategies for patients with celiac disease.

Purpose: To examine patterns of crowdsourcing on the social media platform Twitter among urologists., Methods: Urologists' public Twitter accounts were reviewed for original posts seeking clinical advice or feedback, and associated reply posts, before and after the 140-character-limit expansion in 2017. Predictors of responses to crowdsourcing requests were determined using multivariable regression. When patient data were posted, we noted whether consent was documented., Results: A total of 276 posts in 23 crowdsourcing requests prior to character-limit expansion were analyzed. Reasons for crowdsourcing included requesting solutions to a clinical dilemma (82 posts, 30%); advice seeking about a surgical plan (77 posts, 28%); surveying colleagues' experiences with a new product (64 posts, 23%); and soliciting feedback about a proposed course of action (53 posts, 19%). Recent completion of training (as a proxy for inexperience) did not appear to disproportionately motivate crowdsourcing; authors' median time in practice was 7 years, and authors practicing for ≤ 7 years initiated 57% of requests. 22 (96%) crowdsourcing requests received ≥ 1 reply. Of 15 requests about a specific patient, eight included imaging, but only one cited patient consent. A second analysis of 184 posts in 17 crowdsourcing requests initiated after character-limit expansion demonstrated significantly more authors replying per request (P = 0.01), but no change in the frequency of patient-specific crowdsourcing or citation of consent., Conclusions: Urologists are leveraging Twitter for crowdsourcing clinical guidance and experiential knowledge. Nearly all requests were answered, suggesting low barriers to entry for novice users. Even after character-limit expansion, dissemination of potentially identifiable patient data remains a concern.

Background: Current World Health Organization recommendations for the management of malaria include the need for a parasitological confirmation prior to triggering appropriate treatment. The use of rapid diagnostic tests (RDTs) for malaria has contributed to a better infection recognition and a more targeted treatment. Nevertheless, low-density infections and parasites that fail to produce HRP2 can cause false-negative RDT results. Microscopy has traditionally been the methodology most commonly used to quantify malaria and characterize the infecting species, but the wider use of this technique remains challenging, as it requires trained personnel and processing capacity., Objective: In this study, the feasibility of an on-line system for remote malaria species identification and differentiation has been investigated by crowdsourcing the analysis of digitalized infected thin blood smears by non-expert observers using a mobile app., Methods: An on-line videogame in which players learned how to differentiate the young trophozoite stage of the five Plasmodium species has been designed. Images were digitalized with a smartphone camera adapted to the ocular of a conventional light microscope. Images from infected red blood cells were cropped and puzzled into an on-line game. During the game, players had to decide the malaria species (Plasmodium falciparum, Plasmodium malariae, Plasmodium vivax, Plasmodium ovale, Plasmodium knowlesi) of the infected cells that were shown in the screen. After 2 months, each player's decisions were analysed individually and collectively., Results: On-line volunteers playing the game made more than 500,000 assessments for species differentiation. Statistically, when the choice of several players was combined (n > 25), they were able to significantly discriminate Plasmodium species, reaching a level of accuracy of 99% for all species combinations, except for P. knowlesi (80%). Non-expert decisions on which Plasmodium species was shown in the screen were made in less than 3 s., Conclusion: These findings show that it is possible to train malaria-naïve non-experts to identify and differentiate malaria species in digitalized thin blood samples. Although the accuracy of a single player is not perfect, the combination of the responses of multiple casual gamers can achieve an accuracy that is within the range of the diagnostic accuracy made by a trained microscopist.

Background: Exposure to adverse childhood experiences (ACEs) increases health risk behavior in adulthood and is a risk for premature mortality. For example, ACEs are associated with both tobacco smoking and obesity, which remain significant health challenges for many adults, despite widespread knowledge about the risks. Objective: The present investigation used a novel online crowdsourcing platform (Amazon.com mechanical turk) to study the relationship between ACEs and later tobacco smoking and obesity. Methods : Participants were recruited based on smoking ( n  = 74 smokers; n  = 75 nonsmokers) and stratified based on obesity ( n  = 52 BMI ≥ 30; n  = 97 BMI < 30). Participants had no recent history of other substance use, except alcohol. The relationship between ACE score and smoking and obesity categories was analyzed using logistic regression. Results : The average age of the sample was 38.6-years old and was mostly female (66.4%), employed (82.6%) and college educated (63.1%). Those with 4+ ACEs had a significantly greater odds of cigarette use. Any ACEs exposure was associated with a significantly greater odds of obesity. Conclusions/Importance : Findings are concordant with previous studies and suggest crowdsourcing is a viable platform for studying ACEs and health behavior. Access to large samples and specific populations provided by crowdsourcing could help examine theoretical models about how exposure to ACEs could be connected to later adoption of high-risk behaviors such as tobacco cigarette smoking and obesity.

There is an increasing evidence that smallholder farms contribute substantially to food production globally, yet spatially explicit data on agricultural field sizes are currently lacking. Automated field size delineation using remote sensing or the estimation of average farm size at subnational level using census data are two approaches that have been used. However, both have limitations, for example, automatic field size delineation using remote sensing has not yet been implemented at a global scale while the spatial resolution is very coarse when using census data. This paper demonstrates a unique approach to quantifying and mapping agricultural field size globally using crowdsourcing. A campaign was run in June 2017, where participants were asked to visually interpret very high resolution satellite imagery from Google Maps and Bing using the Geo-Wiki application. During the campaign, participants collected field size data for 130 K unique locations around the globe. Using this sample, we have produced the most accurate global field size map to date and estimated the percentage of different field sizes, ranging from very small to very large, in agricultural areas at global, continental, and national levels. The results show that smallholder farms occupy up to 40% of agricultural areas globally, which means that, potentially, there are many more smallholder farms in comparison with the two different current global estimates of 12% and 24%. The global field size map and the crowdsourced data set are openly available and can be used for integrated assessment modeling, comparative studies of agricultural dynamics across different contexts, for training and validation of remote sensing field size delineation, and potential contributions to the Sustainable Development Goal of Ending hunger, achieve food security and improved nutrition and promote sustainable agriculture., (© 2018 The Authors. Global Change Biology Published by John Wiley & Sons Ltd.)

Objective: To determine the consistency with which government-issued hospital quality ratings and crowdsourced ratings on social media sites identify hospital quality., Data Sources: Hospital ratings from Facebook, Google, and Yelp were linked with Hospital Compare (HC) measures., Study Design: Fixed-effects linear regression model., Principal Findings: Among crowdsourcing sites' best-ranked hospitals, 50-60% were also the best ranked on HC's overall and patient experience ratings; 20% ranked as the worst. Best-ranked hospitals had significantly better clinical quality scores than worst ranked hospitals, but were not more likely to be the highest rated in terms of HC's clinical quality measures alone., Conclusions: Crowdsourcing sites and HC provide comparable information on patient experience; scores were less consistent in terms of risk-adjusted measures of patient safety and clinical quality., (© Health Research and Educational Trust.)

In this study we developed and evaluated a crowdsourcing-based latent semantic analysis (LSA) approach to computerized summary scoring (CSS). LSA is a frequently used mathematical component in CSS, where LSA similarity represents the extent to which the to-be-graded target summary is similar to a model summary or a set of exemplar summaries. Researchers have proposed different formulations of the model summary in previous studies, such as pregraded summaries, expert-generated summaries, or source texts. The former two methods, however, require substantial human time, effort, and costs in order to either grade or generate summaries. Using source texts does not require human effort, but it also does not predict human summary scores well. With human summary scores as the gold standard, in this study we evaluated the crowdsourcing LSA method by comparing it with seven other LSA methods that used sets of summaries from different sources (either experts or crowdsourced) of differing quality, along with source texts. Results showed that crowdsourcing LSA predicted human summary scores as well as expert-good and crowdsourcing-good summaries, and better than the other methods. A series of analyses with different numbers of crowdsourcing summaries demonstrated that the number (from 10 to 100) did not significantly affect performance. These findings imply that crowdsourcing LSA is a promising approach to CSS, because it saves human effort in generating the model summary while still yielding comparable performance. This approach to small-scale CSS provides a practical solution for instructors in courses, and also advances research on automated assessments in which student responses are expected to semantically converge on subject matter content.

Background: As the popularity of aesthetic gluteoplasty continues to grow, there is renewed focus on defining the ideal buttocks. However, the literature lacks studies characterizing an ideal thigh, despite the impact of thigh contour on overall gluteal aesthetic., Objectives: The authors performed the first population analysis of the characteristics of perception of attractive thighs, to identify a role for fat grafting of the thigh in gluteoplasty., Methods: Survey images were digitally modified to create thighs of varying widths and angles relative to fixed buttocks. Thigh-to-buttock ratios and the buttock-thigh junction were studied. Data were stratified and analyzed according to age, gender, and ethnicity of the respondents. Amazon Mechanical Turk was used as a novel crowdsourcing platform for surveying aesthetic preferences., Results: A total of 1034 responses were included of whom 54.4% were male, and 45.6% were female. All age groups and ethnicities were represented. Overall, 43.8% of respondents preferred the widest buttock-thick junction angle on posterior view. There was no clear preference between larger or smaller thigh-to-hip ratios on lateral view., Conclusions: Characteristics of the ideal thigh include wider thighs with greater horizontal projection, creating a more natural contour from the augmented buttock. These findings represent a paradigm shift from the traditionally assumed preference for slender thighs. Plastic surgeons should carefully consider thigh anatomy in their gluteal augmentation patients, as simultaneous thigh augmentation may lead to a more aesthetically pleasing outcome. Further research is needed into best practices and techniques to attain ideal thigh proportions.

Crowdfunding for medical expenses is growing in popularity. Through several websites, patients create public campaign profiles to which donors can contribute. Research on medical crowdfunding is limited, and there have been no studies of crowdfunding in orthopedics. Active medical crowdfunding campaigns for orthopedic trauma, total joint arthroplasty, and spine surgery were queried from a crowdfunding website. The characteristics and outcomes of crowdfunding campaigns were abstracted and analyzed. For this study, 444 campaigns were analyzed, raising a total of $1,443,528. Among the campaigns that received a donation, mean amount raised was $4414 (SE, $611). Multivariate analysis showed that campaigns with unspecified location (odds ratio, 0.26; P=.0008 vs West) and those for total joint arthroplasty (odds ratio, 0.35; P=.0003 vs trauma) had significantly lower odds of receipt of any donation. Description length was associated with higher odds of donation receipt (odds ratio, 1.13 per +100 characters; P<.0001). Among campaigns that received any donation, those with Southern location (-65.5%, P<.0001), international location (-68.5%, P=.0028), and unspecified location (-63.5%, P=.0039) raised lower amounts compared with campaigns in the West. Goal amount was associated with higher amount raised (+3.2% per +$1000, P<.0001). Resources obtained through crowdfunding may be disproportionately available to patients with specific diagnoses, those from specific regions, those who are able to craft a lengthy descriptive narrative, and those with access to robust digital social networks. Clinicians are likely to see a greater proportion of patients turning to crowdfunding as it grows in popularity. Patients may ask physicians for information about crowdfunding or request testimonials to support campaigns. Surgeons should consider their response to such requests individually. These findings shed light on the dynamics of medical crowdfunding and support robust personal and professional deliberation. [Orthopedics. 2018; 41(1):e58-e63.]., (Copyright 2017, SLACK Incorporated.)

Crowdsourcing systems are complex not only because of the huge number of potential strategies for assigning workers to tasks, but also due to the dynamic characteristics associated with workers. Maximizing social welfare in such situations is known to be NP-hard. To address these fundamental challenges, we propose the surprise-minimization-value-maximization (SMVM) approach. By analysing typical crowdsourcing system dynamics, we established a simple and novel worker desirability index (WDI) jointly considering the effect of each worker's reputation, workload and motivation to work on collective productivity. Through evaluating workers' WDI values, SMVM influences individual workers in real time about courses of action which can benefit the workers and lead to high collective productivity. Solutions can be produced in polynomial time and are proven to be asymptotically bounded by a theoretical optimal solution. High resolution simulations based on a real-world dataset demonstrate that SMVM significantly outperforms state-of-the-art approaches. A large-scale 3-year empirical study involving 1,144 participants in over 9,000 sessions shows that SMVM outperforms human task delegation decisions over 80% of the time under common workload conditions. The approach and results can help engineer highly scalable data-driven algorithmic management decision support systems for crowdsourcing.

For Americans experiencing illnesses and disabilities, crowdfunding has become a popular strategy for addressing the extraordinary costs of health care. The political, social, and health consequences of austerity--along with fallout from the 2008 financial collapse and the shortcomings of the Affordable Care Act (ACA)--are made evident in websites like GoFundMe. Here, patients and caregivers create campaigns to solicit donations for medical care, hoping that they will spread widely through social networks. As competition increases among campaigns, patients and their loved ones are obliged to produce compelling and sophisticated appeals. Despite the growing popularity of crowdfunding, little research has explored the usage, impacts, or consequences of the increasing reliance on it for health in the U.S. or abroad. This paper analyzes data from a mixed-methods study conducted from March-September 2016 of 200 GoFundMe campaigns, identified through randomized selection. In addition to presenting exploratory quantitative data on the characteristics and relative success of these campaigns, a more in-depth textual analysis examines how crowdfunders construct narratives about illness and financial need, and attempt to demonstrate their own deservingness. Concerns with the financial burdens of illness, combined with a high proportion of campaigns in states without ACA Medicaid expansion, underscored the importance of crowdfunding as a response to contexts of austerity. Successful crowdfunding requires that campaigners master medical and media literacies; as such, we argue that crowdfunding has the potential to deepen social and health inequities in the U.S. by promoting forms of individualized charity that rely on unequally-distributed literacies to demonstrate deservingness and worth. Crowdfunding narratives also distract from crises of healthcare funding and gaping holes in the social safety net by encouraging hyper-individualized accounts of suffering on media platforms where precarity is portrayed as the result of inadequate self-marketing, rather than the inevitable consequences of structural conditions of austerity., (Copyright © 2017 Elsevier Ltd. All rights reserved.)

Introduction: Adverse events associated with medications are under-reported in postmarketing surveillance systems. A systematic review of published data from 37 studies worldwide (including Canada) found the median under-reporting rate of adverse events to be 94% in spontaneous reporting systems. This scoping review aims to assess the utility of social media and crowd-sourced data to detect and monitor adverse events related to health products including pharmaceuticals, medical devices, biologics and natural health products., Methods and Analysis: Our review conduct will follow the Joanna Briggs Institute scoping review methods manual. Literature searches were conducted in MEDLINE, EMBASE and the Cochrane Library from inception to 13 May 2016. Additional sources included searches of study registries, conference abstracts, dissertations, as well as websites of international regulatory authorities (eg, Food and Drug Administration (FDA), the WHO, European Medicines Agency). Search results will be supplemented by scanning the references of relevant reviews. We will include all publication types including published articles, editorials, websites and book sections that describe use of social media and crowd-sourced data for surveillance of adverse events associated with health products. Two reviewers will perform study selection and data abstraction independently, and discrepancies will be resolved through discussion. Data analysis will involve quantitative (eg, frequencies) and qualitative (eg, content analysis) methods., Dissemination: The summary of results will be sent to Health Canada, who commissioned the review, and other relevant policymakers involved with the Drug Safety and Effectiveness Network. We will compile and circulate a 1-page policy brief and host a 1-day stakeholder meeting to discuss the implications, key messages and finalise the knowledge translation strategy. Findings from this review will ultimately inform the design and development of a data analytics platform for social media and crowd-sourced data for pharmacovigilance in Canada and internationally., Registration Details: Our protocol was registered prospectively with the Open Science Framework (https://osf.io/kv9hu/)., Competing Interests: Conflicts of Interest: None declared., (Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.)

The use of crowdsourcing platforms such as Amazon's Mechanical Turk (MTurk) for data collection in the behavioral sciences has increased substantially in the past several years due in large part to (a) the ability to recruit large samples, (b) the inexpensiveness of data collection, (c) the speed of data collection, and (d) evidence that the data collected are, for the most part, of equal or better quality to that collected in undergraduate research pools. In this review, we first evaluate the strengths and potential limitations of this approach to data collection. Second, we examine how MTurk has been used to date in personality disorder (PD) research and compare the characteristics of such research to PD research conducted in other settings. Third, we compare PD trait data from the Section III trait model of the DSM-5 collected via MTurk to data collected using undergraduate and clinical samples with regard to internal consistency, mean-level differences, and factor structure. Overall, we conclude that platforms such as MTurk have much to offer PD researchers, especially for certain kinds of research (e.g., where large samples are required and there is a need for iterative sampling). Whether MTurk itself remains the predominant model of such platforms is unclear, however, and will largely depend on decisions related to cost effectiveness and the development of alternatives that offer even greater flexibility. (PsycINFO Database Record, ((c) 2017 APA, all rights reserved).)

Much of the data used to measure conflict is extracted from news reports. This is typically accomplished using either expert coders to quantify the relevant information or machine coders to automatically extract data from documents. Although expert coding is costly, it produces quality data. Machine coding is fast and inexpensive, but the data are noisy. To diminish the severity of this tradeoff, we introduce a method for analyzing news documents that uses crowdsourcing, supplemented with computational approaches. The new method is tested on documents about Militarized Interstate Disputes, and its accuracy ranges between about 68 and 76 percent. This is shown to be a considerable improvement over automated coding, and to cost less and be much faster than expert coding.

Purpose: The BLUS (Basic Laparoscopic Urologic Skills) consortium sought to address the construct validity of BLUS tasks and the wider problem of accurate, scalable and affordable skill evaluation by investigating the concordance of 2 novel candidate methods with faculty panel scores, those of automated motion metrics and crowdsourcing., Materials and Methods: A faculty panel of surgeons (5) and anonymous crowdworkers blindly reviewed a randomized sequence of a representative sample of 24 videos (12 pegboard and 12 suturing) extracted from the BLUS validation study (454) using the GOALS (Global Objective Assessment of Laparoscopic Skills) survey tool with appended pass-fail anchors via the same web based user interface. Pre-recorded motion metrics (tool path length, jerk cost etc) were available for each video. Cronbach's alpha, Pearson's R and ROC with AUC statistics were used to evaluate concordance between continuous scores, and as pass-fail criteria among the 3 groups of faculty, crowds and motion metrics., Results: Crowdworkers provided 1,840 ratings in approximately 48 hours, 60 times faster than the faculty panel. The inter-rater reliability of mean expert and crowd ratings was good (α=0.826). Crowd score derived pass-fail resulted in 96.9% AUC (95% CI 90.3-100; positive predictive value 100%, negative predictive value 89%). Motion metrics and crowd scores provided similar or nearly identical concordance with faculty panel ratings and pass-fail decisions., Conclusions: The concordance of crowdsourcing with faculty panels and speed of reviews is sufficiently high to merit its further investigation alongside automated motion metrics. The overall agreement among faculty, motion metrics and crowdworkers provides evidence in support of the construct validity for 2 of the 4 BLUS tasks., (Copyright © 2016 American Urological Association Education and Research, Inc. Published by Elsevier Inc. All rights reserved.)

Background: Crowdsourcing, the process of shifting individual tasks to a large group, may enhance human immunodeficiency virus (HIV) testing interventions. We conducted a noninferiority, randomized controlled trial to compare first-time HIV testing rates among men who have sex with men (MSM) and transgender individuals who received a crowdsourced or a health marketing HIV test promotion video., Methods: Seven hundred twenty-one MSM and transgender participants (≥16 years old, never before tested for HIV) were recruited through 3 Chinese MSM Web portals and randomly assigned to 1 of 2 videos. The crowdsourced video was developed using an open contest and formal transparent judging while the evidence-based health marketing video was designed by experts. Study objectives were to measure HIV test uptake within 3 weeks of watching either HIV test promotion video and cost per new HIV test and diagnosis., Results: Overall, 624 of 721 (87%) participants from 31 provinces in 217 Chinese cities completed the study. HIV test uptake was similar between the crowdsourced arm (37% [114/307]) and the health marketing arm (35% [111/317]). The estimated difference between the interventions was 2.1% (95% confidence interval, -5.4% to 9.7%). Among those tested, 31% (69/225) reported a new HIV diagnosis. The crowdsourced intervention cost substantially less than the health marketing intervention per first-time HIV test (US$131 vs US$238 per person) and per new HIV diagnosis (US$415 vs US$799 per person)., Conclusions: Our nationwide study demonstrates that crowdsourcing may be an effective tool for improving HIV testing messaging campaigns and could increase community engagement in health campaigns., Clinical Trials Registration: NCT02248558., (© The Author 2016. Published by Oxford University Press for the Infectious Diseases Society of America. All rights reserved. For permissions, e-mail journals.permissions@oup.com.)

HowNutsAreTheDutch (Dutch: HoeGekIsNL) is a national crowdsourcing study designed to investigate multiple continuous mental health dimensions in a sample from the general population (n = 12,503). Its main objective is to create an empirically based representation of mental strengths and vulnerabilities, accounting for (i) dimensionality and heterogeneity, (ii) interactivity between symptoms and strengths, and (iii) intra-individual variability. To do so, HowNutsAreTheDutch (HND) makes use of an internet platform that allows participants to (a) compare themselves to other participants via cross-sectional questionnaires and (b) to monitor themselves three times a day for 30 days with an intensive longitudinal diary study via their smartphone. These data enable for personalized feedback to participants, a study of profiles of mental strengths and weaknesses, and zooming into the fine-grained level of dynamic relationships between variables over time. Measuring both psychiatric symptomatology and mental strengths and resources enables for an investigation of their interactions, which may underlie the wide variety of observed mental states in the population. The present paper describes the applied methods and technology, and presents the sample characteristics. Copyright © 2015 John Wiley & Sons, Ltd., (Copyright © 2015 John Wiley & Sons, Ltd.)

This paper focuses on the role of the formal response community's use of social media and crowdsourcing for emergency managers (EMs) in disaster planning, response and recovery in the United States. In-depth qualitative interviews with EMs on the Eastern seaboard at the local, state and federal level demonstrate that emergency management tools are in a state of transition--from formal, internally regulated tools for crisis response to an incorporation of new social media and crowdsourcing tools. The first set of findings provides insight into why many EMs are not using social media, and describes their concerns that result in fear, uncertainty and doubt. Second, this research demonstrates how internal functioning and staffing issues within these agencies present challenges. This research seeks to examine the dynamics of this transition and offer lessons for how to improve its outcomes--critical to millions of people across the United States., (© 2016 The Author(s). Disasters © Overseas Development Institute, 2016.)