Search

Social media companies have struggled to provide a democratically legitimate definition of "Fake News". Reliance on expert judgment has attracted criticism due to a general trust deficit and political polarisation. Approaches reliant on the ``wisdom of the crowds'' are a cost-effective, transparent and inclusive alternative. This paper provides a novel end-to-end methodology to detect fake news on X via "wisdom of the synthetic & representative crowds". We deploy an online survey on the Lucid platform to gather veracity assessments for a number of pandemic-related tweets from crowd-workers. Borrowing from the MrP literature, we train a Hierarchical Bayesian model to predict the veracity of each tweet from the perspective of different personae from the population of interest. We then weight the predicted veracity assessments according to a representative stratification frame, such that decisions about ``fake'' tweets are representative of the overall polity of interest. Based on these aggregated scores, we analyse a corpus of tweets and perform a second MrP to generate state-level estimates of the number of people who share fake news. We find small but statistically meaningful heterogeneity in fake news sharing across US states. At the individual-level: i. sharing fake news is generally rare, with an average sharing probability interval [0.07,0.14]; ii. strong evidence that Democrats share less fake news, accounting for a reduction in the sharing odds of [57.3%,3.9%] relative to the average user; iii. when Republican definitions of fake news are used, it is the latter who show a decrease in the propensity to share fake news worth [50.8%, 2.0%]; iv. some evidence that women share less fake news than men, an effect worth a [29.5%,4.9%] decrease., Comment: 50 pages, 6 figures, 2 tables

We seek to democratise public-opinion research by providing practitioners with a general methodology to make representative inference from cheap, high-frequency, highly unrepresentative samples. We focus specifically on samples which are readily available in moderate sizes. To this end, we provide two major contributions: 1) we introduce a general sample-selection process which we name online selection, and show it is a special-case of selection on the dependent variable. We improve MrP for severely biased samples by introducing a bias-correction term in the style of King and Zeng to the logistic-regression framework. We show this bias-corrected model outperforms traditional MrP under online selection, and achieves performance similar to random-sampling in a vast array of scenarios; 2) we present a protocol to use Large Language Models (LLMs) to extract structured, survey-like data from social-media. We provide a prompt-style that can be easily adapted to a variety of survey designs. We show that LLMs agree with human raters with respect to the demographic, socio-economic and political characteristics of these online users. The end-to-end implementation takes unrepresentative, unsrtuctured social media data as inputs, and produces timely high-quality area-level estimates as outputs. This is Artificially Intelligent Opinion Polling. We show that our AI polling estimates of the 2020 election are highly accurate, on-par with estimates produced by state-level polling aggregators such as FiveThirtyEight, or from MrP models fit to extremely expensive high-quality samples.

Who joins extremist movements? Answering this question poses considerable methodological challenges. Survey techniques are infeasible and selective samples provide no counterfactual. Recruits can be assigned to contextual units, but this is vulnerable to problems of ecological inference. In this article, we take inspiration from epidemiology and elaborate a technique that combines survey and ecological approaches. The multilevel Bayesian case-control design that we propose allows us to identify individual-level and contextual factors patterning the incidence of recruitment, while accounting for rare events, contamination, and spatial autocorrelation. We validate our approach by matching a sample of Islamic State (ISIS) fighters from nine MENA countries with representative population surveys enumerated shortly before recruits joined the movement. High status individuals in their early twenties with university education were more likely to join ISIS. There is more mixed evidence for relative deprivation. We provide software for applied researchers to implement our method.