Search

Detecting organized political campaigns is of paramount importance in fighting against disinformation on social media. Existing approaches for the identification of such organized actions employ techniques mostly from network science, graph machine learning and natural language processing. Their ultimate goal is to analyze the relationships and interactions (e.g. re-posting) among users and the textual similarities of their posts. Despite their effectiveness in recognizing astroturf campaigns, these methods face significant challenges, notably the class imbalance in available training datasets. To mitigate this issue, recent methods usually resort to data augmentation or increasing the number of positive samples, which may not always be feasible or sufficient in real-world settings. Following a different path, in this paper, we propose a novel framework for identifying astroturf campaigns based solely on large language models (LLMs), introducing a Balanced Retrieval-Augmented Generation (Balanced RAG) component. Our approach first gives both textual information concerning the posts (in our case tweets) and the user interactions of the social network as input to a language model. Then, through prompt engineering and the proposed Balanced RAG method, it effectively detects coordinated disinformation campaigns on X (Twitter). The proposed framework does not require any training or fine-tuning of the language model. Instead, by strategically harnessing the strengths of prompt engineering and Balanced RAG, it facilitates LLMs to overcome the effects of class imbalance and effectively identify coordinated political campaigns. The experimental results demonstrate that by incorporating the proposed prompt engineering and Balanced RAG methods, our framework outperforms the traditional graph-based baselines, achieving 2x-3x improvements in terms of precision, recall and F1 scores.

This paper presents the TikTok 2024 U.S. Presidential Election Dataset, a large-scale, resource designed to advance research into political communication and social media dynamics. The dataset comprises 3.14 million videos published on TikTok between November 1, 2023, and October 16, 2024, encompassing video ids and transcripts. Data collection was conducted using the TikTok Research API with a comprehensive set of election-related keywords and hashtags, supplemented by third-party tools to address API limitations and expand content coverage, enabling analysis of hashtag co-occurrence networks that reveal politically aligned hashtags based on ideological affiliations, the evolution of top hashtags over time, and summary statistics that highlight the dataset's scale and richness. This dataset offers insights into TikTok's role in shaping electoral discourse by providing a multimodal view of election-related content. It enables researchers to explore critical topics such as coordinated messaging, misinformation spread, audience engagement, and linguistic trends. The TikTok 2024 U.S. Presidential Election Dataset is publicly available and aims to contribute to the broader understanding of social media's impact on democracy and public opinion.

Social media platforms have become critical spaces for discussing mental health concerns, including eating disorders. While these platforms can provide valuable support networks, they may also amplify harmful content that glorifies disordered cognition and self-destructive behaviors. While social media platforms have implemented various content moderation strategies, from stringent to laissez-faire approaches, we lack a comprehensive understanding of how these different moderation practices interact with user engagement in online communities around these sensitive mental health topics. This study addresses this knowledge gap through a comparative analysis of eating disorder discussions across Twitter/X, Reddit, and TikTok. Our findings reveal that while users across all platforms engage similarly in expressing concerns and seeking support, platforms with weaker moderation (like Twitter/X) enable the formation of toxic echo chambers that amplify pro-anorexia rhetoric. These results demonstrate how moderation strategies significantly influence the development and impact of online communities, particularly in contexts involving mental health and self-harm., Comment: arXiv admin note: text overlap with arXiv:2401.09647

Social media platforms have become vital spaces for public discourse, serving as modern agor\'as where a wide range of voices influence societal narratives. However, their open nature also makes them vulnerable to exploitation by malicious actors, including state-sponsored entities, who can conduct information operations (IOs) to manipulate public opinion. The spread of misinformation, false news, and misleading claims threatens democratic processes and societal cohesion, making it crucial to develop methods for the timely detection of inauthentic activity to protect the integrity of online discourse. In this work, we introduce a methodology designed to identify users orchestrating information operations, a.k.a. \textit{IO drivers}, across various influence campaigns. Our framework, named \texttt{IOHunter}, leverages the combined strengths of Language Models and Graph Neural Networks to improve generalization in \emph{supervised}, \emph{scarcely-supervised}, and \emph{cross-IO} contexts. Our approach achieves state-of-the-art performance across multiple sets of IOs originating from six countries, significantly surpassing existing approaches. This research marks a step toward developing Graph Foundation Models specifically tailored for the task of IO detection on social media platforms., Comment: 9 pages

Social media platforms have become a hub for political activities and discussions, democratizing participation in these endeavors. However, they have also become an incubator for manipulation campaigns, like information operations (IOs). Some social media platforms have released datasets related to such IOs originating from different countries. However, we lack comprehensive control data that can enable the development of IO detection methods. To bridge this gap, we present new labeled datasets about 26 campaigns, which contain both IO posts verified by a social media platform and over 13M posts by 303k accounts that discussed similar topics in the same time frames (control data). The datasets will facilitate the study of narratives, network interactions, and engagement strategies employed by coordinated accounts across various campaigns and countries. By comparing these coordinated accounts against organic ones, researchers can develop and benchmark IO detection algorithms., Comment: 7 pages, 2 figures, 1 table

Approximately 50% of tweets in X's user timelines are personalized recommendations from accounts they do not follow. This raises a critical question: what political content are users exposed to beyond their established networks, and how might this influence democratic discourse online? Due to the black-box nature and constant evolution of social media algorithms, much remains unknown about this aspect of users' content exposure, particularly as it pertains to potential biases in algorithmic curation. Prior research has shown that certain political groups and media sources are amplified within users' in-network tweets. However, the extent to which this amplification affects out-of-network recommendations remains unclear. As the 2024 U.S. Election approaches, addressing this question is essential for understanding the influence of algorithms on online political content consumption and its potential impact on users' perspectives. In this paper, we conduct a three-week audit of X's algorithmic content recommendations using a set of 120 sock-puppet monitoring accounts that capture tweets in their personalized ``For You'' timelines. Our objective is to quantify out-of-network content exposure for right- and left-leaning user profiles and to assess any potential biases in political exposure. Our findings indicate that X's algorithm skews exposure toward a few high-popularity accounts across all users, with right-leaning users experiencing the highest level of exposure inequality. Both left- and right-leaning users encounter amplified exposure to accounts aligned with their own political views and reduced exposure to opposing viewpoints. Additionally, we observe a right-leaning bias in exposure for new accounts within their default timelines., Comment: HUMANS Lab -- Working Paper No. 2024.9 -- The 2024 Election Integrity Initiative -- University of Southern California

Truth Social, launched as a social media platform with a focus on free speech, has become a prominent space for political discourse, attracting a user base with diverse, yet often conservative, viewpoints. As an emerging platform with minimal content moderation, Truth Social has facilitated discussions around contentious social and political issues but has also seen the spread of conspiratorial and hyper-partisan narratives. In this paper, we introduce and release a comprehensive dataset capturing activity on Truth Social related to the upcoming 2024 U.S. Presidential Election, including posts, replies, user interactions, content and media. This dataset comprises 1.5 million posts published between February, 2024 and October 2024, and encompasses key user engagement features and posts metadata. Data collection began in June 2024, though it includes posts published earlier, with the oldest post dating back to February 2022. This offers researchers a unique resource to study communication patterns, the formation of online communities, and the dissemination of information within Truth Social in the run-up to the election. By providing an in-depth view of Truth Social's user dynamics and content distribution, this dataset aims to support further research on political discourse within an alt-tech social media platform. The dataset is publicly available at https://github.com/kashish-s/TruthSocial_2024ElectionInitiative, Comment: HUMANS Lab -- Working Paper No. 2024.8 -- The 2024 Election Integrity Initiative -- University of Southern California

In this paper, we introduce the first release of a large-scale dataset capturing discourse on $\mathbb{X}$ (a.k.a., Twitter) related to the upcoming 2024 U.S. Presidential Election. Our dataset comprises 22 million publicly available posts on X.com, collected from May 1, 2024, to July 31, 2024, using a custom-built scraper, which we describe in detail. By employing targeted keywords linked to key political figures, events, and emerging issues, we aligned data collection with the election cycle to capture evolving public sentiment and the dynamics of political engagement on social media. This dataset offers researchers a robust foundation to investigate critical questions about the influence of social media in shaping political discourse, the propagation of election-related narratives, and the spread of misinformation. We also present a preliminary analysis that highlights prominent hashtags and keywords within the dataset, offering initial insights into the dominant themes and conversations occurring in the lead-up to the election. Our dataset is available at: url{https://github.com/sinking8/usc-x-24-us-election

With its lenient moderation policies and long-standing associations with potentially unlawful activities, Telegram has become an incubator for problematic content, frequently featuring conspiratorial, hyper-partisan, and fringe narratives. In the political sphere, these concerns are amplified by reports of Telegram channels being used to organize violent acts, such as those that occurred during the Capitol Hill attack on January 6, 2021. As the 2024 U.S. election approaches, Telegram remains a focal arena for societal and political discourse, warranting close attention from the research community, regulators, and the media. Based on these premises, we introduce and release a Telegram dataset focused on the 2024 U.S. Presidential Election, featuring over 30,000 chats and half a billion messages, including chat details, profile pictures, messages, and user information. We constructed a network of chats and analyzed the 500 most central ones, examining their shared messages. This resource represents the largest public Telegram dataset to date, offering an unprecedented opportunity to study political discussion on Telegram in the lead-up to the 2024 U.S. election. We will continue to collect data until the end of 2024, and routinely update the dataset released at: https://github.com/leonardo-blas/usc-tg-24-us-election, Comment: HUMANS Lab -- Working Paper No. 2024.5 -- The 2024 Election Integrity Initiative -- University of Southern California

Coordinated information operations remain a persistent challenge on social media, despite platform efforts to curb them. While previous research has primarily focused on identifying these operations within individual platforms, this study shows that coordination frequently transcends platform boundaries. Leveraging newly collected data of online conversations related to the 2024 U.S. Election across $\mathbb{X}$ (formerly, Twitter), Facebook, and Telegram, we construct similarity networks to detect coordinated communities exhibiting suspicious sharing behaviors within and across platforms. Proposing an advanced coordination detection model, we reveal evidence of potential foreign interference, with Russian-affiliated media being systematically promoted across Telegram and $\mathbb{X}$. Our analysis also uncovers substantial intra- and cross-platform coordinated inauthentic activity, driving the spread of highly partisan, low-credibility, and conspiratorial content. These findings highlight the urgent need for regulatory measures that extend beyond individual platforms to effectively address the growing challenge of cross-platform coordinated influence campaigns., Comment: HUMANS Lab -- Working Paper No. 2024.7 -- The 2024 Election Integrity Initiative -- University of Southern California

Information Operations (IOs) pose a significant threat to the integrity of democratic processes, with the potential to influence election-related online discourse. In anticipation of the 2024 U.S. presidential election, we present a study aimed at uncovering the digital traces of coordinated IOs on $\mathbb{X}$ (formerly Twitter). Using our machine learning framework for detecting online coordination, we analyze a dataset comprising election-related conversations on $\mathbb{X}$ from May 2024. This reveals a network of coordinated inauthentic actors, displaying notable similarities in their link-sharing behaviors. Our analysis shows concerted efforts by these accounts to disseminate misleading, redundant, and biased information across the Web through a coordinated cross-platform information operation: The links shared by this network frequently direct users to other social media platforms or suspicious websites featuring low-quality political content and, in turn, promoting the same $\mathbb{X}$ and YouTube accounts. Members of this network also shared deceptive images generated by AI, accompanied by language attacking political figures and symbolic imagery intended to convey power and dominance. While $\mathbb{X}$ has suspended a subset of these accounts, more than 75% of the coordinated network remains active. Our findings underscore the critical role of developing computational models to scale up the detection of threats on large social media platforms, and emphasize the broader implications of these techniques to detect IOs across the wider Web., Comment: First Monday 29(11), 2024

Bluesky is a Twitter-like decentralized social media platform that has recently grown in popularity. After an invite-only period, it opened to the public worldwide on February 6th, 2024. In this paper, we provide a longitudinal analysis of user activity in the two months around the opening, studying changes in the general characteristics of the platform due to the rapid growth of the user base. We observe a broad distribution of activity similar to more established platforms, but a higher volume of original than reshared content, and very low toxicity. After opening to the public, Bluesky experienced a large surge in new users and activity, especially posting English and Japanese content. In particular, several accounts entered the discussion with suspicious behavior, like following many accounts and sharing content from low-credibility news outlets. Some of these have already been classified as spam or suspended, suggesting effective moderation., Comment: Long paper that has been accepted at the ASONAM 2024 conference

Understanding susceptibility to online influence is crucial for mitigating the spread of misinformation and protecting vulnerable audiences. This paper investigates susceptibility to influence within social networks, focusing on the differential effects of influence-driven versus spontaneous behaviors on user content adoption. Our analysis reveals that influence-driven adoption exhibits high homophily, indicating that individuals prone to influence often connect with similarly susceptible peers, thereby reinforcing peer influence dynamics, whereas spontaneous adoption shows significant but lower homophily. Additionally, we extend the Generalized Friendship Paradox to influence-driven behaviors, demonstrating that users' friends are generally more susceptible to influence than the users themselves, de facto establishing the notion of Susceptibility Paradox in online social influence. This pattern does not hold for spontaneous behaviors, where friends exhibit fewer spontaneous adoptions. We find that susceptibility to influence can be predicted using friends' susceptibility alone, while predicting spontaneous adoption requires additional features, such as user metadata. These findings highlight the complex interplay between user engagement and characteristics in spontaneous content adoption. Our results provide new insights into social influence mechanisms and offer implications for designing more effective moderation strategies to protect vulnerable audiences.

Social media platforms actively moderate content glorifying harmful behaviors like eating disorders, which include anorexia and bulimia. However, users have adapted to evade moderation by using coded hashtags. Our study investigates the prevalence of moderation evaders on the popular social media platform TikTok and contrasts their use and emotional valence with mainstream hashtags. We notice that moderation evaders and mainstream hashtags appear together, indicating that vulnerable users might inadvertently encounter harmful content even when searching for mainstream terms. Additionally, through an analysis of emotional expressions in video descriptions and comments, we find that mainstream hashtags generally promote positive engagement, while moderation evaders evoke a wider range of emotions, including heightened negativity. These findings provide valuable insights for content creators, platform moderation efforts, and interventions aimed at cultivating a supportive online environment for discussions on mental health and eating disorders., Comment: 10 pages, 5 figures, 2 tables

Social media has become a crucial conduit for the swift dissemination of information during global crises. However, this also paves the way for the manipulation of narratives by malicious actors. This research delves into the interaction dynamics between coordinated (malicious) entities and organic (regular) users on Twitter amidst the Gaza conflict. Through the analysis of approximately 3.5 million tweets from over 1.3 million users, our study uncovers that coordinated users significantly impact the information landscape, successfully disseminating their content across the network: a substantial fraction of their messages is adopted and shared by organic users. Furthermore, the study documents a progressive increase in organic users' engagement with coordinated content, which is paralleled by a discernible shift towards more emotionally polarized expressions in their subsequent communications. These results highlight the critical need for vigilance and a nuanced understanding of information manipulation on social media platforms.

The kick-off of vaccination campaigns in Europe, starting in late December 2020, has been followed by the online spread of controversies and conspiracies surrounding vaccine validity and efficacy. We study Twitter discussions in three major European languages (Italian, German, and French) during the vaccination campaign. Moving beyond content analysis to explore the structural aspects of online discussions, our investigation includes an analysis of polarization and the potential formation of echo chambers, revealing nuanced behavioral and topical differences in user interactions across the analyzed countries. Notably, we identify strong anti- and pro-vaccine factions exhibiting heterogeneous temporal polarization patterns in different countries. Through a detailed examination of news-sharing sources, we uncover the widespread use of other media platforms like Telegram and YouTube for disseminating low-credibility information, indicating a concerning trend of diminishing news credibility over time. Our findings on Twitter discussions during the COVID-19 vaccination campaign in major European languages expose nuanced behavioral distinctions, revealing the profound impact of polarization and the emergence of distinct anti-vaccine and pro-vaccine advocates over time., Comment: 17 pages (including references), 14 figures, 1 table, to be published at 16th ACM Web Science Conference 2024

Proprietary public APIs play a crucial and growing role as research tools among social scientists. Among such APIs, Google's machine learning-based Perspective API is extensively utilized for assessing the toxicity of social media messages, providing both an important resource for researchers and automatic content moderation. However, this paper exposes an important bias in Perspective API concerning German language text. Through an in-depth examination of several datasets, we uncover intrinsic language biases within the multilingual model of Perspective API. We find that the toxicity assessment of German content produces significantly higher toxicity levels than other languages. This finding is robust across various translations, topics, and data sources, and has significant consequences for both research and moderation strategies that rely on Perspective API. For instance, we show that, on average, four times more tweets and users would be moderated when using the German language compared to their English translation. Our findings point to broader risks associated with the widespread use of proprietary APIs within the computational social sciences., Comment: 12 pages, 12 figures. Please check and cite the published version of this paper in the Proceedings of the 19th AAAI International Conference on Web and Social Media (ICWSM'25)

Internet memes have emerged as a novel format for communication and expressing ideas on the web. Their fluidity and creative nature are reflected in their widespread use, often across platforms and occasionally for unethical or harmful purposes. While computational work has already analyzed their high-level virality over time and developed specialized classifiers for hate speech detection, there have been no efforts to date that aim to holistically track, identify, and map internet memes posted on social media. To bridge this gap, we investigate whether internet memes across social media platforms can be contextualized by using a semantic repository of knowledge, namely, a knowledge graph. We collect thousands of potential internet meme posts from two social media platforms, namely Reddit and Discord, and develop an extract-transform-load procedure to create a data lake with candidate meme posts. By using vision transformer-based similarity, we match these candidates against the memes cataloged in IMKG -- a recently released knowledge graph of internet memes. We leverage this grounding to highlight the potential of our proposed framework to study the prevalence of memes on different platforms, map them to IMKG, and provide context about memes on social media., Comment: 10 pages, 7 figures, 2 tables

Social media influence campaigns pose significant challenges to public discourse and democracy. Traditional detection methods fall short due to the complexity and dynamic nature of social media. Addressing this, we propose a novel detection method using Large Language Models (LLMs) that incorporates both user metadata and network structures. By converting these elements into a text format, our approach effectively processes multilingual content and adapts to the shifting tactics of malicious campaign actors. We validate our model through rigorous testing on multiple datasets, showcasing its superior performance in identifying influence efforts. This research not only offers a powerful tool for detecting campaigns, but also sets the stage for future enhancements to keep up with the fast-paced evolution of social media-based influence tactics.

Misinformation proliferation on social media platforms is a pervasive threat to the integrity of online public discourse. Genuine users, susceptible to others' influence, often unknowingly engage with, endorse, and re-share questionable pieces of information, collectively amplifying the spread of misinformation. In this study, we introduce an empirical framework to investigate users' susceptibility to influence when exposed to unreliable and reliable information sources. Leveraging two datasets on political and public health discussions on Twitter, we analyze the impact of exposure on the adoption of information sources, examining how the reliability of the source modulates this relationship. Our findings provide evidence that increased exposure augments the likelihood of adoption. Users tend to adopt low-credibility sources with fewer exposures than high-credibility sources, a trend that persists even among non-partisan users. Furthermore, the number of exposures needed for adoption varies based on the source credibility, with extreme ends of the spectrum (very high or low credibility) requiring fewer exposures for adoption. Additionally, we reveal that the adoption of information sources often mirrors users' prior exposure to sources with comparable credibility levels. Our research offers critical insights for mitigating the endorsement of misinformation by vulnerable users, offering a framework to study the dynamics of content exposure and adoption on social media platforms.

Social media platforms, particularly Twitter, have become pivotal arenas for influence campaigns, often orchestrated by state-sponsored information operations (IOs). This paper delves into the detection of key players driving IOs by employing similarity graphs constructed from behavioral pattern data. We unveil that well-known, yet underutilized network properties can help accurately identify coordinated IO drivers. Drawing from a comprehensive dataset of 49 million tweets from six countries, which includes multiple verified IOs, our study reveals that traditional network filtering techniques do not consistently pinpoint IO drivers across campaigns. We first propose a framework based on node pruning that emerges superior, particularly when combining multiple behavioral indicators across different networks. Then, we introduce a supervised machine learning model that harnesses a vector representation of the fused similarity network. This model, which boasts a precision exceeding 0.95, adeptly classifies IO drivers on a global scale and reliably forecasts their temporal engagements. Our findings are crucial in the fight against deceptive influence campaigns on social media, helping us better understand and detect them., Comment: Accepted at the 2024 ACM Web Conference

Online hate messaging is a pervasive issue plaguing the well-being of social media users. This research empirically investigates a novel theory positing that online hate may be driven primarily by the pursuit of social approval rather than a direct desire to harm the targets. Results show that toxicity is homophilous in users' social networks and that a user's propensity for hostility can be predicted by their social networks. We also illustrate how receiving greater or fewer social engagements in the form of likes, retweets, quotes, and replies affects a user's subsequent toxicity. We establish a clear connection between receiving social approval signals and increases in subsequent toxicity. Being retweeted plays a particularly prominent role in escalating toxicity. Results also show that not receiving expected levels of social approval leads to decreased toxicity. We discuss the important implications of our research and opportunities to combat online hate.

Despite their drawbacks and unintended consequences, social media networks have recently emerged as a crucial resource for individuals in distress, particularly during times of crisis. These platforms serve as a means to seek assistance and support, share reliable information, and appeal for action and solidarity. In this paper, we examine the online networks of support during the Russia-Ukraine conflict by analyzing four major social media networks: Twitter, Facebook, Instagram, and YouTube. Using a large dataset of 68 million posts, we explore the temporal patterns and interconnectedness between these platforms and online support websites. Our analysis highlights the prevalence of crowdsourcing and crowdfunding websites as the two main support platforms to mobilize resources and solicit donations, revealing their purpose and contents, and investigating different support-seeking and -receiving practices. Overall, our study underscores the potential of social media in facilitating online support in distressed environments through grassroots mobilization, contributing to the growing body of research on the positive impact of online platforms in promoting social good and protecting vulnerable populations during times of crisis and conflict., Comment: Presented at ICWSM2023 Workshop "Data for the Wellbeing of Most Vulnerable"

Detecting misinformation threads is crucial to guarantee a healthy environment on social media. We address the problem using the data set created during the COVID-19 pandemic. It contains cascades of tweets discussing information weakly labeled as reliable or unreliable, based on a previous evaluation of the information source. The models identifying unreliable threads usually rely on textual features. But reliability is not just what is said, but by whom and to whom. We additionally leverage on network information. Following the homophily principle, we hypothesize that users who interact are generally interested in similar topics and spreading similar kind of news, which in turn is generally reliable or not. We test several methods to learn representations of the social interactions within the cascades, combining them with deep neural language models in a Multi-Input (MI) framework. Keeping track of the sequence of the interactions during the time, we improve over previous state-of-the-art models.

The COVID-19 pandemic has intensified numerous social issues that warrant academic investigation. Although information dissemination has been extensively studied, the silenced voices and censored content also merit attention due to their role in mobilizing social movements. In this paper, we provide empirical evidence to explore the relationships among COVID-19 regulations, censorship, and protest through a series of social incidents occurred in China during 2022. We analyze the similarities and differences between censored articles and discussions on r/china\_irl, the most popular Chinese-speaking subreddit, and scrutinize the temporal dynamics of government censorship activities and their impact on user engagement within the subreddit. Furthermore, we examine users' linguistic patterns under the influence of a censorship-driven environment. Our findings reveal patterns in topic recurrence, the complex interplay between censorship activities, user subscription, and collective commenting behavior, as well as potential linguistic adaptation strategies to circumvent censorship. These insights hold significant implications for researchers interested in understanding the survival mechanisms of marginalized groups within censored information ecosystems.

The proliferation of harmful content shared online poses a threat to online information integrity and the integrity of discussion across platforms. Despite various moderation interventions adopted by social media platforms, researchers and policymakers are calling for holistic solutions. This study explores how a target platform could leverage content that has been deemed harmful on a source platform by investigating the behavior and characteristics of Twitter users responsible for sharing moderated YouTube videos. Using a large-scale dataset of 600M tweets related to the 2020 U.S. election, we find that moderated Youtube videos are extensively shared on Twitter and that users who share these videos also endorse extreme and conspiratorial ideologies. A fraction of these users are eventually suspended by Twitter, but they do not appear to be involved in state-backed information operations. The findings of this study highlight the complex and interconnected nature of harmful cross-platform information diffusion, raising the need for cross-platform moderation strategies., Comment: 14 pages, 8 figures

With the spread of high-speed Internet and portable smart devices, the way people access and consume information has drastically changed. However, this presents many challenges, including information overload, personal data leakage, and misinformation diffusion. Across the spectrum of risks that Internet users face nowadays, this work focuses on understanding how young people perceive and deal with false information. Within an experimental campaign involving 183 students, we presented six different news items to the participants and invited them to browse the Internet to assess the veracity of the presented information. Our results suggest that online search is more likely to lead students to validate true news than to refute false ones. We found that students change their opinion about a specific piece of information more often than their global idea about a broader topic. Also, our experiment reflected that most participants rely on online sources to obtain information and access the news, and those getting information from books and Internet browsing are the most accurate in assessing the veracity of a news item. This work provides a principled understanding of how young people perceive and distinguish true and false pieces of information, identifying strengths and weaknesses amidst young subjects and contributing to building tailored digital information literacy strategies for youth.

Nowadays, false and unverified information on social media sway individuals' perceptions during major geo-political events and threaten the quality of the whole digital information ecosystem. Since the Russian invasion of Ukraine, several fact-checking organizations have been actively involved in verifying stories related to the conflict that circulated online. In this paper, we leverage a public repository of fact-checked claims to build a methodological framework for automatically identifying false and unsubstantiated claims spreading on Twitter in February 2022. Our framework consists of two sequential models: First, the claim detection model identifies whether tweets incorporate a (false) claim among those considered in our collection. Then, the claim retrieval model matches the tweets with fact-checked information by ranking verified claims according to their relevance with the input tweet. Both models are based on pre-trained language models and fine-tuned to perform a text classification task and an information retrieval task, respectively. In particular, to validate the effectiveness of our methodology, we consider 83 verified false claims that spread on Twitter during the first week of the invasion, and manually annotate 5,872 tweets according to the claim(s) they report. Our experiments show that our proposed methodology outperforms standard baselines for both claim detection and claim retrieval. Overall, our results highlight how social media providers could effectively leverage semi-automated approaches to identify, track, and eventually moderate false information that spreads on their platforms., Comment: 7 pages, 2 figures, WWW23 Companion Proceedings

The aftermath of the 2020 US Presidential Election witnessed an unprecedented attack on the democratic values of the country through the violent insurrection at Capitol Hill on January 6th, 2021. The attack was fueled by the proliferation of conspiracy theories and misleading claims about the integrity of the election pushed by political elites and fringe communities on social media. In this study, we explore the evolution of fringe content and conspiracy theories on Twitter in the seven months leading up to the Capitol attack. We examine the suspicious coordinated activity carried out by users sharing fringe content, finding evidence of common adversarial manipulation techniques ranging from targeted amplification to manufactured consensus. Further, we map out the temporal evolution of, and the relationship between, fringe and conspiracy theories, which eventually coalesced into the rhetoric of a stolen election, with the hashtag #stopthesteal, alongside QAnon-related narratives. Our findings further highlight how social media platforms offer fertile ground for the widespread proliferation of conspiracies during major societal events, which can potentially lead to offline coordinated actions and organized violence., Comment: 11 pages (including references), 8 figures, 1 table. Accepted at The 18th International AAAI Conference on Web and Social Media

In the current context where online platforms have been effectively weaponized in a variety of geo-political events and social issues, Internet memes make fair content moderation at scale even more difficult. Existing work on meme classification and tracking has focused on black-box methods that do not explicitly consider the semantics of the memes or the context of their creation. In this paper, we pursue a modular and explainable architecture for Internet meme understanding. We design and implement multimodal classification methods that perform example- and prototype-based reasoning over training cases, while leveraging both textual and visual SOTA models to represent the individual cases. We study the relevance of our modular and explainable models in detecting harmful memes on two existing tasks: Hate Speech Detection and Misogyny Classification. We compare the performance between example- and prototype-based methods, and between text, vision, and multimodal models, across different categories of harmfulness (e.g., stereotype and objectification). We devise a user-friendly interface that facilitates the comparative analysis of examples retrieved by all of our models for any given meme, informing the community about the strengths and limitations of these explainable methods.

Online social media represent an oftentimes unique source of information, and having access to reliable and unbiased content is crucial, especially during crises and contentious events. We study the spread of propaganda and misinformation that circulated on Facebook and Twitter during the first few months of the Russia-Ukraine conflict. By leveraging two large datasets of millions of social media posts, we estimate the prevalence of Russian propaganda and low-credibility content on the two platforms, describing temporal patterns and highlighting the disproportionate role played by superspreaders in amplifying unreliable content. We infer the political leaning of Facebook pages and Twitter users sharing propaganda and misinformation, and observe they tend to be more right-leaning than the average. By estimating the amount of content moderated by the two platforms, we show that only about 8-15% of the posts and tweets sharing links to Russian propaganda or untrustworthy sources were removed. Overall, our findings show that Facebook and Twitter are still vulnerable to abuse, especially during crises: we highlight the need to urgently address this issue to preserve the integrity of online conversations., Comment: Accepted at WebSci'2023

The detection of state-sponsored trolls operating in influence campaigns on social media is a critical and unsolved challenge for the research community, which has significant implications beyond the online realm. To address this challenge, we propose a new AI-based solution that identifies troll accounts solely through behavioral cues associated with their sequences of sharing activity, encompassing both their actions and the feedback they receive from others. Our approach does not incorporate any textual content shared and consists of two steps: First, we leverage an LSTM-based classifier to determine whether account sequences belong to a state-sponsored troll or an organic, legitimate user. Second, we employ the classified sequences to calculate a metric named the "Troll Score", quantifying the degree to which an account exhibits troll-like behavior. To assess the effectiveness of our method, we examine its performance in the context of the 2016 Russian interference campaign during the U.S. Presidential election. Our experiments yield compelling results, demonstrating that our approach can identify account sequences with an AUC close to 99% and accurately differentiate between Russian trolls and organic users with an AUC of 91%. Notably, our behavioral-based approach holds a significant advantage in the ever-evolving landscape, where textual and linguistic properties can be easily mimicked by Large Language Models (LLMs): In contrast to existing language-based techniques, it relies on more challenging-to-replicate behavioral cues, ensuring greater resilience in identifying influence campaigns, especially given the potential increase in the usage of LLMs for generating inauthentic content. Finally, we assessed the generalizability of our solution to various entities driving different information operations and found promising results that will guide future research., Comment: 22

Social media provide a fertile ground where conspiracy theories and radical ideas can flourish, reach broad audiences, and sometimes lead to hate or violence beyond the online world itself. QAnon represents a notable example of a political conspiracy that started out on social media but turned mainstream, in part due to public endorsement by influential political figures. Nowadays, QAnon conspiracies often appear in the news, are part of political rhetoric, and are espoused by significant swaths of people in the United States. It is therefore crucial to understand how such a conspiracy took root online, and what led so many social media users to adopt its ideas. In this work, we propose a framework that exploits both social interaction and content signals to uncover evidence of user radicalization or support for QAnon. Leveraging a large dataset of 240M tweets collected in the run-up to the 2020 US Presidential election, we define and validate a multivariate metric of radicalization. We use that to separate users in distinct, naturally-emerging, classes of behaviors associated to radicalization processes, from self-declared QAnon supporters to hyper-active conspiracy promoters. We also analyze the impact of Twitter's moderation policies on the interactions among different classes: we discover aspects of moderation that succeed, yielding a substantial reduction in the endorsement received by hyper-active QAnon accounts. But we also uncover where moderation fails, showing how QAnon content amplifiers are not deterred or affected by Twitter intervention. Our findings refine our understanding of online radicalization processes, reveal effective and ineffective aspects of moderation, and call for the need to further investigate the role social media play in the spread of conspiracies., Comment: 12 pages, 11 figures, 2 tables

Social media moderation policies are often at the center of public debate, and their implementation and enactment are sometimes surrounded by a veil of mystery. Unsurprisingly, due to limited platform transparency and data access, relatively little research has been devoted to characterizing moderation dynamics, especially in the context of controversial events and the platform activity associated with them. Here, we study the dynamics of account creation and suspension on Twitter during two global political events: Russia's invasion of Ukraine and the 2022 French Presidential election. Leveraging a large-scale dataset of 270M tweets shared by 16M users in multiple languages over several months, we identify peaks of suspicious account creation and suspension, and we characterize behaviours that more frequently lead to account suspension. We show how large numbers of accounts get suspended within days from their creation. Suspended accounts tend to mostly interact with legitimate users, as opposed to other suspicious accounts, often making unwarranted and excessive use of reply and mention features, and predominantly sharing spam and harmful content. While we are only able to speculate about the specific causes leading to a given account suspension, our findings shed light on patterns of platform abuse and subsequent moderation during major events., Comment: See published version at EPJ Data Science

Stating your gender pronouns, along with your name, is becoming the new norm of self-introductions at school, at the workplace, and online. The increasing prevalence and awareness of nonconforming gender identities put discussions of developing gender-inclusive language at the forefront. This work presents the first empirical research on gender pronoun usage on large-scale social media. Leveraging a Twitter dataset of over 2 billion tweets collected continuously over two years, we find that the public declaration of gender pronouns is on the rise, with most people declaring as using she series pronouns, followed by he series pronouns, and a smaller but considerable amount of non-binary pronouns. From analyzing Twitter posts and sharing activities, we can discern users who use gender pronouns from those who do not and also distinguish users of various gender identities. We further illustrate the relationship between explicit forms of social network exposure to gender pronouns and their eventual gender pronoun adoption. This work carries crucial implications for gender-identity studies and initiates new research directions in gender-related fairness and inclusion, as well as support against online harassment and discrimination on social media., Comment: 11 pages, 7 figures, 1 table

The importance of emotions in service and in product design is well known. However, it is still not very well understood how users' emotions can be incorporated in a product or service lifecycle. We argue that this gap is due to a lack of a methodological framework for an effective investigation of the emotional response of persons when using products and services. Indeed, the emotional response of users is generally investigated by means of methods (e.g., surveys) that are not effective for this purpose. In our view, Virtual Reality (VR) technologies represent the perfect medium to evoke and recognize users' emotional response, as well as to prototype products and services (and, for the latter, even deliver them). In this paper, we first provide our definition of emotion-driven services, and then we propose a novel methodological framework, referred to as the Virtual-Reality-Based Emotion-Elicitation-and-Recognition loop (VEE-loop), that can be exploited to realize it. Specifically, the VEE-loop consists in a continuous monitoring of users' emotions, which are then provided to service designers as an implicit users' feedback. This information is used to dynamically change the content of the VR environment, until the desired affective state is solicited. Finally, we discuss issues and opportunities of this VEE-loop, and we also present potential applications of the VEE-loop in research and in various application areas.

Social media have been deliberately used for malicious purposes, including political manipulation and disinformation. Most research focuses on high-resource languages. However, malicious actors share content across countries and languages, including low-resource ones. Here, we investigate whether and to what extent malicious actors can be detected in low-resource language settings. We discovered that a high number of accounts posting in Tagalog were suspended as part of Twitter's crackdown on interference operations after the 2016 US Presidential election. By combining text embedding and transfer learning, our framework can detect, with promising accuracy, malicious users posting in Tagalog without any prior knowledge or training on malicious content in that language. We first learn an embedding model for each language, namely a high-resource language (English) and a low-resource one (Tagalog), independently. Then, we learn a mapping between the two latent spaces to transfer the detection model. We demonstrate that the proposed approach significantly outperforms state-of-the-art models, including BERT, and yields marked advantages in settings with very limited training data -- the norm when dealing with detecting malicious activity in online platforms.

Nowadays, social media represent persuasive tools that have been progressively weaponized to affect people's beliefs, spread manipulative narratives, and sow conflicts along divergent factions. Software-controlled accounts (i.e., bots) are one of the main actors associated with manipulation campaigns, especially in the political context. Uncovering the strategies behind bots' activities is of paramount importance to detect and curb such campaigns. In this paper, we present a long term (one year) analysis of bots activity on Twitter in the run-up to the 2018 U.S. Midterm Elections. We identify different classes of accounts based on their nature (bot vs. human) and engagement within the online discussion and we observe that hyperactive bots played a pivotal role in the dissemination of conspiratorial narratives, while dominating the political debate since the year before the election. Our analysis, on the horizon of the upcoming U.S. 2020 Presidential Election, reveals both alarming findings of humans' susceptibility to bots and actionable insights that can contribute to curbing coordinated campaigns.

Since the 2016 US Presidential election, social media abuse has been eliciting massive concern in the academic community and beyond. Preventing and limiting the malicious activity of users, such as trolls and bots, in their manipulation campaigns is of paramount importance for the integrity of democracy, public health, and more. However, the automated detection of troll accounts is an open challenge. In this work, we propose an approach based on Inverse Reinforcement Learning (IRL) to capture troll behavior and identify troll accounts. We employ IRL to infer a set of online incentives that may steer user behavior, which in turn highlights behavioral differences between troll and non-troll accounts, enabling their accurate classification. As a study case, we consider the troll accounts identified by the US Congress during the investigation of Russian meddling in the 2016 US Presidential election. We report promising results: the IRL-based approach is able to accurately detect troll accounts (AUC=89.1%). The differences in the predictive features between the two classes of accounts enables a principled understanding of the distinctive behaviors reflecting the incentives trolls and non-trolls respond to.

The tremendous popularity gained by Online Social Networks (OSNs) raises natural concerns about user privacy in social media platforms. Though users in OSNs can tune their privacy by deliberately deciding what to share, the interaction with other individuals within the social network can expose, and eventually disclose, sensitive information. Among all the sharable personal data, geo-location is particularly interesting. On one hand, users tend to consider their current location as a very sensitive information, avoiding to share it most of the time. On the other hand, service providers are interested to extract and utilize geo-tagged data to offer tailored services. In this work, we consider the problem of inferring the current location of a user utilizing only the available information of other social contacts in the OSN. For this purpose, we employ a graph-based deep learning architecture to learn a model between the users' known and unknown geo-location during a considered period of time. As a study case, we consider Twitter, where the user generated content (i.e., tweet) can embed user's current location. Our experiments validate our approach and further confirm the concern related to data privacy in OSNs. Results show the presence of a critical-mass phenomenon, i.e., if at least 10% of the users provide their tweets with geo-tags, then the privacy of all the remaining users is seriously put at risk. In fact, our approach is able to localize almost 50% of the tweets with an accuracy below 1km relying only on a small percentage of available information.

Recent research brought awareness of the issue of bots on social media and the significant risks of mass manipulation of public opinion in the context of political discussion. In this work, we leverage Twitter to study the discourse during the 2018 US midterm elections and analyze social bot activity and interactions with humans. We collected 2.6 million tweets for 42 days around the election day from nearly 1 million users. We use the collected tweets to answer three research questions: (i) Do social bots lean and behave according to a political ideology? (ii) Can we observe different strategies among liberal and conservative bots? (iii) How effective are bot strategies? We show that social bots can be accurately classified according to their political leaning and behave accordingly. Conservative bots share most of the topics of discussion with their human counterparts, while liberal bots show less overlap and a more inflammatory attitude. We studied bot interactions with humans and observed different strategies. Finally, we measured bots embeddedness in the social network and the effectiveness of their activities. Results show that conservative bots are more deeply embedded in the social network and more effective than liberal bots at exerting influence on humans., Comment: arXiv admin note: text overlap with arXiv:1902.00043

One of the hallmarks of a free and fair society is the ability to conduct a peaceful and seamless transfer of power from one leader to another. Democratically, this is measured in a citizen population's trust in the electoral system of choosing a representative government. In view of the well documented issues of the 2016 US Presidential election, we conducted an in-depth analysis of the 2018 US Midterm elections looking specifically for voter fraud or suppression. The Midterm election occurs in the middle of a 4 year presidential term. For the 2018 midterms, 35 senators and all the 435 seats in the House of Representatives were up for re-election, thus, every congressional district and practically every state had a federal election. In order to collect election related tweets, we analyzed Twitter during the month prior to, and the two weeks following, the November 6, 2018 election day. In a targeted analysis to detect statistical anomalies or election interference, we identified several biases that can lead to wrong conclusions. Specifically, we looked for divergence between actual voting outcomes and instances of the #ivoted hashtag on the election day. This analysis highlighted three states of concern: New York, California, and Texas. We repeated our analysis discarding malicious accounts, such as social bots. Upon further inspection and against a backdrop of collected general election-related tweets, we identified some confounding factors, such as population bias, or bot and political ideology inference, that can lead to false conclusions. We conclude by providing an in-depth discussion of the perils and challenges of using social media data to explore questions about election manipulation.

Understanding the forces governing human behavior and social dynamics is a challenging problem. Individuals' decisions and actions are affected by interlaced factors, such as physical location, homophily, and social ties. In this paper, we propose to examine the role that distinct communities, linked to these factors, play as sources of social influence. The ego network is typically used in the social influence analysis. Our hypothesis is that individuals are embedded in communities not only related to their direct social relationships, but that involve different and complex forces. We analyze physical, homophily, and social communities to evaluate their relation with subjects' behavior. We prove that social influence is correlated with these communities, and each one of them is (differently) significant for individuals. We define community-based features, which reflect the subject involvement in these groups, and we use them with a supervised learning algorithm to predict subject participation in social events. Results indicate that both communities and ego network are relevant sources of social influence, confirming that the ego network alone is not sufficient to explain this phenomenon. Moreover, we classify users according to the degree of social influence they experienced with respect to their groups, recognizing classes of behavioral phenotypes. To our knowledge, this is the first work that proves the existence of phenotypes related to the social influence phenomenon.

Influence propagation in social networks has recently received large interest. In fact, the understanding of how influence propagates among subjects in a social network opens the way to a growing number of applications. Many efforts have been made to quantitatively measure the influence probability between pairs of subjects. Existing approaches have two main drawbacks: (i) they assume that the influence probabilities are independent of each other, and (ii) they do not consider the actions not performed by the subject (but performed by her/his friends) to learn these probabilities. In this paper, we propose to address these limitations by employing a deep learning approach. We introduce a Deep Neural Network (DNN) framework that has the capability for both modeling social influence and for predicting human behavior. To empirically validate the proposed framework, we conduct experiments on a real-life (offline) dataset of an Event-Based Social Network (EBSN). Results indicate that our approach outperforms existing solutions, by efficiently resolving the limitations previously described.