Search

Recent studies have shown that macroscopic patterns of continuity and change over the course of centuries can be detected through the analysis of time series extracted from massive textual corpora. Similar data-driven approaches have already revolutionized the natural sciences and are widely believed to hold similar potential for the humanities and social sciences, driven by the mass-digitization projects that are currently under way, and coupled with the ever-increasing number of documents which are ‘born digital’. As such, new interactive tools are required to discover and extract macroscopic patterns from these vast quantities of textual data. Here we present History Playground, an interactive web-based tool for discovering trends in massive textual corpora. The tool makes use of scalable algorithms to first extract trends from textual corpora, before making them available for real-time search and discovery, presenting users with an interface to explore the data. Included in the tool are algorithms for standardization, regression, change-point detection in the relative frequencies of n-grams, multi-term indices, and comparison of trends across different corpora.

SummaryThe state of an individual's mental health depends on many factors. Determination of the importance of any particular factor within a population needs access to unbiased data. We used publicly available data-sets to investigate, at a population level, how surrogates of mental health covary with light exposure. We found strong seasonal patterns of antidepressant prescriptions, which show stronger correlations with day length than levels of solar energy. Levels of depression in a population can therefore be determined by proxy indicators such as web query logs. Furthermore, these proxies for depression correlate with day length rather than solar energy.Declaration of interestNone.

We have digitised a corpus of Italian newspapers published in 1873–1914 in Gorizia, the county town of an area in the North Adriatic at the crossroad of the Latin, Slavic and Germanic civilizations, then part of the Habsburg Empire and now divided between Italy and Slovenia. This new corpus (of 47,466 pages) is analysed along with a comparable set of local Slovenian newspapers, already digitised by the Slovenian National Library. This large and multilingual effort in digital humanities reveals the statistical traces of events and ideas that shaped a remarkable place and period. The emerging picture is one of rapid cultural, social and technological transformation, and of rising national awareness, combining the larger European pattern with uniquely local aspects.

Many modern Artificial Intelligence (AI) systems make use of data embeddings, particularly in the domain of Natural Language Processing (NLP). These embeddings are learnt from data that has been gathered “from the wild” and have been found to contain unwanted biases. In this paper we make three contributions towards measuring, understanding and removing this problem. We present a rigorous way to measure some of these biases, based on the use of word lists created for social psychology applications; we observe how gender bias in occupations reflects actual gender bias in the same occupations in the real world; and finally we demonstrate how a simple projection can significantly reduce the effects of embedding bias. All this is part of an ongoing effort to understand how trust can be built into AI systems.

When analysing human activities using data mining or machine learning techniques, it can be useful to infer properties such as the gender or age of the people involved. This paper focuses on the sub-problem of gender recognition, which has been studied extensively in the literature, with two main problems remaining unsolved: how to improve the accuracy on real-world face images, and how to generalise the models to perform well on new datasets. We address these problems by collecting five million weakly labelled face images, and performing three different experiments, investigating: The performance difference between convolutional neural networks (CNNs) of differing depths and a support vector machine approach using local binary pattern features on the same training data, the effect of contextual information on classification accuracy, and the ability of convolutional neural networks and large amounts of training data to generalise to cross-database classification. We report record-breaking results on both the Labeled Faces in the Wild (LFW) dataset, achieving an accuracy of 98.90%, and the Images of Groups (GROUPS) dataset, achieving an accuracy of 91.34% for cross-database gender classification.

Previous studies have shown that it is possible to detect macroscopic patterns of cultural change over periods of centuries by analyzing large textual time series, specifically digitized books. This method promises to empower scholars with a quantitative and data-driven tool to study culture and society, but its power has been limited by the use of data from books and simple analytics based essentially on word counts. This study addresses these problems by assembling a vast corpus of regional newspapers from the United Kingdom, incorporating very fine-grained geographical and temporal information that is not available for books. The corpus spans 150 years and is formed by millions of articles, representing 14% of all British regional outlets of the period. Simple content analysis of this corpus allowed us to detect specific events, like wars, epidemics, coronations, or conclaves, with high accuracy, whereas the use of more refined techniques from artificial intelligence enabled us to move beyond counting words by detecting references to named entities. These techniques allowed us to observe both a systematic underrepresentation and a steady increase of women in the news during the 20th century and the change of geographic focus for various concepts. We also estimate the dates when electricity overtook steam and trains overtook horses as a means of transportation, both around the year 1900, along with observing other cultural transitions. We believe that these data-driven approaches can complement the traditional method of close reading in detecting trends of continuity and change in historical corpora.

News content analysis is usually preceded by a labour-intensive coding phase, where experts extract key information from news items. The cost of this phase imposes limitations on the sample sizes that can be processed, and therefore to the kind of questions that can be addressed. In this paper we describe an approach that incorporates text-analysis technologies for the automation of some of these tasks, enabling us to analyse data sets that are many orders of magnitude larger than those normally used. The patterns detected by our method include: (1) similarities in writing style among several outlets, which reflect reader demographics; (2) gender imbalance in media content and its relation with topic; (3) the relationship between topic and popularity of articles.

Understanding changes in the mood and mental health of large populations is a challenge, with the need for large numbers of samples to uncover any regular patterns within the data. The use of data generated by online activities of healthy individuals offers the opportunity to perform such observations on the large scales and for the long periods that are required. Various studies have previously examined circadian fluctuations of mood in this way. In this study, we investigate seasonal fluctuations in mood and mental health by analyzing the access logs of Wikipedia pages and the content of Twitter in the UK over a period of four years. By using standard methods of Natural Language Processing, we extract daily indicators of negative affect, anxiety, anger and sadness from Twitter and compare this with the overall daily traffic to Wikipedia pages about mental health disorders. We show that both negative affect on Twitter and access to mental health pages on Wikipedia follow an annual cycle, both peaking during the winter months. Breaking this down into specific moods and pages, we find that peak access to the Wikipedia page for Seasonal Affective Disorder coincides with the peak period for the sadness indicator in Twitter content, with both most over-expressed in November and December. A period of heightened anger and anxiety on Twitter partly overlaps with increased information seeking about stress, panic and eating disorders on Wikipedia in the late winter and early spring. Finally, we compare Twitter mood indicators with various weather time series, finding that negative affect and anger can be partially explained in terms of the climatic temperature and photoperiod, sadness can be partially explained by the photoperiod and the perceived change in the photoperiod, while anxiety is partially explained by the level of precipitation. Using these multiple sources of data allows us to have access to inexpensive, although indirect, information about collective variations in mood over long periods of time, in turn helping us to begin to separate out the various possible causes of these fluctuations.

Feminist news media researchers have long contended that masculine news values shape journalists' quotidian decisions about what is newsworthy. As a result, it is argued, topics and issues traditionally regarded as primarily of interest and relevance to women are routinely marginalised in the news, while men's views and voices are given privileged space. When women do show up in the news, it is often as "eye candy," thus reinforcing women's value as sources of visual pleasure rather than residing in the content of their views. To date, evidence to support such claims has tended to be based on small-scale, manual analyses of news content. In this article, we report on findings from our large-scale, data-driven study of gender representation in online English language news media. We analysed both words and images so as to give a broader picture of how gender is represented in online news. The corpus of news content examined consists of 2,353,652 articles collected over a period of six months from more than 950 different news outlets. From this initial dataset, we extracted 2,171,239 references to named persons and 1,376,824 images resolving the gender of names and faces using automated computational methods. We found that males were represented more often than females in both images and text, but in proportions that changed across topics, news outlets and mode. Moreover, the proportion of females was consistently higher in images than in text, for virtually all topics and news outlets; women were more likely to be represented visually than they were mentioned as a news actor or source. Our large-scale, data-driven analysis offers important empirical evidence of macroscopic patterns in news content concerning the way men and women are represented.

The discovery of patterns and events in the physical world by analysis of multiple streams of sensor data can provide benefit to society in more than just surveillance applications by focusing on automated means for social scientists, anthropologists and marketing experts to detect macroscopic trends and changes in the general population. This goal complements analogous efforts in documenting trends in the digital world, such as those in social media monitoring. In this paper we show how the contents of a street webcam, processed with state-of-the-art deep networks, can provide information about patterns in clothing and their relation to weather information. In particular, we analyze a large time series of street webcam images, using a deep network trained for garment detection, and demonstrate how the garment distribution over time significantly correlates to weather and temporal patterns. Finally, we additionally provide a new and improved labelled dataset of garments for training and benchmarking purposes, reporting \(58.19\,\%\) overall accuracy on the ACS test set, the best performance yet obtained.

We study the task of learning the preferences of online readers of news, based on their past choices. Previous work has shown that it is possible to model this situation as a competition between articles, where the most appealing articles of the day are those selected by the most users. The appeal of an article can be computed from its textual content, and the evaluation function can be learned from training data. In this paper, we show how this task can benefit from an efficient algorithm, based on hashing representations, which enables it to be deployed on high intensity data streams. We demonstrate the effectiveness of this approach on four real world news streams, compare it with standard approaches, and describe a new online demonstration based on this technology.

Vast data-streams from social networks like Twitter and Facebook contain a people's opinions, fears and dreams. Thomas Lansdall-Welfare, Vasileios Lampos and Nello Cristianini exploit a whole new tool for social scientists.