Search

Background and aims: Using reports of randomised trials of smoking cessation interventions as a test case, this study aimed to develop and evaluate machine learning (ML) algorithms for extracting information from study reports and predicting outcomes as part of the Human Behaviour Change Project. Methods: Researchers manually annotated 70 items of information (‘entities’) in 512 reports of randomised trials of smoking cessation interventions covering intervention content and delivery, population, setting, outcome and study methodology using the Behaviour Change Intervention Ontology. These entities were used to train ML algorithms to extract the information automatically. The information extraction ML algorithm involved a named-entity recognition system using the ‘FLAIR’ framework. The manually annotated intervention, population, setting and study entities were used to develop a deep-learning algorithm using multiple layers of long-short-term-memory (LSTM) components to predict smoking cessation outcomes. Results: The F1 evaluation score, derived from the false positive and false negative rates (range 0-1), for the information extraction algorithm averaged 0.42 across different types of entity (SD=0.22, range 0.05 to 0.88) compared with an average human annotator’s score of 0.75 (SD=0.15, range 0.38 to 1.00). The algorithm for assigning entities to study arms (e.g., intervention or control) was not successful. The ML outcome prediction algorithm did not outperform prediction based just on the mean outcome value or a linear regression model. Conclusions: While some success was achieved in using ML to extract information from reports of randomised trials of smoking cessation interventions, we identified major challenges that could be addressed by greater standardisation in the way that studies are reported. Outcome prediction from smoking cessation studies may benefit from development of novel algorithms, e.g., using ontological information to inform ML.

Artificial intelligence (AI) is defined as the ability of machines to perform tasks that are usually associated with intelligent beings. Argument and debate are fundamental capabilities of human intelligence, essential for a wide range of human activities, and common to all human societies. The development of computational argumentation technologies is therefore an important emerging discipline in AI research1. Here we present Project Debater, an autonomous debating system that can engage in a competitive debate with humans. We provide a complete description of the system’s architecture, a thorough and systematic evaluation of its operation across a wide range of debate topics, and a detailed account of the system’s performance in its public debut against three expert human debaters. We also highlight the fundamental differences between debating with humans as opposed to challenging humans in game competitions, the latter being the focus of classical ‘grand challenges’ pursued by the AI research community over the past few decades. We suggest that such challenges lie in the ‘comfort zone’ of AI, whereas debating with humans lies in a different territory, in which humans still prevail, and for which novel paradigms are required to make substantial progress. An artificial intelligence system that can engage in a competitive debate with humans is presented.

Findings from randomized controlled trials (RCTs) of behaviour change interventions encode much of our knowledge on intervention efficacy under defined conditions. Predicting outcomes of novel interventions in novel conditions can be challenging, as can predicting differences in outcomes between different interventions or different conditions. To predict outcomes from RCTs, we propose a generic framework of combining the information from two sources - i) the instances (comprised of surrounding text and their numeric values) of relevant attributes, namely the intervention, setting and population characteristics of a study, and ii) abstract representation of the categories of these attributes themselves. We demonstrate that this way of encoding both the information about an attribute and its value when used as an embedding layer within a standard deep sequence modeling setup improves the outcome prediction effectiveness.

Social Determinants of Health (SDoH) are an increasingly important part of the broader research and public health efforts in understanding individuals’ physical and mental well-being. Despite this, non-clinical factors affecting health are poorly recorded in electronic health databases and techniques to study how SDoH might relate to population outcomes are lacking. This paper proposes an approach to systematically identify and quantify associations between SDoH and health-related outcomes in a specific cohort of people by (1) leveraging published evidence from literature to build a knowledge graph of health and social factor associations and (2) analysing a large dataset of claims and medical records where those associations may be found. This work demonstrates how the proposed approach could be used to generate hypotheses and inform further research on SDoH in a data-driven manner.

One of the main tasks in argument mining is the retrieval of argumentative content pertaining to a given topic. Most previous work addressed this task by retrieving a relatively small number of relevant documents as the initial source for such content. This line of research yielded moderate success, which is of limited use in a real-world system. Furthermore, for such a system to yield a comprehensive set of relevant arguments, over a wide range of topics, it requires leveraging a large and diverse corpus in an appropriate manner. Here we present a first end-to-end high-precision, corpus-wide argument mining system. This is made possible by combining sentence-level queries over an appropriate indexing of a very large corpus of newspaper articles, with an iterative annotation scheme. This scheme addresses the inherent label bias in the data and pinpoints the regions of the sample space whose manual labeling is required to obtain high-precision among top-ranked candidates.

Cet article se situe dans le contexte de « machine reading » et porte sur la detection automatique d’inferences textuelles posee comme une reformulation d’enonces. De maniere a pouvoir evaluer et etudier les capacites des machines pour reconnaitre automatiquement ces inferences, nous traitons un corpus ecrit de questions-reponses. Nous elaborons d’abord une taxonomie des inferences qui repose sur les procedes linguistiques en jeu. Nous proposons ensuite une methode automatique pour les modeliser.

This paper presents the results of a new approach to discover related health and social factors during the COVID-19 pandemic. The approach leverages a knowledge graph of related concepts mined from a corpus of published evidence (PubMed) prior to the pandemic. Population trends from online searches were used to identify social determinants of health (SDoH) concepts that trended high at the outset of the pandemic from a list of SDoH topics from the World Health Organization (WHO). The trending concepts were then mapped to the knowledge graph and a subsequent analysis of the derived insights, spanning two years, was conducted. This paper suggests an approach to derive new related health and social factors that may have either played a role in, or been affected by, the onset of the global COVID-19 pandemic. In particular, our results show how, from a list of SDoH topics, Food Security, Unemployment trended the highest at the start of the pandemic. Further work is needed to continue to ascertain the validity of the derived relations in a population health context and to improve mining insights from published evidence.

This paper presents the results of a new approach to discover related health and social factors during the COVID-19 pandemic. The approach leverages a knowledge graph of related concepts mined from a corpus of published evidence (PubMed) prior to the pandemic. Population trends from online searches were used to identify social determinants of health (SDoH) concepts that trended high at the outset of the pandemic from a list of SDoH topics from the World Health Organization (WHO). The trending concepts were then mapped to the knowledge graph and a subsequent analysis of the derived insights, spanning two years, was conducted. This paper suggests an approach to derive new related health and social factors that may have either played a role in, or been affected by, the onset of the global COVID-19 pandemic. In particular, our results show how, from a list of SDoH topics, Food Security, Unemployment trended the highest at the start of the pandemic. Further work is needed to continue to ascertain the validity of the derived relations in a population health context and to improve mining insights from published evidence.

Due to the fast pace at which randomized controlled trials are published in the health domain, researchers, consultants and policymakers would benefit from more automatic ways to process them by both extracting relevant information and automating the meta-analysis processes. In this paper, we present a novel methodology based on natural language processing and reasoning models to 1) extract relevant information from RCTs and 2) predict potential outcome values on novel scenarios, given the extracted knowledge, in the domain of behavior change for smoking cessation.

Tasks, Datasets and Evaluation Metrics are important concepts for understanding experimental scientific papers. However, most previous work on information extraction for scientific literature mainly focuses on the abstracts only, and does not treat datasets as a separate type of entity (Zadeh and Schumann, 2016; Luan et al., 2018). In this paper, we present a new corpus that contains domain expert annotations for Task (T), Dataset (D), Metric (M) entities on 2,000 sentences extracted from NLP papers. We report experiment results on TDM extraction using a simple data augmentation strategy and apply our tagger to around 30,000 NLP papers from the ACL Anthology. The corpus is made publicly available to the community for fostering research on scientific publication summarization (Erera et al., 2019) and knowledge discovery., accepted at EACL 2021

Social determinants of health (SDoH) are the factors which lie outside of the traditional health system, such as employment or access to nutritious foods, that influence health outcomes. Some efforts have focused on identifying vulnerable populations during the COVID-19 pandemic, however, both the short- and long-term social impacts of the pandemic on individuals and populations are not well understood. This paper presents a pipeline to discover health outcomes and related social factors based on trending SDoH at population-level using Google Trends. A knowledge graph was built from a corpus of research literature (PubMed) and the social determinants that trended high at the start of the pandemic were examined. This paper reports on related social and health concepts which may be impacted by the COVID-19 outbreak and may be important to monitor as the pandemic evolves. The proposed pipeline should have wider applicability in surfacing related social or clinical characteristics of interest, outbreak surveillance, or to mine relations between social and health concepts that can, in turn, help inform and support citizen-centred services.

Social determinants of health (SDoH) are the factors which lie outside of the traditional health system, such as employment or access to nutritious foods, that influence health outcomes. Some efforts have focused on identifying vulnerable populations during the COVID-19 pandemic, however, both the short- and long-term social impacts of the pandemic on individuals and populations are not well understood. This paper presents a pipeline to discover health outcomes and related social factors based on trending SDoH at population-level using Google Trends. A knowledge graph was built from a corpus of research literature (PubMed) and the social determinants that trended high at the start of the pandemic were examined. This paper reports on related social and health concepts which may be impacted by the COVID-19 outbreak and may be important to monitor as the pandemic evolves. The proposed pipeline should have wider applicability in surfacing related social or clinical characteristics of interest, outbreak surveillance, or to mine relations between social and health concepts that can, in turn, help inform and support citizen-centred services.

With the advancement in argument detection, we suggest to pay more attention to the challenging task of identifying the more convincing arguments. Machines capable of responding and interacting with humans in helpful ways have become ubiquitous. We now expect them to discuss with us the more delicate questions in our world, and they should do so armed with effective arguments. But what makes an argument more persuasive? What will convince you? In this paper, we present a new data set, IBM-EviConv, of pairs of evidence labeled for convincingness, designed to be more challenging than existing alternatives. We also propose a Siamese neural network architecture shown to outperform several baselines on both a prior convincingness data set and our own. Finally, we provide insights into our experimental results and the various kinds of argumentative value our method is capable of detecting., Comment: accepted to ACL 2019 - long paper

The process of obtaining high quality labeled data for natural language understanding tasks is often slow, error-prone, complicated and expensive. With the vast usage of neural networks, this issue becomes more notorious since these networks require a large amount of labeled data to produce satisfactory results. We propose a methodology to blend high quality but scarce strong labeled data with noisy but abundant weak labeled data during the training of neural networks. Experiments in the context of topic-dependent evidence detection with two forms of weak labeled data show the advantages of the blending scheme. In addition, we provide a manually annotated data set for the task of topic-dependent evidence detection. We believe that blending weak and strong labeled data is a general notion that may be applicable to many language understanding tasks, and can especially assist researchers who wish to train a network but have a small amount of high quality labeled data for their task of interest.

While the fast-paced inception of novel tasks and new datasets helps foster active research in a community towards interesting directions, keeping track of the abundance of research activity in different areas on different datasets is likely to become increasingly difficult. The community could greatly benefit from an automatic system able to summarize scientific results, e.g., in the form of a leaderboard. In this paper we build two datasets and develop a framework (TDMS-IE) aimed at automatically extracting task, dataset, metric and score from NLP papers, towards the automatic construction of leaderboards. Experiments show that our model outperforms several baselines by a large margin. Our model is a first step towards automatic leaderboard construction, e.g., in the NLP domain., ACL 2019