Search

Open-domain Question Answering models that directly leverage question-answer (QA) pairs, such as closed-book QA (CBQA) models and QA-pair retrievers, show promise in terms of speed and memory compared with conventional models which retrieve and read from text corpora. QA-pair retrievers also offer interpretable answers, a high degree of control, and are trivial to update at test time with new knowledge. However, these models fall short of the accuracy of retrieve-and-read systems, as substantially less knowledge is covered by the available QA-pairs relative to text corpora like Wikipedia. To facilitate improved QA-pair models, we introduce Probably Asked Questions (PAQ), a very large resource of 65M automatically generated QA-pairs. We introduce a new QA-pair retriever, RePAQ, to complement PAQ. We find that PAQ preempts and caches test questions, enabling RePAQ to match the accuracy of recent retrieve-and-read models, whilst being significantly faster. Using PAQ, we train CBQA models which outperform comparable baselines by 5%, but trail RePAQ by over 15%, indicating the effectiveness of explicit retrieval. RePAQ can be configured for size (under 500MB) or speed (over 1K questions per second) while retaining high accuracy. Lastly, we demonstrate RePAQ’s strength at selective QA, abstaining from answering when it is likely to be incorrect. This enables RePAQ to “back-off” to a more expensive state-of-the-art model, leading to a combined system which is both more accurate and 2x faster than the state-of-the-art model alone.

This paper presents a multi-timescale receding horizon framework for the load forecast of large power customers. The future load pattern of individual users could be very difficult to predict because of its chronological and high volatile properties. Also, the sampling of nonaggregated load data may suffer from severe information missing issues. To address these challenges, we first develop an online singular value thresholding (SVT) algorithm, which utilizes the approximate low-rank property of load data matrices to efficiently recover the missing information. Then, a combinatorial deep learning method is developed, which applies the multi-layer perception (MLP) neural network and the long short-term memory (LSTM) neural network with gated recurrent unit (GRU) to deal with the short-term and ultra-short-term load forecast, respectively. Specifically, an early stopping strategy is designed and implemented to avoid the over-fitting of model training. Moreover, the receding time window is imposed to dynamically update the data recovery and load forecast outcomes, which supports the online computing on a Spark platform. Numerical experiments on real-world load data from North China confirms the effectiveness of the proposed methodology, which can support more complex applications in embedded systems and cyber physical systems.

Failing to consider the strong correlations between weights and topological properties in capacity-weighted networks renders test results on the scale-free property unreliable. According to the preferential attachment mechanism, existing high-degree nodes normally attract new nodes. However, in capacity-weighted networks, the weights of existing edges increase as the network grows. We propose an optimized simplification method and apply it to international trade networks. Our study covers more than 1200 product categories annually from 1995 to 2018. We find that, on average, 38%, 38% and 69% of product networks in export, import and total trade are scale-free. Furthermore, the scale-free characteristics differ depending on the technology. Counter to expectations, the exports of high-technology products are distributed worldwide rather than concentrated in a few developed countries. Our research extends the scale-free exploration of capacity-weighted networks and demonstrates that choosing appropriate filtering methods can clarify the properties of complex networks.

In this paper, we aim to improve abstractive dialogue summarization quality and, at the same time, enable granularity control. Our model has two primary components and stages: 1) a two-stage generation strategy that generates a preliminary summary sketch serving as the basis for the final summary. This summary sketch provides a weakly supervised signal in the form of pseudo-labeled interrogative pronoun categories and key phrases extracted using a constituency parser. 2) A simple strategy to control the granularity of the final summary, in that our model can automatically determine or control the number of generated summary sentences for a given dialogue by predicting and highlighting different text spans from the source text. Our model achieves state-of-the-art performance on the largest dialogue summarization corpus SAMSum, with as high as 50.79 in ROUGE-L score. In addition, we conduct a case study and show competitive human evaluation results and controllability to human-annotated summaries., Comment: ACL-Findings 2021. Code is released at https://github.com/salesforce/ConvSumm

Background The biological network is highly dynamic. Functional relations between genes can be activated or deactivated depending on the biological conditions. On the genome-scale network, subnetworks that gain or lose local expression consistency may shed light on the regulatory mechanisms related to the changing biological conditions, such as disease status or tissue developmental stages. Results In this study, we develop a new method to select genes and modules on the existing biological network, in which local expression consistency changes significantly between clinical conditions. The method is called DNLC: Differential Network Local Consistency. In simulations, our algorithm detected artificially created local consistency changes effectively. We applied the method on two publicly available datasets, and the method detected novel genes and network modules that were biologically plausible. Conclusions The new method is effective in finding modules in which the gene expression consistency change between clinical conditions. It is a useful tool that complements traditional differential expression analyses to make discoveries from gene expression data. The R package is available at https://cran.r-project.org/web/packages/DNLC.

We demonstrate the integration of the Archives Unleashed Toolkit, a scalable platform for exploring web archives, with Google's Tensor-Flow deep learning toolkit to provide scholars with content-based image analysis capabilities. By applying pretrained deep neural networks for object detection, we are able to extract images of common objects from a 4TB web archive of GeoCities, which we then compile into browsable collages. This case study illustrates the types of interesting analyses enabled by combining big data and deep learning capabilities.

A core problem of information retrieval (IR) is relevance matching, which is to rank documents by relevance to a user’s query. On the other hand, many NLP problems, such as question answering and paraphrase identification, can be considered variants of semantic matching, which is to measure the semantic distance between two pieces of short texts. While at a high level both relevance and semantic matching require modeling textual similarity, many existing techniques for one cannot be easily adapted to the other. To bridge this gap, we propose a novel model, HCAN (Hybrid Co-Attention Network), that comprises (1) a hybrid encoder module that includes ConvNet-based and LSTM-based encoders, (2) a relevance matching module that measures soft term matches with importance weighting at multiple granularities, and (3) a semantic matching module with co-attention mechanisms that capture context-aware semantic relatedness. Evaluations on multiple IR and NLP benchmarks demonstrate state-of-the-art effectiveness compared to approaches that do not exploit pretraining on external data. Extensive ablation studies suggest that relevance and semantic matching signals are complementary across many problem settings, regardless of the choice of underlying encoders.

During the past 30 years there has been growing interests in global value chains (GVC) across various disciplines including economics, business & management, economic geography, operational research, computer science, engineering, and so forth. In order to further explore GVC research, this paper employs bibliometric analysis based on co-occurrence network; specifically, it investigates the temporal evolution of disciplines and keywords co-occurrences, as well as the reference co-citation analysis between 1995 and 2014, in order to uncover the evolution of disciplines and research fronts, and identify the intellectual base of global value chains research.

We propose a Multi-task learning approach for Abstractive Text Summarization (MATS), motivated by the fact that humans have no difficulty performing such task because they have the capabilities of multiple domains. Specifically, MATS consists of three components: (i) a text categorization model that learns rich category-specific text representations using a bi-LSTM encoder; (ii) a syntax labeling model that learns to improve the syntax-aware LSTM decoder; and (iii) an abstractive text summarization model that shares its encoder and decoder with the text categorization and the syntax labeling tasks, respectively. In particular, the abstractive text summarization model enjoys significant benefit from the additional text categorization and syntax knowledge. Our experimental results show that MATS outperforms the competitors.1

Spammer detection on social network is a challenging problem. The rigid anti-spam rules have resulted in emergence of "smart" spammers. They resemble legitimate users who are difficult to identify. In this paper, we present a novel spammer classification approach based on Latent Dirichlet Allocation(LDA), a topic model. Our approach extracts both the local and the global information of topic distribution patterns, which capture the essence of spamming. Tested on one benchmark dataset and one self-collected dataset, our proposed method outperforms other state-of-the-art methods in terms of averaged F1-score., NAACL-HLT 2016, Student Research Workshop

Testing-as-a-service (TaaS) is a new model to provide testing capabilities to end users. Users save the cost of complicated maintenance and upgrade effort, and service providers can upgrade their services without impact on the end-users. Due to uneven volumes of concurrent requests, it is important to address the elasticity of TaaS platform in a cloud environment. Scheduling and dispatching algorithms are developed to improve the utilization of computing resources. We develop a prototype of TaaS over cloud, and evaluate the scalability of the platform by increasing the test task load, analyze the distribution of computing time on test task scheduling and test task processing over the cloud, and examine the performance of proposed algorithms by comparing others

This paper describes a concept to support scalable multicast communications for small audio/video conferencing groups on the Internet. The solution presented in this paper is based on extensions of IPv6 and the session description protocol (SDP). A goal of the concept called multicast for small conferences (MSC) is the smooth deployment in the Internet.