Method and dataset entity mining in scientific literature: A CNN + BiLSTM model with self-attention

The traditional literature analysis mainly focuses on the literature metadata such as topics, authors, keywords, references, and rarely pays attention to the main content of papers. However, in many scientific domains such as science, computing, engineering, the methods and datasets involved in the papers published carry important information and are quite useful for domain analysis and recommendation. Method and dataset entities have various forms, which are more difficult to recognize than common entities. In this paper, we propose a novel Method and Dataset Entity Recognition model (MDER), which is able to effectively extract the method and dataset entities from the main textual content of scientific papers. The model is the first to combine rule embedding, a parallel structure of Convolutional Neural Network (CNN) and a two-layer Bi-directional Long Short-Term Memory (BiLSTM) with the self-attention mechanism. We evaluate the proposed model on datasets constructed from the published papers of different research areas in computer science. Our model performs well in multiple areas and features a good capacity for cross-area learning and recognition. Ablation study indicates that building different modules collectively contributes to the good entity recognition performance as a whole. The data augmentation positively contributes to model training, making our model much more robust. We finally apply the proposed model on PAKDD papers published from 2009–2019 to mine insightful results over a long time span. 1