1. Keywords extraction with deep neural network model
- Author
-
Qingyu Yin, Mingxiang Tuo, Le Qi, Ting Liu, Yu Zhang, and Xuxiang Wang
- Subjects
0209 industrial biotechnology ,Focus (computing) ,Training set ,Artificial neural network ,business.industry ,Computer science ,Cognitive Neuroscience ,Deep learning ,Feature selection ,02 engineering and technology ,Machine learning ,computer.software_genre ,Computer Science Applications ,Task (project management) ,020901 industrial engineering & automation ,Artificial Intelligence ,0202 electrical engineering, electronic engineering, information engineering ,020201 artificial intelligence & image processing ,Artificial intelligence ,business ,computer ,Word (computer architecture) ,Sentence - Abstract
Keywords can express the main content of an article or a sentence. Keywords extraction is a critical issue in many Natural Language Processing (NLP) applications and can improve the performance of many NLP systems. The traditional methods of keywords extraction are based on machine learning or graph model. The performance of these methods is influenced by the feature selection and the manually defined rules. In recent years, with the emergence of deep learning technology, learning features automatically with the deep learning algorithm can improve the performance of many tasks. In this paper, we propose a deep neural network model for the task of keywords extraction. We make two extensions on the basis of traditional LSTM model. First, to better utilize both the historic and following contextual information of the given target word, we propose a target center-based LSTM model (TC-LSTM), which learns to encode the target word by considering its contextual information. Second, on the basis of TC-LSTM model, we apply the self-attention mechanism, which enables our model has an ability to focus on informative parts of the associated text. In addition, we also introduce a two-stage training method, which takes advantage of large-scale pseudo training data. Experimental results show the advantage of our method, our model beats all the baseline systems all across the board. And also, the two-stage training method is of great significance for improving the effectiveness of the model.
- Published
- 2020
- Full Text
- View/download PDF