Search

In this paper, three methods of extracting single document summary by combining supervised learning with unsupervised learning are proposed. The purpose of these three methods is to measure the importance of sentences by combining the statistical features of sentences and the relationship between sentences at the same time. The first method uses supervised model and graph model to score sentences separately, and then linear combination of scores is used as the final score of sentences. In the second method, the graph model is used as an independent feature of the supervised model to evaluate the importance of sentences. The third method is to score the importance of sentences by supervised model, then as a priori value of nodes in the graph model, and finally use biased graph model to score sentences. On the data sets of DUC2001 and DUC2002, the ROUGE method is used as the evaluation criterion, which shows that the three methods have achieved good results, and are superior to the methods of extracting summary only using supervised learning or unsupervised learning. We also validate that priori knowledge can improve the accuracy of key sentence selection in graph model.

When reading an article, especially a professional article, we often encounter words or phrases that we don't recognize. They may be specific domain terms or emerging entities. When we can't guess their meaning from the article, we generally refer to the terminology dictionary or search for related content on the Internet to understand them. Some researchers have used natural language generation (NLG) models to explain these unknown phrases in recent years automatically. Still, current NLG models have difficulties generating long sentences with good coherence, and they are difficult to generate multiple sentences that describe unknown phrases from different angles. Therefore, this paper proposes a model that can judge whether an existing sentence can explain a certain phrase, called TransExplain. TransExplain can use LSTM, convolutional neural network, and attention mechanism to extract multiple sentence features and map them to a fixed-dimensional semantic feature vector. By calculating the cosine similarity between the semantic feature vector and the unknown phrase vector, it is judged whether the sentence can explain the semantics of the unknown phrase. And a loss function called positive and negative means square error is introduced to improve the model's ability that distinguishes negative examples. For this task, we provided a Chinese dataset containing phrases and explanation pairs in 7 important domains. On this dataset, TransExplain can achieve better results than previous similar tasks.

The essence of extractive summarization is to measure the importance of sentences in the document. When extracting summary from a single document, it is difficult to comprehensively and effectively evaluate the importance of sentences due to the lack of information. In this paper, we propose a kind of single document summarization method using information from documents under the same topic. This method integrates the topic information from neighborhood documents and statistical information from the target document to calculate the score of sentences. Then the scoring results are used as a prior scores for each sentence in the target document. After the target document is represented by the sentence graph, the final score of the sentences are obtained by the biased random walk algorithm. Finally, the Maximal Marginal Relevance (MMR) algorithm is used to select the sentences to form summary. The experimental results on the DUC2001 and DUC2002 datasets show that the effect of extracting summary is improved by incorporating information from the documents under the same topic.

Currently, there are two main methods of learning phrase embedding: the distribution method and the composition method. The distribution method treats a phrase as an entirety and learns phrase embedding based on the context of the phrase. Its disadvantage is that it completely ignores the semantics of the component words of the phrase and the data sparseness problem. The composition method calculates the phrase embedding from the embedding of component words. The existing composition methods fail to represent the semantics of phrases well. Because of the above problems, we take Chinese, for example, and propose a new composition method to generate phrase embedding from the component word embedding, named TransPhrase. It is a neural network that can use LSTM to learn the order information of component words, use the attention mechanism to learn the important information of component words, and use a fully connected network to learn the semantic information of component words, and finally predict phrase embedding. It can solve the data sparseness problem and properly and fully represent the semantics of phrases. Our evaluation of three Chinese phrase-level semantic tasks shows that the comprehensive performance of TransPhrase's phrase representation is better than the composition method, the distribution method, and the pre-trained language model.

Phrase embedding can improve the performance of multiple NLP tasks. Most of the previous phrase-embedding methods that only use the external or internal semantic information of phrases to learn phrase embedding are challenging to solve the problem of data sparseness and have poor semantic presentation ability. To solve the above issues, in this paper, we propose an autoencoder-based method to combine pre-trained phrase embeddings and phrase component word embeddings into new phrase embeddings through complex non-linear transformations. This method uses both internal and external semantic information of phrases to generate new phrases with better semantic expression capabilities. This method can also generate well-represented phrase embeddings when only pre-trained component word embeddings are used as input to solve the problem of data sparseness effectively. We have designed two models for this method. The first one uses an FCNN(Fully Connected Neural Network) as the encoder and decoder, which we call AE-F. The second one uses the attention mechanism shared by the parameters of encoder and decoder to proportionally allocate the outputs of an LSTM and an FCNN, which we call it AE-ALF. We evaluated them in terms of phrase similarity and phrase classification and used two English datasets and two Chinese datasets. Experimental results show that AE-F and AE-ALF methods using pre-trained phrase embeddings and component word embeddings exceed 17 baseline methods, and AE-F and AE-ALF perform similarly. With only pre-trained component word embeddings, AE-F and AE-ALF also exceed most baseline methods, and AE-ALF performs better than AE-F.