Search

In recent years, deep learning has revolutionized natural language processing (NLP) by enabling the development of models that can learn complex representations of language data, leading to significant improvements in performance across a wide range of NLP tasks. Deep learning models for NLP typically use large amounts of data to train deep neural networks, allowing them to learn the patterns and relationships in language data. This is in contrast to traditional NLP approaches, which rely on hand-engineered features and rules to perform NLP tasks. The ability of deep neural networks to learn hierarchical representations of language data, handle variable-length input sequences, and perform well on large datasets makes them well-suited for NLP applications. Driven by the exponential growth of textual data and the increasing demand for condensed, coherent, and informative summaries, text summarization has been a critical research area in the field of NLP. Applying deep learning to text summarization refers to the use of deep neural networks to perform text summarization tasks. In this survey, we begin with a review of fashionable text summarization tasks in recent years, including extractive, abstractive, multi-document, and so on. Next, we discuss most deep learning-based models and their experimental results on these tasks. The paper also covers datasets and data representation for summarization tasks. Finally, we delve into the opportunities and challenges associated with summarization tasks and their corresponding methodologies, aiming to inspire future research efforts to advance the field further. A goal of our survey is to explain how these methods differ in their requirements as understanding them is essential for choosing a technique suited for a specific setting. This survey aims to provide a comprehensive review of existing techniques, evaluation methodologies, and practical applications of automatic text summarization. [ABSTRACT FROM AUTHOR]

In this digital era, there is a tremendous increase in the volume of data, which adds difficulties to the person who utilizes particular applications, such as websites, email, and news. Text summarization targets to reduce the complexity of obtaining statistics from the websites as it compresses the textual document to a short summary without affecting the relevant information. The crucial step in multi-document summarization is obtaining a relationship between the cross-sentence. However, the conventional methods fail to determine the inter-sentence relationship, especially in long documents. This research develops a graph-based neural network to attain an inter-sentence relationship. The significant step in the proposed multi-document text summarization model is forming the weighted graph embedding features. Furthermore, the weighted graph embedding features are utilized to evaluate the relationship between the document's words and sentences. Finally, the bidirectional long short-term memory (BiLSTM) classifier is utilized to summarize the multi-document text summarization. The experimental analysis uses the three standard datasets, the Daily Mail dataset, Document Understanding Conference (DUC) 2002, and Document Understanding Conference (DUC) 2004 dataset. The experimental outcome demonstrates that the proposed weighted graph embedding feature + BiLSTM model exceeds all the conventional methods with Precision, Recall, and F1 score of 0.5352, 0.6296, and 0.5429, respectively. [ABSTRACT FROM AUTHOR]

Patents are recognised as an important source of scientific knowledge. The automatic summarisation process of patents can assist in the organisation, and, consequently, the access to the contents of patent databases. The main contribution of this work is to carry out a study of training approaches of a hybrid summarisation model to create concise, single sentence summaries for patent documents. The experiments were executed using a dataset containing more than 80,000 patents, made available by the United States Patent and Trademark Office. Comparative experiments between the selected model and seven state-of-the-art models in extractive, abstractive and hybrid text summarisation (HTS) were performed. The results obtained showed that the selected approach produces better results than extractive and HTS models, and yields good prospects in extremely concise summaries. It is concluded that the study of different training approaches, coupled with the analysis of the attention words weights in the final results, is an important step in this process, impacting directly the choice of the final summarisation model. Besides this, the results of the experiments suggest that the removal of stop words from the input text did not generate better results, although the attention words extracted with the model without stop words were, in general, better. [ABSTRACT FROM AUTHOR]

Abstract text summarization is a classic sequence-to-sequence natural language generation task. In order to improve the quality of unsupervised abstract text summarization in unsupervised mode, we propose two constraints for training text summarization model, embedding space constraint and information ratio constraint. We construct a generative adversarial network with two discriminators based on these two constraints (TC-SUM-GAN). We use unsupervised and supervised methods to train the model in the experiment. Experimental results show that the ROUGE-1 value of the unsupervised TC-SUM-GAN increases by 1 2. 5 7 points compared with the basic model and at least 1.96 points compared with other comparative models. The ROUGE scores of the supervised TC-SUM-GAN are also improved. TC-SUM-GAN achieves very competitive results for the metrics of ROUGE-1 and ROUGE-2. In addition, the abstracts generated by our model are closer to those generated manually. [ABSTRACT FROM AUTHOR]

Aspect-based summarization differs from generic text-summarization in which the generated summary must be conditioned on a given topic. A fundamental challenge to the aspect-based summarization approach is the lack of labeled data for training models, which limits the usage of supervised methods. One approach to address this issue is to introduce human intervention to generate unique datasets per aspect. However, there is a large number of possible aspects to summarize which makes this option impossible to scale. This limits the use of typical modeling techniques, and requires methods which excel in few-shot, or ideally zero-shot regimes. Hence, in this research, we propose a modular, two-step approach that does not need any aspect-based supervision. This research combines recent advances in zero-shot text classification and generic summarization in a novel way. The backbone of the proposed approach is a transformer network trained for the task of textual entailment, which is used to reduce a document to the set of on topic sentences. In the experiments, our model achieves a new state of the art compared to other unsupervised models on the MA-News dataset (ROUGE-1 35.70 and ROUGE-2 15.52), and even outperforms fine-tuned models without any supervision of its own. [ABSTRACT FROM AUTHOR]

With an increasing number of new summarization systems proposed in recent years, an automatic text evaluation metric that can accurately and reliably rate the performance of summarization systems has been a pressing need. However, current automatic text evaluation metrics can only measure one or certain aspects of the quality between two summary texts and do not agree with human judgments consistently. In this paper, we show that combining multiple well-chosen evaluation metrics and training predictive models using human annotated datasets can lead to more reliable evaluation scores than using any individual automatic metric. Our predictive models trained on a human annotated subset of the CNN/DailyMail corpus demonstrate significant improvements (e.g. approximately 25% along coherence dimension) over selected individual metrics. Furthermore, a concise meta-evaluation on automatic metrics is provided along with an analysis of the performance of 12 predictive models. We also investigate the sensitivity of automatic metrics when mixed together for training these models. We have made the code, the instructions for experiment setup, and the trained models available as a tool for comparing and evaluating text summarization systems. a https://github.com/bzhao2718/ReliableSummEvalReg. [ABSTRACT FROM AUTHOR]

Large volumes of structured and semi-structured data are being generated every day. Processing this large amount of data and extracting important information is a challenging task. The goal of an automatic text summarization is to preserve the key information and the overall meaning of the article to be summarized. In this paper, a graph-based approach is followed to generate an extractive summary, where sentences of the article are considered as vertices, and weighted edges are introduced based on the cosine similarities among the vertices. A possible subset of maximal independent sets of vertices of the graph is identified with the assumption that adjacent vertices provide sentences with similar information. The degree centrality and clustering coefficient of the vertices are used to compute the score of each of the maximal independent sets. The set with the highest score provides the final summary of the article. The proposed method is evaluated using the benchmark BBC News data to demonstrate its effectiveness and is applied to the COVID-19 Twitter data to express its applicability in topic modeling. Both the application and comparative study with other methods illustrate the efficacy of the proposed methodology. [ABSTRACT FROM AUTHOR]