Search

Research on graph-based automatic text summarization for Arabic, the official language of 26 nations with over 200 million speakers, as well as other prevalent languages, has recently increased due to the ability of these approaches to handle linguistic peculiarities such as complex morphological linkages. The present paper proposes a graph-based extractive Arabic text summarization (GEATS) technique that employs word embedding and PageRank algorithms for feature extraction and sentence ordering. The efficiency of the GEATS approach versus the state-of-the-art methods is analyzed based on the quality of the produced summaries over the F-measure values. The findings indicated that it outperformed the nearest alternative by an advantage of over 7.5%. [ABSTRACT FROM AUTHOR]

It is important for abstractive summarization models to understand the important parts of the original document and create a natural summary accordingly. Recently, studies have been conducted to incorporate important parts of the original document during learning and have shown good performance. However, these studies are effective for explicit datasets but not implicit datasets which are relatively more abstract. This study addresses the challenge of summarizing implicit datasets, which have a lower deviation in the significance of important sentences compared to explicit datasets. A multi-task learning approach that reflects information about salient and incidental objects during the learning process was proposed. This was achieved by adding a contrastive objective to the fine-tuning process of the encoder-decoder language model. The salient and incidental parts were selected based on the ROUGE-L F1 score and their relationships were learned through triplet loss. The proposed method was evaluated using five benchmark summarization datasets, including two explicit and three implicit. The experimental results showed a greater improvement in implicit datasets, particularly for the highly abstractive XSum dataset, compared to the vanilla fine-tuning method in both the BART-base and T5-small models. [ABSTRACT FROM AUTHOR]

In this study, we introduce and evaluate a novel extractive text summarization methodology, "SummarEyes," based on the visual interaction of the user with the text, using eye-tracking data, as opposed to the traditional approaches based on analysis of textual content only. We conducted a large-scale user study aiming to collect eye-tracking data while reading the text to be summarized. We utilized various user's implicit attention metrics to generate novel eye-tracking-based text summarization models and compared them both to eye-tracking models typically using only a single feature of the gaze duration and to traditional, as well as state-of-the-art summarization methods, based solely on textual features. The models' quality was evaluated in terms of ROUGE scores using intrinsic evaluation on the datasets we had generated, relating gaze behavior to personalized and DUC gold-standard summaries. The experimental results showed that "SummarEyes" significantly outperformed the other summarizers in predicting both the user's personalized summarization and the generic gold standard summaries. With the increasing availability of eye-tracking technology, this research can lead to a new generation of effective user-centric text summarization tools. [ABSTRACT FROM AUTHOR]

A vast amount of textual information on the internet has amplified the importance of text summarization models. Abstractive summarization generates original words and sentences that may not exist in the source document to be summarized. Such abstractive models may suffer from shortcomings such as linguistic acceptability and hallucinations. Recall-Oriented Understudy for Gisting Evaluation (ROUGE) is a metric commonly used to evaluate abstractive summarization models. However, due to its n-gram-based approach, it ignores several critical linguistic aspects. In this work, we propose Similarity, Entailment, and Acceptability Score (SEAScore), an automatic evaluation metric for evaluating abstractive text summarization models using the power of state-of-the-art pre-trained language models. SEAScore comprises three language models (LMs) that extract meaningful linguistic features from candidate and reference summaries and a weighted sum aggregator that computes an evaluation score. Experimental results show that our LM-based SEAScore metric correlates better with human judgment than standard evaluation metrics such as ROUGE-N and BERTScore. [ABSTRACT FROM AUTHOR]

Identifying and extracting valuable information from textual documents in the form of cohesively and appropriately developed summaries is one of the most challenging tasks in text mining and natural language processing. In this article, we present a sequential Markov model, equipped with Bayesian inference, to estimate the degree of importance of sentences in a document and thereby address the text summarisation problem. The proposed methodology models the extractive sentence summarisation as a Bayesian state estimation problem, where the system state is the importance degree of each sentence in a document. The transition and observation models are derived using a nonlinear dynamical system identification based on a recurrent feedback neural model that predicts the sentence observation using the sentence input data. In the end, the transition and observation probability density functions are modelled using a mixture density network. The performance assessment of the system has been carried out by investigating the optimal feature dimensionality and the impact of the model parameters on the system accuracy, using entropy-based risk and loss-based risk measures. Finally, the superiority of the proposed methodology over the state of the art in extractive summarisation is discussed and verified by reporting the recall, precision and accuracy on the real-world benchmark data sets. [ABSTRACT FROM AUTHOR]

The most recent and precise biological and healthcare knowledge is critical in the current outbreak such as COVID. In today's small world, everyone needs timely and appropriate medical information to prevent contagious diseases. Extraction of important information from medical conversations and dissemination to patients and doctors may benefit in the treatments of doctor tiredness and patient amnesia. Problem: Automatic text summarizing is essential for gaining great knowledge in any topic in an efficient and productive manner. The material included in health records is vital to our understanding of kind illness and its manifestations. Creating a comprehensive and standard kind of content is becoming an unavoidable and crucial problem in the medical process as a result of the massive amounts of fragmented data created in many sectors. Approach: The purpose of this study is to employ NLP‐based deep learning algorithms for text summary that perform well on linguistic text summarization data, and then modify/adapt these for biomedical domain‐specific text summarization. This paper provides an approach developed in‐house for condensing ill‐punctuated or unpunctuated discussion transcripts into more intelligible summaries, which combines topic modelling and phrase selection with punctuation restorations. For autonomous synthesis of medical reports from biomedical transcripts, this research proposes using an end‐to‐end summarization technique, Deep Dense Long Short Term Memory Network (LSTM), followed by Convolutional Neural Network (CNN). Results: Extensive testing, examination, and comparing have demonstrated that this summarizer works well for medical transcript summarization. The suggested approach achieved an average ROUGE score of 93.5% using a single document summary. Furthermore, by comparing new techniques to previous ones, the utility and accuracy of novel strategies would be shown. The results reveal that models trained on ordinary language provide comparable results on a biomedical testing set, with one model outperforming the linguistic test set. [ABSTRACT FROM AUTHOR]

Question‐driven automatic text summarization is a popular technique to produce concise and informative answers to specific questions using a document collection. Both query‐based and question‐driven summarization may not produce reliable summaries nor contain relevant information if they do not take advantage of extractive and abstractive summarization mechanisms to improve performance. In this article, we propose a novel extractive and abstractive hybrid framework designed for question‐driven automatic text summarization. The framework consists of complimentary modules that work together to generate an effective summary: (1) discovering appropriate non‐redundant sentences as plausible answers using an open‐domain multi‐hop question answering system based on a convolutional neural network, multi‐head attention mechanism and reasoning process; and (2) a novel paraphrasing generative adversarial network model based on transformers rewrites the extracted sentences in an abstractive setup. Experiments show this framework results in more reliable abstractive summary than competing methods. We have performed extensive experiments on public datasets, and the results show our model can outperform many question‐driven and query‐based baseline methods (an R1, R2, RL increase of 6%–7% for over the next highest baseline). [ABSTRACT FROM AUTHOR]

The number of scientific publications nowadays is rapidly increasing, causing information overload for researchers and making it hard for scholars to keep up to date with current trends and lines of work. Recent work has tried to address this problem by developing methods for automated summarization in the scholarly domain, but concentrated so far only on monolingual settings, primarily English. In this paper, we consequently explore how state-of-the-art neural abstract summarization models based on a multilingual encoder–decoder architecture can be used to enable cross-lingual extreme summaries of scholarly texts. To this end, we compile a new abstractive cross-lingual summarization dataset for the scholarly domain in four different languages, which enables us to train and evaluate models that process English papers and generate summaries in German, Italian, Chinese and Japanese. We present our new X-SCITLDR dataset for multilingual summarization and thoroughly benchmark different models based on a state-of-the-art multilingual pre-trained model, including a two-stage pipeline approach that independently summarizes and translates, as well as a direct cross-lingual model. We additionally explore the benefits of intermediate-stage training using English monolingual summarization and machine translation as intermediate tasks and analyze performance in zero- and few-shot scenarios. Finally, we investigate how to make our approach more efficient on the basis of knowledge distillation methods, which make it possible to shrink the size of our models, so as to reduce the computational complexity of the summarization inference. [ABSTRACT FROM AUTHOR]

Whenever there is too much information out there, it is desirable to summarize. If humans are trying to create the summary, it will take lot of time. Now to make the problem of summarizing information easier and more effortless one can automate the summarization process which can reduce the time taken in creating summary. This is called as automatic summarization. The two ways of summarization are extractive summarization and abstractive summarization. Extractive summarization and its applications have been the subject of extensive research and have received state of art solution. But abstractive summarization still is a progressive field as it is difficult to create abstractive summary as humans do. Also, it is still a question i.e., how to evaluate the quality of a summary? Therefore, this paper is a comprehensive survey on the dataset used with its details and statistics, analysis of various abstractive summarization techniques and important parameters for evaluating the quality of summary. Deep leaning based models have given new direction in this field. The author also focuses on problems and challenges faced in the generation of summary which are opening the future research scope in this domain. [ABSTRACT FROM AUTHOR]

Document summarization is the task of generating a shorter form of document with important information content. Automatic text summarization has been developed for this process and is still widely used. It is divided into two main parts as extractive summarization and abstractive summarization. In this study, we used sentence ranking methods for extractive summarization for Turkish news text within the scope of the experimental study. We used different summarization rates, 20%, 30%, 40%, 50% and 60%. Summarization results were evaluated with the ROUGE ve BLEU metrics. We proposed new methods based on major vowel harmony and minor vowel harmony features. We obtained high evaluation results in both ROUGE ve BLEU metrics with major vowel harmony and minor vowel harmony features. Additionally, we studied a hybrid model using major vowel harmony and minor vowel harmony rules together. We obtained the best results with major vowel harmony, minor vowel harmony, and hybrid model (major vowel harmony and minor vowel harmony together). We compared the three proposed methods with the BERTurk model prepared for Turkish based on Google BERT. The results obtained gave very close results to this state-of-the-art method and showed that it is worth developing. [ABSTRACT FROM AUTHOR]

Due to the enormous content of text available online through emails, social media, and news articles, it has become complicated to summarize the textual information from multiple documents. Text summarization automatically creates a comprehensive description of the document that retains its informative contents through the keywords, where Multi-Document Summarization (MDS) is a productive tool for data accumulation that creates a concise and informative summary from the documents. In order to extract the relevant information from the documents, Graph neural networks (GNNs) is the neural structure that detains the interrelation of the graph by progressing the messages between the graphical nodes. In the current years, the advanced version of GNNs, such as graph attention network (GAN), graph recurrent network, and graph convolutional network (GCN) provides a remarkable performance in text summarization with the advantage of deep learning techniques. Hence, in this survey, graph approaches for text summarization has been analyzed and discussed, where the recent text summarization model based on Deep learning techniques are highlighted. Further, the article provides the taxonomy to abstract the design pattern of Neural Networks and conducts a comprehensive of the existing text summarization model. Finally, the review article enlists the future direction of the researcher, which would motivate the enthusiastic and novel contributions in text summarizations. [ABSTRACT FROM AUTHOR]

The aim of the work is to prove the possibility of determining the representativeness of a corpus for training a neural model before conducting resource-intensive experiments. In this work, we investigated the dependence of the summarization model on the number of word stems in it. The work was carried out on a synthetic summarization dataset for the Kazakh language. Taking the number of word stems as the representativeness metric, an analysis of the quality of the work of three summarization models was performed depending on the number of word stems in the training dataset. These training datasets differ in the number of rows. To obtain these datasets, we split the training dataset into three parts of different sizes. On the test files, BLEU scores were obtained for each model during the inference process. The highest BLEU scores are obtained for the model trained on the largest amount of data. When the train dataset was reduced by 50 percent, the score decreased from 4 to 25. On the smallest dataset, the score dropped from 25 to 31. The experimental part of the work showed that the model with the largest number of stems shows the highest BLEU score. The scientific contribution of the work is the experimental proof of the representativeness of the training corpus by the number of stems before training the neural model. [ABSTRACT FROM AUTHOR]

In recent years, deep learning has revolutionized natural language processing (NLP) by enabling the development of models that can learn complex representations of language data, leading to significant improvements in performance across a wide range of NLP tasks. Deep learning models for NLP typically use large amounts of data to train deep neural networks, allowing them to learn the patterns and relationships in language data. This is in contrast to traditional NLP approaches, which rely on hand-engineered features and rules to perform NLP tasks. The ability of deep neural networks to learn hierarchical representations of language data, handle variable-length input sequences, and perform well on large datasets makes them well-suited for NLP applications. Driven by the exponential growth of textual data and the increasing demand for condensed, coherent, and informative summaries, text summarization has been a critical research area in the field of NLP. Applying deep learning to text summarization refers to the use of deep neural networks to perform text summarization tasks. In this survey, we begin with a review of fashionable text summarization tasks in recent years, including extractive, abstractive, multi-document, and so on. Next, we discuss most deep learning-based models and their experimental results on these tasks. The paper also covers datasets and data representation for summarization tasks. Finally, we delve into the opportunities and challenges associated with summarization tasks and their corresponding methodologies, aiming to inspire future research efforts to advance the field further. A goal of our survey is to explain how these methods differ in their requirements as understanding them is essential for choosing a technique suited for a specific setting. This survey aims to provide a comprehensive review of existing techniques, evaluation methodologies, and practical applications of automatic text summarization. [ABSTRACT FROM AUTHOR]

Pretrained models in abstractive summarization enable the summarization model to generate coherent summaries flexibly, but it also introduces unfactual hallucinations when generating novel words. To circumvent this drawback, we propose A Guided Tensor Product framework for abstractive Summarization (GTPSum). GTPSum utilizes highlighted sentence guidance signal and role embedding in the form of guided tensor product representations (GTPRs) to enhance the generation capacity for short document text. The guidance signal assists the model in focusing on crucial information, while the role embedding aids GTPSum in capturing the syntactic structure relationships present in the sentences. By employing this approach, the framework can not only focus on the essential information from the source document but also grasp the inherent sentence structure, resulting in generated summaries that closely align with the original content. The experimental results demonstrate the strong performance of our approach on the CNN/Daily Mail (44.28/20.88/41.08 ROUGE-1/-2/-L) and performed well on the other three datasets. Moreover, the comparison between our model generated summaries and those generated by the baseline models through human evaluation shows that our summaries contain more critical information, have better readability, and still maintain succinctness. [ABSTRACT FROM AUTHOR]

Text summarizing (TS) takes key information from a source text and condenses it for the user while retaining the primary material. When it comes to text summaries, the most difficult problem is to provide broad topic coverage and diversity in a single summary. Overall, text summarization addresses the fundamental need to distill large volumes of information into more manageable and digestible forms, making it a crucial technology in the era of information abundance. It benefits individuals, businesses, researchers, and various other stakeholders by enhancing efficiency and comprehension in dealing with textual data. In this paper, proposed a novel Modified Generative adversarial network (MGAN) for summarize the text. The proposed model involves three stages namely pre-processing, Extractive summarization, and summary generation. In the first Phase, the Text similarity dataset is pre-processed using Lowering Casing, Tokenization, Lemmatization, and, Stop Word Removal. In the second Phase, the Extractive summarization is done in three steps Generating similarity metrics, Sentence Ranking, and Sentence Extractive. In the third stage, a generative adversarial network (GAN) employs summary generation to jointly train the discriminative model D and the generative model G. To classify texts and annotate their syntax, Generative Model G employs a convolutional neural network called Bidirectional Gated Recursive Unit (CNN-BiGRU). The performance analysis of the proposed MGAN is calculated based on the parameters like accuracy, specificity, Recall, and Precision metrics. The proposed MGAN achieves an accuracy range of 99%. The result shows that the proposed MGAN improves the overall accuracy better than 9%, 6.5% and 5.4% is DRM, LSTM, and CNN respectively. [ABSTRACT FROM AUTHOR]

In this digital era, there is a tremendous increase in the volume of data, which adds difficulties to the person who utilizes particular applications, such as websites, email, and news. Text summarization targets to reduce the complexity of obtaining statistics from the websites as it compresses the textual document to a short summary without affecting the relevant information. The crucial step in multi-document summarization is obtaining a relationship between the cross-sentence. However, the conventional methods fail to determine the inter-sentence relationship, especially in long documents. This research develops a graph-based neural network to attain an inter-sentence relationship. The significant step in the proposed multi-document text summarization model is forming the weighted graph embedding features. Furthermore, the weighted graph embedding features are utilized to evaluate the relationship between the document's words and sentences. Finally, the bidirectional long short-term memory (BiLSTM) classifier is utilized to summarize the multi-document text summarization. The experimental analysis uses the three standard datasets, the Daily Mail dataset, Document Understanding Conference (DUC) 2002, and Document Understanding Conference (DUC) 2004 dataset. The experimental outcome demonstrates that the proposed weighted graph embedding feature + BiLSTM model exceeds all the conventional methods with Precision, Recall, and F1 score of 0.5352, 0.6296, and 0.5429, respectively. [ABSTRACT FROM AUTHOR]

The biomedical field specialists often face difficulties in summarization of available online literature. The biomedical archives like PubMed contain large set of lengthy documents, which needs to be shortened for further analysis of relevant data. In this work, we have introduced Similarity-specific Teaching and Learning-based optimization (Sim-TLBO) framework, which is combined with controlled stochastic selection of most relevant sentences for effective summarization. This method of utilizing metaheuristic teaching–learning approach along with multiple objective functions analysis provided an agile and robust solution. The output summary from the proposed solution is evaluated and compared based on ROUGE performance and various objective parameters including cohesion, non-redundancy, coverage, and readability. The hybrid text similarity (HTS) based on BERT similarity technique resulted in optimum ROUGE-1 values of 0.82 and 0.79 at 50% and 30% compression rates, respectively. The results from the presented framework when compared to other benchmark and baseline summarization models demonstrated significant performance improvement. [ABSTRACT FROM AUTHOR]

This paper presents a query-based extractive text summarization approach by using sense-oriented semantic relatedness measure. To find the query relevant sentences, we have to find semantic relatedness measure between query and input text sentences. To find the relatedness score, we need to know the exact sense of the words present in query and input text sentences. Word sense disambiguation (WSD) finds the actual meaning of a word according to its context of the sentence. We have proposed a WSD technique to extract query relevant sentences which is used to find a sense-oriented sentence semantic relatedness score between the query and input text sentence. Here, a feature-based method is presented to find semantic relatedness score between query and input text sentence. Finally the proposed query-based text summary method uses relevant and redundancy-free features to form cluster. There is a high probability that same featured cluster may contain redundant sentences. Therefore, a redundancy removal method is proposed to get redundancy-free sentences. In the end, redundancy-free query relevant sentences are obtained with an information rich summary. We have evaluated our proposed WSD technique with other existing methods by using Senseval and SemEval datasets and proposed Sense-Oriented Sentence Semantic Relatedness Score by using Li et al. dataset. We compare our proposed query-based extractive text summarization method with other methods participated in Document Understanding Conference and as well as with current methods. Evaluation and comparison state that the proposed query-based extractive text summarization method outperforms many existing and recent methods. [ABSTRACT FROM AUTHOR]

Fine-tuning a pre-trained sequence-to-sequence-based language model has significantly advanced the field of abstractive summarization. However, the early models of abstractive summarization were limited by the gap between training and inference, and they did not fully utilize the potential of the language model. Recent studies have introduced a two-stage framework that allows the second-stage model to re-rank the candidate summary generated by the first-stage model, to resolve these limitations. In this study, we point out that the supervision method performed in the existing re-ranking model of the two-stage abstractive summarization framework cannot learn detailed and complex information of the data. In addition, we present the problem of positional bias in the existing encoder–decoder-based re-ranking model. To address these two limitations, this study proposes a hierarchical supervision method that jointly performs summary and sentence-level supervision. For sentence-level supervision, we designed two sentence-level loss functions: intra- and inter-intra-sentence ranking losses. Compared to the existing abstractive summarization model, the proposed method exhibited a performance improvement for both the CNN/DM and XSum datasets. The proposed model outperformed the baseline model under a few-shot setting. [ABSTRACT FROM AUTHOR]

The purpose of government news summarization is to extract the most important information from official government news reports. It is important for readers to be able to understand government news quickly in the age of information overload. Compared with other types of news, government news reports have more detailed content and possess a more normative format, resulting in greater length. To resolve the contradiction between the length of these reports and the trend of fragmented reading, this research proposes an automatic text summarization model based on multiple features. First, the features are extracted using the TF–IDF algorithm and word vector embedding method based on the bidirectional encoder representation from the transformers model. Second, we score the sentences based on the position, keywords, and similarity features. Finally, the top-ranked sentence is selected to form the summarization. To verify the effectiveness and superiority of the proposed method, Edmundson and ROUGE were adopted. First, based on the Edmundson evaluation criteria, the summarization results of various methods were scored. The score differences between ATS summarization based on the proposed method and manual summarization were minimal across consistency, grammaticality, time sequence, conciseness, and readability, with values of 0.14, 0.18, 0.12, 0.10, and 0.16, respectively. This suggests that our summarization exhibits the highest similarity to manual summarization. Second, we evaluated the summarization results using the ROUGE evaluation criteria. The results indicate that the proposed method achieved significantly higher scores compared to other models. Specifically, for character-level ROUGE-1, the P, R, and F scores reached 0.84, 0.93, and 0.88, respectively. At the word-level ROUGE-1, the P, R, and F scores were 0.81, 0.89, and 0.85, respectively, which demonstrates a noticeable improvement over other models. Furthermore, compared to manual methods, the proposed method has advantages in assisting the reader to obtain important information rapidly. [ABSTRACT FROM AUTHOR]

Text summarization (TS) plays a crucial role in natural language processing (NLP) by automatically condensing and capturing key information from text documents. Its significance extends to diverse fields, including engineering, healthcare, and others, where it offers substantial time and resource savings. However, manual summarization is a laborious task, prompting the need for automated text summarization systems. In this paper, we propose a novel strategy for extractive summarization that leverages a generative adversarial network (GAN)-based method and Bidirectional Encoder Representations from Transformers (BERT) word embedding. BERT, a transformer-based architecture, processes sentence bidirectionally, considering both preceding and following words. This contextual understanding empowers BERT to generate word representations that carry a deeper meaning and accurately reflect their usage within specific contexts. Our method adopts a generator and discriminator within the GAN framework. The generator assesses the likelihood of each sentence in the summary while the discriminator evaluates the generated summary. To extract meaningful features in parallel, we introduce three dilated convolution layers in the generator and discriminator. Dilated convolution allows for capturing a larger context and incorporating long-range dependencies. By introducing gaps between filter weights, dilated convolution expands the receptive field, enabling the model to consider a broader context of words. To encourage the generator to explore diverse sentence combinations that lead to high-quality summaries, we introduce various noises to each document within our proposed GAN. This approach allows the generator to learn from a range of sentence permutations and select the most suitable ones. We evaluate the performance of our proposed model using the CNN/Daily Mail dataset. The results, measured using the ROUGE metric, demonstrate the superiority of our approach compared to other tested methods. This confirms the effectiveness of our GAN-based strategy, which integrates dilated convolution layers, BERT word embedding, and a generator-discriminator framework in achieving enhanced extractive summarization performance. [ABSTRACT FROM AUTHOR]

Query‐focused multi‐document summarization (Qf‐MDS) is a sub‐task of automatic text summarization that aims to extract a substitute summary from a document cluster of the same topic and based on a user query. Unlike other summarization tasks, Qf‐MDS has specific research challenges including the differences and similarities across related document sets, the high degree of redundancy inherent in the summaries created from multiple related sources, relevance to the given query, topic diversity in the produced summary and the small source‐to‐summary compression ratio. In this work, we propose a semantic diversity feature based query‐focused extractive summarizer (SDbQfSum) built on powerful text semantic representation techniques underpinned with Wikipedia commonsense knowledge in order to address the query‐relevance, centrality, redundancy and diversity challenges. Specifically, a semantically parsed document text is combined with knowledge‐based vectorial representation to extract effective sentence importance and query‐relevance features. The proposed monolingual summarizer is evaluated on a standard English dataset for automatic query‐focused summarization tasks, that is, the DUC2006 dataset. The obtained results show that our summarizer outperforms most state‐of‐the‐art related approaches on one or more ROUGE measures achieving 0.418, 0.092 and 0.152 in ROUGE‐1, ROUGE‐2, and ROUGE‐SU4 respectively. It also attains competitive performance with the slightly outperforming system(s), for example, the difference between our system's result and best system in ROUGE‐1 is just 0.006. We also found through the conducted experiments that our proposed custom cluster merging algorithm significantly reduces information redundancy while maintaining topic diversity across documents. [ABSTRACT FROM AUTHOR]

Aiming at the problems of traditional text abstract extraction algorithms for processing long text of Chinese patents and unsatisfactory results of long abstract generation, the PatBertSum algorithm is proposed, which enables the algorithm to process long (more than 1500 words) patent text with high efficiency and generate high-quality long (more than 200 words) text summaries. The method is based on the improved BertSum algorithm model, using the new CLTPDS patented text dataset, processing long texts by Head-Tail, transforming Chinese input representations, generating sentence vectors using a pre-trained model, and capturing internal text features and text structure features to extract summaries. Experimentally, this paper demonstrates that the method has improved the recall and F-value of ROUGE by more than 8 percentage points compared with existing methods. [ABSTRACT FROM AUTHOR]

The paper describes automatic summarization as one of the topic that helps elearning system to be more adaptable on content generation. This article treat automatic summarization with approaches that provide the ability to summarize texts for different languages. In the case of this article, it is about the English, Romanian and Russian languages. The paper contains both the description of the problem and different approaches already used by other researchers. Next, the data with which the automatic summarization experiments were carried out were described. The metrics with which we can evaluate the quality of the summarization result were presented. Finally, some thoughts were formulated regarding the results obtained in the experiment. [ABSTRACT FROM AUTHOR]

The increasing amount of legal information available online is overwhelming for both citizens and legal professionals, making it difficult and time-consuming to find relevant information and keep up with the latest legal developments. Automatic text summarization techniques can be highly beneficial as they save time, reduce costs, and lessen the cognitive load of legal professionals. However, applying these techniques to legal documents poses several challenges due to the complexity of legal documents and the lack of needed resources, especially in linguistically under-resourced languages, such as the Greek language. In this paper, we address automatic summarization of Greek legal documents. A major challenge in this area is the lack of suitable datasets in the Greek language. In response, we developed a new metadata-rich dataset consisting of selected judgments from the Supreme Civil and Criminal Court of Greece, alongside their reference summaries and category tags, tailored for the purpose of automated legal document summarization. We also adopted several state-of-the-art methods for abstractive and extractive summarization and conducted a comprehensive evaluation of the methods using both human and automatic metrics. Our results: (i) revealed that, while extractive methods exhibit average performance, abstractive methods generate moderately fluent and coherent text, but they tend to receive low scores in relevance and consistency metrics; (ii) indicated the need for metrics that capture better a legal document summary's coherence, relevance, and consistency; (iii) demonstrated that fine-tuning BERT models on a specific upstream task can significantly improve the model's performance. [ABSTRACT FROM AUTHOR]

A novel methodology is introduced for extracting entities from noisy scanned documents by using end-to-end data and reformulating the entity extraction task as a text summarization problem. This approach offers two significant advantages over traditional entity extraction methods while maintaining comparable performance. First, it utilizes preexisting data to construct datasets, thereby eliminating the need for labor-intensive annotation procedures. Second, it employs multitask learning, enabling the training of a model via a single dataset. To evaluate our approach against state-of-the-art methods, we adapted three commonly used datasets, namely, Conference on Natural Language Learning (CoNLL++), few-shot named entity recognition (Few-NERD), and WikiNEuRal domain adaptation (WikiNEuRal + DA), to the format required by our methodology. We subsequently fine-tuned four sequence-to-sequence models: text-to-text transfer transformer (T5), fine-tuned language net T5 (FLAN-T5), bidirectional autoregressive transformer (BART), and pretraining with extracted gap sentences for abstractive summarization sequence-to-sequence models (PEGASUS). The results indicate that, in the absence of optical character recognition (OCR) noise, the BART model performs comparably to state-of-the-art methods. Furthermore, the performance degradation was limited to 3.49–5.23% when 39–62% of the sentences contained OCR noise. This performance is significantly superior to that of previous studies, which reported a 10–20% decrease in the F1 score with texts that had a 20% OCR error rate. Our experimental results demonstrate that a single model trained via our methodology can reliably extract entities from noisy OCRed texts, unlike existing state-of-the-art approaches, which require separate models for correcting OCR errors and extracting entities. [ABSTRACT FROM AUTHOR]

Text summarization research is significant and challenging in the domain of natural language processing. Abstractive text summarization mainly uses the encoder-decoder framework, wherein the encoder component does not have a sufficient semantic comprehension of the input text, and there are exposure biases and semantic inconsistencies between the reference and generated summaries during the training process. We propose an improved encoder-decoder model that incorporates a hierarchical attention mechanism and multiobjective reinforcement learning. The encoder introduces a multihead self-attention mechanism to allow for the acquisition of more comprehensive semantic information from multiple angles and dimensions, while the decoder introduces a pointer-generator network to solve the out-of-vocabulary problem. Multiobjective reinforcement learning methods are constructed throughout the training process to optimize the model in terms of addressing exposure bias, maintaining semantic consistency, and enhancing readability. The results of the comparative experiments demonstrate that the proposed model significantly improved in terms of the ROUGE evaluation metric, and the generated summaries were semantically similar to the reference summaries. [ABSTRACT FROM AUTHOR]

Abstractive Text summarization aims to generate a short summary for a document while preserving salient information. Recently, contrastive learning has been extended from visual representation to summarization tasks. At present, the methods of contrastive learning summarization focus on modeling the global semantics of source documents, targets and candidate summaries to maximize their similarities. However, they ignore the influence of sentence semantics in the document. In this paper, we propose a sentence-level salience contrastive learning method to help the model capture salient information and denoise. The model expresses the sentence salience according to the semantic similarity between the summaries and sentences of the source document, and integrates the similarity distance into the contrastive loss in the form of soft weights. Therefore, our model maximize the similarity between summaries and salient information, while minimizing the similarity between summaries and potential noise. We have verified our method in three widely used datasets, CNN/Daily Mail, XSum and PubMed. The experimental results show that the proposed method can significantly improve the baseline performance and achieve competitive performance in the existing contrastive learning methods. • We propose a contrastive learning method for abstractive text summarization. • We construct contrastive samples in the source sentences. • We incorporate the salient distance-aware weightsinto the contrastive loss. • Experiments show that our model has obviously better performance. [ABSTRACT FROM AUTHOR]

In this study, we conceptualize two approaches, model-based and text-based, grounded on mental models and discourse comprehension theories, to computerized summary analysis. We juxtapose the model-based approach with the text-based approach to explore shared knowledge dimensions and associated measures from both approaches and use them to examine changes in students' summaries over time. We used 108 cases in which we computed model-based and text-based measures for two versions of students' summaries (i.e., initial and final revisions), resulting in a total of 216 observations. We used correlations, Principal Components Analysis (PCA), and Linear Mixed-Effects models. This exploratory investigation suggested a shortlist of text-based measures, and the findings of the PCA demonstrated that both model-based and text-based measures explained the three-dimensional model (i.e., surface, structure, and semantic). Overall, model-based measures were better for tracking changes in the surface dimension, while text-based measures were descriptive of the structure dimension. Both approaches worked well for the semantic dimension. The tested text-based measures can serve as a cross-reference to evaluate students' summaries along with the model-based measures. The current study shows the potential of using multidimensional measures to provide formative feedback on students' knowledge structure and writing styles along the three dimensions. • This study introduces two types of Automated Summary Evaluation (ASE) approaches. • We explored shareable knowledge dimensions and their measures in both ASE approaches. • We employed PCA and Mixed-Effects models to assess 216 observations across 108 cases. • PCA showed measures from both approaches explained the three-dimensional model. • Both approaches' measures could enable multidimensional assessment and feedback. [ABSTRACT FROM AUTHOR]

The original article can be found online at https://doi.org/10.1007/s00521-023-08680-0.Correction to: Neural Computing and Applications (2023) 35:18553–18570https://doi.org/10.1007/s00521-023-08680-0In this article the author's name Shuaimin Li was incorrectly written as Shuimin Li.The original article has been corrected.Publisher's NoteSpringer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.By Shuaimin Li and Jungang XuReported by Author; Author [Extracted from the article]

Text summarization is a widely-researched problem among scholars in the field of natural language processing. Multiple techniques have been proposed to help tackle this problem, yet some of these methodologies may still exhibit limitations such as the requirements for large training datasets, which might not always be possible, but more importantly, the lack of interpretability or transparency of the model. In this paper, we propose using meta-learning algorithm to train a deep learning model for extractive text summarization and then using various explanatory techniques such as SHAP (Shapley, 1953), linear regression (Lederer, 2022), decision trees (Fürnkranz, 2010), and input modification to gain insights into the model's decision making process. To evaluate the effectiveness of our approach, we will compare it to other popular natural language processing models like BERT (Miller, 2019) or XLNET (Yang et al., 2020) using the ROUGE metrics (Lin, 2004). Overall, our proposed approach provides a promising solution to the limitations of existing methods and a framework for improving the explainability of deep learning models in other natural language processing tasks. • Bi-LSTM and Meta-learning produce better results compared to other popular models. • Meta-learning can be utilized to improve performance in data-scarce scenarios. • Use SHAP, regression, decision tree, and text mutation to enhance interpretability. [ABSTRACT FROM AUTHOR]

Summarizing trending topics in large collections of Facebook posts is particularly relevant to profile social user activities and interests. However, automatically generating these summaries poses significant challenges due to the high heterogeneity of the input data, the limited fluency of extractive summaries, and the absence of abstractive summarization methods capable of handling multiple posts simultaneously. Existing abstractive models are either not suited to handle large post collections or disregard topic-level text relations. In this work, we present TASP , a novel tool for trending topic detection and summarization from English-written Facebook posts. It trains abstractive summarization models on multi-post collections by leveraging a shortlist of authoritative posts published by renowned newspapers. At inference time, TASP first creates clusters of semantically similar social posts, each one representing a distinct topic, using pre-trained transformer-based language models. Then, it generates abstractive summaries of the clusters for which authoritative information is missing. To the best of our knowledge, TASP is the first available tool suited to abstractive multi-post summarization. We test our approach on a large-scale dataset of real Facebook posts. The results show (1) The higher effectiveness of transformer-based approaches in generating topic-specific post clusters compared to traditional methods. (2) The importance of attending long pieces of text in multi-post abstractive summary generation. • A tool for abstractive summarization of multi-post collections is proposed. • It exploits topic modeling techniques to handle post heterogeneity. • It overcomes the limitations of existing approaches summarizing one post at a time. • It leverages transformers for both topic modeling and summarization. • It reports a showcase of a real large-scale data collection of UK Facebook posts. [ABSTRACT FROM AUTHOR]

Opinion mining aims to detect and extract relevant information from large quantity of customer reviews. Automatic opinion summarization then seeks to create a consensual point of view often oriented toward the main sentiment of clients to render their experience. Although factual information is valuable for companies to understand what works or not in their products, summarization approaches that convey objectivity, and constructive feedback from customer reviews have yet to be explored. We propose an adversarial multi-task learning model for document summarization to address this new issue. Our algorithm combines an autoencoder for document summarization with a gradient reversal layer to learn independent representations of subjective and sentiment-based material. We assess and compare our method on the Amazon product review dataset where we introduce an original evaluation dataset for objective summarization. We further completed the analysis with neutrality and objectivity metrics. This study demonstrates that the generated summaries carry out relevant and objective content but also emphasize the importance of various processes and layers in multi-task learners to control the information effectively. • Objective opinion summarization of customer reviews. • Unsupervised autoencoder for summarization. • Sentiment disentanglement with gradient reversal layer. • Dataset and metrics to assess neutrality and objectivity of summaries. [ABSTRACT FROM AUTHOR]

The enormous quantity of text makes it challenging for users to obtain the key information and knowledge. Automatic text summarization can alleviate this problem by providing reliable summaries for massive text documents. During the last decade, significant achievements have been made in text summarization. We conduct this survey to explore what research community is focused on, the application scenarios of summarization, the state-of-the-art techniques and methods, and to analyze the challenges and future direction. We summarize that incorporating with natural language processing, previous text summarization research applied knowledge-based methods, graph-based methods, statistical learning methods, and deep learning methods. Applying large language model to text summarization is still in its early stages. By analyzing current research progress, we conclude that understand semantic information and specific domain knowledge is required for text summarization, and the conciseness and readability of the summary should be ensured. The future research opportunity is automatic knowledge summarization, and more research effort is urgently needed to explore. • Significant achievements of automatic text summarization have been made. • We aim to explore the research community of automatic text summarization. • Incorporating with NLP, text summarization applied knowledge-based, graph-based and machine learning methods. • The combination of extractive and abstractive strategies is required, which can ensure the quality of the summary. • The future research opportunity is automatic knowledge summarization. [ABSTRACT FROM AUTHOR]

Specifically focusing on the landscape of abstractive text summarization, as opposed to extractive techniques, this survey presents a comprehensive overview, delving into state-of-the-art techniques, prevailing challenges, and prospective research directions. We categorize the techniques into traditional sequence-to-sequence models, pre-trained large language models, reinforcement learning, hierarchical methods, and multi-modal summarization. Unlike prior works that did not examine complexities, scalability and comparisons of techniques in detail, this review takes a comprehensive approach encompassing state-of-the-art methods, challenges, solutions, comparisons, limitations and charts out future improvements — providing researchers an extensive overview to advance abstractive summarization research. We provide vital comparison tables across techniques categorized — offering insights into model complexity, scalability and appropriate applications. The paper highlights challenges such as inadequate meaning representation, factual consistency, controllable text summarization, cross-lingual summarization, and evaluation metrics, among others. Solutions leveraging knowledge incorporation and other innovative strategies are proposed to address these challenges. The paper concludes by highlighting emerging research areas like factual inconsistency, domain-specific, cross-lingual, multilingual, and long-document summarization, as well as handling noisy data. Our objective is to provide researchers and practitioners with a structured overview of the domain, enabling them to better understand the current landscape and identify potential areas for further research and improvement. [Display omitted] • Overview of state-of-the-art techniques in abstractive text summarization. • Comparative analysis of models in abstractive summarization. • Identification of challenges and potential improvements in the field. • Exploration of future research directions and emerging frontiers. • Holistic survey of abstractive text summarization. [ABSTRACT FROM AUTHOR]

Automatic Text Summarization (ATS) involves estimating the salience of information and creating coherent summaries that include all relevant and important information from the original text. Extensive research has been carried out on ATS since 1958, gradually evolving from simple to advanced techniques, including machine learning-based, neural network-based, and deep learning-based techniques. Progress has been made in both extractive and abstractive methods throughout this development. Despite numerous surveys on ATS, there remains a notable absence of a comprehensive literature review encompassing the latest advancements in deep learning techniques for text summarization. Therefore, this paper provides a Systematic Literature Review (SLR) of deep learning-based text summarization in both types (extractive and abstractive) between 2014 and 2023. To the best of our knowledge, this is the first SLR that offers a comprehensive overview of extractive and abstractive text summarization techniques based on Deep Learning models. According to the defined inclusion and exclusion criteria, 73 deep learning-based text summarization studies are selected for further investigation. The structure of the review is organized as follows. Firstly, it identifies and examines the deep learning models employed in both extractive and abstractive text summarization. Then, the input text's representation methods are identified and discussed clearly. Next, training strategies used in supervised extractive summarization techniques are identified. Furthermore, mechanisms that improve the abstractive summarization process are identified. Additionally, the most commonly used datasets and their advantages and disadvantages are discussed. The most commonly used evaluation metrics are also identified. Finally, the challenges and possible solutions to guide future research in the field are discussed. • Analyze the latest deep learning-based text summarization techniques. • Introduce a new taxonomy of deep learning models for text summarization. • Identify methods for representing input text clearly. • Identify training strategies for supervised extractive summarization techniques. • Identify the mechanisms used in abstractive summarization techniques. [ABSTRACT FROM AUTHOR]

Abstractive dialogue summarization requires distilling and rephrasing key information from noisy multi-speaker documents. Combining pre-trained language models with input augmentation techniques has recently led to significant research progress. However, existing solutions still struggle to select relevant chat segments, primarily relying on open-domain and unsupervised annotators not tailored to the actual needs of the summarization task. In this paper, we propose DearWatson , a task-aware utterance-level annotation framework for improving the effectiveness and interpretability of pre-trained dialogue summarization models. Precisely, we learn relevant utterances in the source document and mark them with special tags, that then act as supporting evidence for the generated summary. Quantitative experiments are conducted on two datasets made up of real-life messenger conversations. The results show that DearWatson allows model attention to focus on salient tokens, achieving new state-of-the-art results in three evaluation metrics, including semantic and factuality measures. Human evaluation proves the superiority of our solution in semantic consistency and recall. Finally, extensive ablation studies confirm each module's importance, also exploring different annotation strategies and parameter-efficient fine-tuning of large generative language models. [ABSTRACT FROM AUTHOR]

Development of effective automatic summarization approaches for legal documents suffer from several challenges like extremely long document-summary pairs, lack of large scale training datasets with tractable document-summary token lengths, etc. In this work, we deal with the problem of legal document summarization by building a modified abstractive dataset from the original dataset. This ensures that the length of each document-summary pair is manageable and can be processed by the state of the art summarization approaches (such as BART). Secondly, we deal with the data scarcity problem by creating more number of training samples, from each of the original document-summary pair. This is done by creating multiple extractive summaries from each sample in the original dataset, following which ground-truth summary sentences are assigned to each of the extractive summary to generate new training samples. This results in a larger training dataset that can be utilized for fine-tuning summarization models. Our proposed approach has been evaluated on two different legal datasets- BillSum and Forum of Information Retrieval Evaluation (FIRE). With respect to the ROUGE metrics, the proposed approach is able to outperform pre-trained BART model fine-tuned on original dataset by (3 − 8) % for FIRE test sets, and by (1 − 3) % for the BillSum test sets. Considering the BERTScore metrics, the proposed approach obtains (1 − 2) % improvements on the FIRE test sets, while for the BillSum test sets (3 − 8) % improvements are observed. Such improvements suggest that the proposed dataset building approach can help achieve improved abstractive summarization of lengthy legal documents. • Lengthy nature and data scarcity are the two main challenges with legal documents. • Lengthy legal document summarization is handled by the proposed approach. • The data scarcity problem is handled by creating a feasible abstractive dataset. • A novel Extract-Then-Assign (ETA) approach is proposed. • ETA approach can greatly improve abstractive summarization of legal documents. [ABSTRACT FROM AUTHOR]

With the development of pre-trained language models and large-scale datasets, automatic text summarization has attracted much attention from the community of natural language processing, but the progress of automatic summarization evaluation has stagnated. Although there have been efforts to improve automatic summarization evaluation, ROUGE has remained one of the most popular metrics for nearly 20 years due to its competitive evaluation performance. However, ROUGE is not perfect, there are studies have shown that it is suffering from inaccurate evaluation of abstractive summarization and limited diversity of generated summaries, both caused by lexical bias. To avoid the bias of lexical similarity, more and more meaningful embedding-based metrics have been proposed to evaluate summaries by measuring semantic similarity. Due to the challenge of accurately measuring semantic similarity, none of them can fully replace ROUGE as the default automatic evaluation toolkit for text summarization. To address the aforementioned problems, we propose a compromise evaluation framework (ROUGE-SEM) for improving ROUGE with semantic information, which compensates for the lack of semantic awareness through a semantic similarity module. According to the differences in semantic similarity and lexical similarity, summaries are classified into four categories for the first time, including good-summary, pearl-summary, glass-summary, and bad-summary. In particular, the back-translation technique is adopted to rewrite pearl-summary and glass-summary that are inaccurately evaluated by ROUGE to alleviate lexical bias. Through this pipeline framework, summaries are first classified by candidate summary classifier, then rewritten by categorized summary rewriter, and finally scored by rewritten summary scorer, which are efficiently evaluated in a manner consistent with human behavior. When measured using Pearson, Spearman, and Kendall rank coefficients, our proposal achieves comparable or higher correlations with human judgments than several state-of-the-art automatic summarization evaluation metrics in dimensions of coherence, consistency, fluency, and relevance. This also suggests that improving ROUGE with semantics is a promising direction for automatic summarization evaluation. [ABSTRACT FROM AUTHOR]

A long-running event represents a continuous stream of information on a given topic, such as natural disasters, stock market updates, or even ongoing customer relationship. These news stories include hundreds of individual, time-dependent texts. Simultaneously, new technologies have profoundly transformed the way we consume information. The need to obtain quick, relevant, and digest updates continuously has become a crucial issue and creates new challenges for the task of automatic document summarization. To that end, we introduce an innovative unsupervised method based on two competing sequence-to-sequence models to produce short updated summaries. The proposed architecture relies on several parameters to balance the outputs from the two autoencoders. This relation enables the overall model to correlate generated summaries with relevant information coming from both current and previous news iterations. Depending on the model configuration, we are then able to control the novelty or the consistency of terms included in generated summaries. We evaluate our method on a modified version of the TREC 2013, 2014, and 2015 datasets to track continuous events from a single source. We not only achieve state-of-the-art performance similar to other more complex unsupervised sentence compression approaches, but also influence the information included in the model in the summaries. • Incremental summarization of real-time news events. • Unsupervised autoencoder for summarization of short texts related to the same event. • Semi-extractive method of accounting for information updates in the sentence compression task. • Promoting novelty or cohesion of information in incremental and update summarization processes. [ABSTRACT FROM AUTHOR]

Feature Maximization is a feature selection method that deals efficiently with textual data: to design systems that are altogether language-agnostic, parameter-free and do not require additional corpora to function. We propose to evaluate its use in text summarization, in particular in cases where documents are structured. We first experiment this approach in a single-document summarization context. We evaluate it on the DUC AQUAINT corpus and show that despite the unstructured nature of the corpus, our system is above the baseline and produces encouraging results. We also observe that the produced summaries seem robust to redundancy. Next, we evaluate our method in the more appropriate context of SciSumm challenge, which is dedicated to research publications summarization. These publications are structured in sections and our class-based approach is thus relevant. We more specifically focus on the task that aims to summarize papers using those that refer to them. We consider and evaluate several systems using our approach dealing with specific bag of words. Furthermore, in these systems, we also evaluate cosine and graph-based distance for sentence weighting and comparison. We show that our Feature Maximization based approach performs very well in the SciSumm 2016 context for the considered task, providing better results than the known results so far, and obtaining high recall. We thus demonstrate the flexibility and the relevance of Feature Maximization in this context. [ABSTRACT FROM AUTHOR]

Abstractive summarization is flexible and allows the model to generate new words and phrases. However, the familiar words are more likely to be selected as abstract candidate words in the process of abstractive summarization, causing the generated abstract to diverge from the reference. In our consideration, this is caused by representation degeneration of the pre-trained word embedding. Therefore, this paper proposes a general abstractive summarization framework with dynamic word embedding representation correction (RepSum). The representation correction algorithm identifies the dimension most relevant to word frequency and eliminates the word frequency features. Then the distribution of word embeddings will be more even. As a result, the words will be selected as candidate words without frequency bias to improve the quality of the abstract. The experimental results illustrate that RepSum performs better than the benchmark model in summary quality, demonstrating our method's effectiveness. [Display omitted] • An embedding representation correction algorithm based on word frequency feature elimination is proposed. • The algorithm identifies the dimension most relevant to word frequency and eliminates the word frequency features. • Then the distribution of word embeddings will be more even and the words will be selected as candidate words without frequency bias. [ABSTRACT FROM AUTHOR]