Search

Data augmentation in Natural Language Processing (NLP) faces various challenges that hinder its widespread adoption, unlike its ever-present usage in the field of vision. It is even more the case for the text summarization task where one should focus on both article and summary. In this paper, we review the effect of back translation augmentation, present the diverse beam search decoding strategy, and masking as a method to generate synthetic data for text summarization. The approaches will be evaluated by ROUGE score, novelty, summary length, and GPT-4 to analyze their effectiveness. Our proposed framework presents multiple combinations of back translation and masking for articles, along with diverse augmentation for summaries. Although applicable to networks of any size, we decided to use BART-large, a relatively smaller model, in order to conduct a larger number of experiments. The experiments demonstrated superior performance across all specified metrics when compared to fine-tuning BART-large on the CNN/Dailymail dataset. Specifically, we showed a significant improvement in novelty; 158% and 56% increase rate for bigrams and unigrams, respectively. It could eliminate some copyright concerns around generating content similar to human writing. Additionally, the GPT-4 assessment indicates that models trained using the augmentation technique tend to capture important information more effectively than the baseline model. [ABSTRACT FROM AUTHOR]

Whenever there is too much information out there, it is desirable to summarize. If humans are trying to create the summary, it will take lot of time. Now to make the problem of summarizing information easier and more effortless one can automate the summarization process which can reduce the time taken in creating summary. This is called as automatic summarization. The two ways of summarization are extractive summarization and abstractive summarization. Extractive summarization and its applications have been the subject of extensive research and have received state of art solution. But abstractive summarization still is a progressive field as it is difficult to create abstractive summary as humans do. Also, it is still a question i.e., how to evaluate the quality of a summary? Therefore, this paper is a comprehensive survey on the dataset used with its details and statistics, analysis of various abstractive summarization techniques and important parameters for evaluating the quality of summary. Deep leaning based models have given new direction in this field. The author also focuses on problems and challenges faced in the generation of summary which are opening the future research scope in this domain. [ABSTRACT FROM AUTHOR]

The widespread use of the internet today, along with the rapidly increasing information, has brought along great information pollution. it has become a big problem for internet users to obtain meaningful data from this large and noisy data. Text summarization, which is generally used on texts obtained from digital media, has also been used for summarizing scientific articles in different fields. in this study, a scientific text summary study was carried out to be used on Turkish articles written in the field of informatics. A large Turkish Informatics Literature dataset was created with the articles collected from Dergipark. in addition to the text pre-processing studies available in the literature on this dataset, a new original pre-processing function has been developed by the scientific article format. While summarizing, Deep Belief Networks (DBN), which has an increasing use in the field of natural language processing in the literature, has been used. To measure the performance of the developed system, reference summaries were created with the BERT algorithm, which is a pre-trained natural language processing model. After the scientific articles were summarized with BERT and Deep Belief Networks, the abstracts were compared with BERT Score and BART Score, a specialized comparison metric of the BERT Model. The results showed that the developed Turkish informatics Literature Summarization Method constitutes a summary of a scientific article with 0.78 F-Score and 0.68 BART Score in the BERT Score metric. [ABSTRACT FROM AUTHOR]

Automatic text summarization is the task of creating concise and fluent summaries without human intervention while preserving the meaning of the original text document. To increase the readability of the languages, a summary should be generated. In this paper, a novel Nesterov-accelerated Adaptive Moment Estimation Optimization based on Long Short-Term Memory [NADAM-LSTM] has been proposed to summarize the text. The proposed NADAM-LSTM model involves three stages namely pre-processing, summary generation, and parameter tuning. Initially, the Giga word Corpus dataset is pre-processed using Tokenization, Word Removal, Stemming, Lemmatization, and Normalization for removing irrelevant data. In the summary generation phase, the text is converted to the word-to-vector method. Further, the text is fed to LSTM to summarize the text. The parameter of the LSTM is then tuned using NADAM Optimization. The performance analysis of the proposed NADAM-LSTM is calculated based on parameters like accuracy, specificity, Recall, Precision, and F1 score. The suggested NADAM-LSTM achieves an accuracy range of 99.5%. The result illustrates that the proposed NADAM-LSTM enhances the overall accuracy better than 12%, 2.5%, and 1.5% in BERT, CNN-LSTM, and RNN respectively. [ABSTRACT FROM AUTHOR]

The purpose of government news summarization is to extract the most important information from official government news reports. It is important for readers to be able to understand government news quickly in the age of information overload. Compared with other types of news, government news reports have more detailed content and possess a more normative format, resulting in greater length. To resolve the contradiction between the length of these reports and the trend of fragmented reading, this research proposes an automatic text summarization model based on multiple features. First, the features are extracted using the TF–IDF algorithm and word vector embedding method based on the bidirectional encoder representation from the transformers model. Second, we score the sentences based on the position, keywords, and similarity features. Finally, the top-ranked sentence is selected to form the summarization. To verify the effectiveness and superiority of the proposed method, Edmundson and ROUGE were adopted. First, based on the Edmundson evaluation criteria, the summarization results of various methods were scored. The score differences between ATS summarization based on the proposed method and manual summarization were minimal across consistency, grammaticality, time sequence, conciseness, and readability, with values of 0.14, 0.18, 0.12, 0.10, and 0.16, respectively. This suggests that our summarization exhibits the highest similarity to manual summarization. Second, we evaluated the summarization results using the ROUGE evaluation criteria. The results indicate that the proposed method achieved significantly higher scores compared to other models. Specifically, for character-level ROUGE-1, the P, R, and F scores reached 0.84, 0.93, and 0.88, respectively. At the word-level ROUGE-1, the P, R, and F scores were 0.81, 0.89, and 0.85, respectively, which demonstrates a noticeable improvement over other models. Furthermore, compared to manual methods, the proposed method has advantages in assisting the reader to obtain important information rapidly. [ABSTRACT FROM AUTHOR]

The rapid development of technology has resulted in a surge in the volume of digital data available. This situation creates a problem for users who need assistance in locating specific information within this massive collection of data, resulting in a time-consuming process. Automatic Text Summarization systems have been developed as a more effective solution than traditional summarization techniques to address this problem and improve user access to relevant information. It is well known that researchers in the health sciences find it difficult to keep up with the latest literature due to their busy schedules. This study aims to produce comprehensive abstracts of Turkish-language scientific papers in the field of health sciences. Although abstracts of scientific papers are already available, more thorough summaries are still needed. To the best of our knowledge, no previous attempt has been made to automatically summarize Turkish language health science papers. For this purpose, a dataset of 105 Turkish papers was collected from DergiPark. Term Frequency, Term Frequency-Inverse Document Frequency, Latent Semantic Analysis, TextRank, and Latent Dirichlet Allocation algorithms were chosen as extractive text summarization methods due to their frequent use in this field. The performance of the text summarization models was evaluated using recall, precision, and F-score metrics, and the algorithms gave satisfactory results for Turkish. [ABSTRACT FROM AUTHOR]

Nowadays, individuals can be overwhelmed by a huge number of documents being present in daily life. Capturing the necessary details is often a challenge. Therefore, it is rather important to summarize documents to obtain the main information quickly. There currently exist automatic approaches to this task, but their quality is often not properly assessed. State-of-the-art metrics rely on human-generated summaries as a reference for the evaluation. If no reference is given, the assessment will be challenging. Therefore, in the absence of human-generated reference summaries, we investigated an alternative approach to how machine-generated summaries can be evaluated. For this, we focus on the original text or document to retrieve a metric that allows a direct evaluation of automatically generated summaries. This approach is particularly helpful in cases where it is difficult or costly to find reference summaries. In this paper, we present a novel metric called Summary Score without Reference—SUSWIR—which is based on four factors already known in the text summarization community: Semantic Similarity, Redundancy, Relevance, and Bias Avoidance Analysis, overcoming drawbacks of common metrics. Therefore, we aim to close a gap in the current evaluation environment for machine-generated text summaries. The novel metric is introduced theoretically and tested on five datasets from their respective domains. The conducted experiments yielded noteworthy outcomes, employing the utilization of SUSWIR. [ABSTRACT FROM AUTHOR]

Considering the ever-growing volume of electronic documents made available in our daily lives, the need for an efficient tool to capture their gist increases as well. Automatic text summarization, which is a process of shortening long text and extracting valuable information, has been of great interest for decades. Due to the difficulties of semantic understanding and the requirement of large training data, the development of this research field is still challenging and worth investigating. In this paper, we propose an automated text summarization approach with the adaptation of static and contextual representations based on an extractive approach to address the research gaps. To better obtain the semantic expression of the given text, we explore the combination of static embeddings from GloVe (Global Vectors) and the contextual embeddings from BERT (Bidirectional Encoder Representations from Transformer) and GPT (Generative Pre-trained Transformer) based models. In order to reduce human annotation costs, we employ policy gradient reinforcement learning to perform unsupervised training. We conduct empirical studies on the public dataset, Gigaword. The experimental results show that our approach achieves promising performance and is competitive with various state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

The increasing amount of legal information available online is overwhelming for both citizens and legal professionals, making it difficult and time-consuming to find relevant information and keep up with the latest legal developments. Automatic text summarization techniques can be highly beneficial as they save time, reduce costs, and lessen the cognitive load of legal professionals. However, applying these techniques to legal documents poses several challenges due to the complexity of legal documents and the lack of needed resources, especially in linguistically under-resourced languages, such as the Greek language. In this paper, we address automatic summarization of Greek legal documents. A major challenge in this area is the lack of suitable datasets in the Greek language. In response, we developed a new metadata-rich dataset consisting of selected judgments from the Supreme Civil and Criminal Court of Greece, alongside their reference summaries and category tags, tailored for the purpose of automated legal document summarization. We also adopted several state-of-the-art methods for abstractive and extractive summarization and conducted a comprehensive evaluation of the methods using both human and automatic metrics. Our results: (i) revealed that, while extractive methods exhibit average performance, abstractive methods generate moderately fluent and coherent text, but they tend to receive low scores in relevance and consistency metrics; (ii) indicated the need for metrics that capture better a legal document summary's coherence, relevance, and consistency; (iii) demonstrated that fine-tuning BERT models on a specific upstream task can significantly improve the model's performance. [ABSTRACT FROM AUTHOR]

Due to the obvious significant expansion in the number of online Arabic textual information, Arabic Text Summarization has become a focus of intense research. Manual text summarization necessitates a large investment of time, effort, and money. Hence, Automatic Arabic Text Summarization (AATS) is currently necessary to create accurate and relevant summaries from the huge amount of accessible content. The developed techniques and methodologies for AATS are still in their immaturity because of the intrinsic complexity of the structure and morphology of the Arabic language. AATS methods could be categorized as extractive, abstractive, hybrid extractive to abstractive. The extractive method selects and combines the most important sentences from the input document(s) to produce the summary. While the abstractive method needs deep understanding of the input document(s) for creating the summary with sentences that differ from the original ones. The extractive to abstractive method is a hybrid strategy for creating an informative and cohesive summary using the extractive summary as a first step. This paper provides a detailed explanation of the fundamental issues related to Arabic text summarization. It describes and analyzes the various methods and systems currently in use, traces their history and monitors their performance. The challenges and trends are explored. [ABSTRACT FROM AUTHOR]

The enormous quantity of text makes it challenging for users to obtain the key information and knowledge. Automatic text summarization can alleviate this problem by providing reliable summaries for massive text documents. During the last decade, significant achievements have been made in text summarization. We conduct this survey to explore what research community is focused on, the application scenarios of summarization, the state-of-the-art techniques and methods, and to analyze the challenges and future direction. We summarize that incorporating with natural language processing, previous text summarization research applied knowledge-based methods, graph-based methods, statistical learning methods, and deep learning methods. Applying large language model to text summarization is still in its early stages. By analyzing current research progress, we conclude that understand semantic information and specific domain knowledge is required for text summarization, and the conciseness and readability of the summary should be ensured. The future research opportunity is automatic knowledge summarization, and more research effort is urgently needed to explore. • Significant achievements of automatic text summarization have been made. • We aim to explore the research community of automatic text summarization. • Incorporating with NLP, text summarization applied knowledge-based, graph-based and machine learning methods. • The combination of extractive and abstractive strategies is required, which can ensure the quality of the summary. • The future research opportunity is automatic knowledge summarization. [ABSTRACT FROM AUTHOR]

Automatic Text Summarization (ATS) is an essential field in natural language processing that attempts to condense large text documents so that users can assimilate information quickly. It finds uses in medical document summarization, review generation, and opinion mining. This work investigated an unsupervised extractive summarization approach that combined clustering with topic modeling to reduce topic bias. Latent Dirichlet Allocation was used for topic modeling, while K-Medoids clustering was employed for summary generation. The approach was evaluated on three datasets—Wikihow, CNN/DailyMail, and the DUC2002 Corpus. The Recall Oriented-Understudy for Gisting Evaluation (ROUGE) metrics were used for comparative analysis against recently reported techniques, specifically ROUGE-1 (R-1), ROUGE-2 (R-2), and ROUGE-L (R-L). The suggested framework offered scores of 34.80%, 9.13%, and 32.30% on the Wikihow Dataset, 43.90%, 19.01%, and 41.50% on the CNN/DailyMail Dataset, and 49.35%, 31.53%, and 41.72% on the DUC2002 Corpus (R-1, R-2, R-L respectively). These reported metrics are found to be superior when compared to similar recent works. Further, execution time of the proposed method was also recorded and compared with counterparts, which established its superior speed. Based on these promising outcomes, it was concluded that an unsupervised extractive summarization approach with greater subtopic focus significantly improves over generic topic modeling semantic and deep learning approaches. [ABSTRACT FROM AUTHOR]