Search

Fine-tuning a pre-trained sequence-to-sequence-based language model has significantly advanced the field of abstractive summarization. However, the early models of abstractive summarization were limited by the gap between training and inference, and they did not fully utilize the potential of the language model. Recent studies have introduced a two-stage framework that allows the second-stage model to re-rank the candidate summary generated by the first-stage model, to resolve these limitations. In this study, we point out that the supervision method performed in the existing re-ranking model of the two-stage abstractive summarization framework cannot learn detailed and complex information of the data. In addition, we present the problem of positional bias in the existing encoder–decoder-based re-ranking model. To address these two limitations, this study proposes a hierarchical supervision method that jointly performs summary and sentence-level supervision. For sentence-level supervision, we designed two sentence-level loss functions: intra- and inter-intra-sentence ranking losses. Compared to the existing abstractive summarization model, the proposed method exhibited a performance improvement for both the CNN/DM and XSum datasets. The proposed model outperformed the baseline model under a few-shot setting.

This paper presents an in-depth study of ChatGPT, a state-of-the-art language model that is revolutionizing generative text. We provide a comprehensive analysis of its architecture, training data, and evaluation metrics and explore its advancements and enhancements over time. Additionally, we examine the capabilities and limitations of ChatGPT in natural language processing (NLP) tasks, including language translation, text summarization, and dialogue generation. Furthermore, we compare ChatGPT to other language generation models and discuss its applicability in various tasks. Our study also addresses the ethical and privacy considerations associated with ChatGPT and provides insights into mitigation strategies. Moreover, we investigate the role of ChatGPT in cyberattacks, highlighting potential security risks. Lastly, we showcase the diverse applications of ChatGPT in different industries and evaluate its performance across languages and domains. This paper offers a comprehensive exploration of ChatGPT's impact on the NLP field. [ABSTRACT FROM AUTHOR]

Abstract In this study, we introduce and evaluate a novel extractive text summarization methodology, “SummarEyes,” based on the visual interaction of the user with the text, using eye-tracking data, as opposed to the traditional approaches based on analysis of textual content only. We conducted a large-scale user study aiming to collect eye-tracking data while reading the text to be summarized. We utilized various user’s implicit attention metrics to generate novel eye-tracking-based text summarization models and compared them both to eye-tracking models typically using only a single feature of the gaze duration and to traditional, as well as state-of-the-art summarization methods, based solely on textual features. The models’ quality was evaluated in terms of ROUGE scores using intrinsic evaluation on the datasets we had generated, relating gaze behavior to personalized and DUC gold-standard summaries. The experimental results showed that “SummarEyes” significantly outperformed the other summarizers in predicting both the user’s personalized summarization and the generic gold standard summaries. With the increasing availability of eye-tracking technology, this research can lead to a new generation of effective user-centric text summarization tools. [ABSTRACT FROM AUTHOR]

Problem solving applications require users to exercise caution in their data usage practices. Prior to installing these applications, users are encouraged to read and comprehend the terms of service, which address important aspects such as data privacy, processes, and policies (referred to as information elements). However, these terms are often lengthy and complex, making it challenging for users to fully grasp their content. Additionally, existing transparency analytics tools typically rely on the manual extraction of information elements, resulting in a time-consuming process. To address these challenges, this paper proposes a novel approach that combines information visualization and machine learning analyses to automate the retrieval of information elements. The methodology involves the creation and labeling of a dataset derived from multiple software terms of use. Machine learning models, including naïve Bayes, BART, and LSTM, are utilized for the classification of information elements and text summarization. Furthermore, the proposed approach is integrated into our existing visualization tool TranspVis to enable the automatic detection and display of software information elements. The system is thoroughly evaluated using a database-connected tool, incorporating various metrics and expert opinions. The results of our study demonstrate the promising potential of our approach, serving as an initial step in this field. Our solution not only addresses the challenge of extracting information elements from complex terms of service but also provides a foundation for future research in this area. [ABSTRACT FROM AUTHOR]

This work proposes a sentence network-based approach for performing the task of multi-document text summarization. The sentences of the input set of documents are represented by the nodes of the network. Weighted edges are added between the nodes to represent the semantic similarity between the corresponding sentences. The network has a multilayer structure, where each layer corresponds to an individual input document. This helps in effective differentiation between the inter-document and intra-document edges. A hyperparameter, namely layering factor, has been used to alter the strength of inter-document connections through reinforcement or weakening. It is hypothesized that the summary sentence nodes must act as effective information spreaders in the sentence network. Summary generation is performed by identifying the influential nodes of the network using VoteRank scheme. A comparative study with different network measures, such as Weighted Degree, PageRank, Betweenness centrality, and Closeness centrality reveals the efficacy of the proposed VoteSumm technique for multi-document text summarization. Improved performance is observed when an additional pre-processing step of syntactic simplification is applied on the raw text. Performance is further improved when keyword information is included in the simplified texts. [ABSTRACT FROM AUTHOR]

A novel approach to the semantic comparison of texts based on the metric method to calculate semantic distances between lexical units is put forward. The supplementary semantic information is provided through semes of the words composing the texts, or through their semantic fields. The proposed method takes into account semantic polarity and yields, for two paraphrase sentences, more feasible results than the conventional approaches based on word occurrences. The described approach may be useful for linguistic theory as well as for a variety of Natural Language Processing tasks based on supervised learning that require semantic information: computer lexicography, semantic analysis, information search and retrieval, document classification, text summarization, and understanding machine translation and others. [ABSTRACT FROM AUTHOR]

In this paper, we present a rigorous mathematical examination of generative pre-trained transformer (GPT) models and their autoregressive self-supervised learning mechanisms. We begin by defining natural language space and knowledge space, which are two key concepts for understanding the dimensionality reduction process in GPT-based large language models (LLMs). By exploring projection functions and their inverses, we establish a framework for analyzing the language generation capabilities of these models. We then investigate the GPT representation space, examining its implications for the models' approximation properties. Finally, we discuss the limitations and challenges of GPT models and their learning mechanisms, considering trade-offs between complexity and generalization, as well as the implications of incomplete inverse projection functions. Our findings demonstrate that GPT models possess the capability to encode knowledge into low-dimensional vectors through their autoregressive self-supervised learning mechanism. This comprehensive analysis provides a solid mathematical foundation for future advancements in GPT-based LLMs, promising advancements in natural language processing tasks such as language translation, text summarization, and question answering due to improved understanding and optimization of model training and performance. [ABSTRACT FROM AUTHOR]

Initiated by the University Consortium of Geographic Information Science (UCGIS), the GIS&T Body of Knowledge (BoK) is a community‐driven endeavor to define, develop, and document geospatial topics related to geographic information science and technologies (GIS&T). In recent years, GIS&T BoK has undergone rigorous development in terms of its topic re‐organization and content updating, resulting in a new digital version of the project. While the BoK topics provide useful materials for researchers and students to learn about GIS, the semantic relationships among the topics, such as semantic similarity, should also be identified so that a better and automated topic navigation can be achieved. Currently, the related topics are either defined manually by editors or authors, which may result in an incomplete assessment of topic relationships. To address this challenge, our research evaluates the effectiveness of multiple natural language processing (NLP) techniques in extracting semantics from text, including both deep neural networks and traditional machine learning approaches. Besides, a novel text summarization—KACERS (Keyword‐Aware Cross‐Encoder‐Ranking Summarizer)—is proposed to generate a semantic summary of scientific publications. By identifying the semantic linkages among key topics, this work guides the future development and content organization of the GIS&T BoK project. It also offers a new perspective on the use of machine learning techniques for analyzing scientific publications and demonstrates the potential of the KACERS summarizer in semantic understanding of long text documents. [ABSTRACT FROM AUTHOR]

The tremendous increase in documents available on the Web has turned finding the relevant pieces of information into a challenging, tedious, and time-consuming activity. Text summarization is an important natural language processing (NLP) task used to reduce the reading requirements of text. Automatic text summarization is an NLP task that consists of creating a shorter version of a text document which is coherent and maintains the most relevant information of the original text. In recent years, automatic text summarization has received significant attention, as it can be applied to a wide range of applications such as the extraction of highlights from scientific papers or the generation of summaries of news articles. In this research project, we are focused mainly on abstractive text summarization that extracts the most important contents from a text in a rephrased form. The main purpose of this project is to summarize texts in German. Unfortunately, most pretrained models are only available for English. We therefore focused on the German BERT multilingual model and the BART monolingual model for English, with a consideration of translation possibilities. As the source of the experiment setup, took the German Wikipedia article dataset and compared how well the multilingual model performed for German text summarization when compared to using machine-translated text summaries from monolingual English language models. We used the ROUGE-1 metric to analyze the quality of the text summarization. [ABSTRACT FROM AUTHOR]

Featured Application: Hierarchical clause annotation could be applied in many downstream tasks of natural language processing, including abstract meaning representation parsing, semantic dependency parsing, text summarization, argument mining, information extraction, question answering, machine translation, etc. Most natural-language-processing (NLP) tasks suffer performance degradation when encountering long complex sentences, such as semantic parsing, syntactic parsing, machine translation, and text summarization. Previous works addressed the issue with the intuition of decomposing complex sentences and linking simple ones, such as rhetorical-structure-theory (RST)-style discourse parsing, split-and-rephrase (SPRP), text simplification (TS), simple sentence decomposition (SSD), etc. However, these works are not applicable for semantic parsing such as abstract meaning representation (AMR) parsing and semantic dependency parsing due to misalignments with semantic relations and unavailabilities to preserve the original semantics. Following the same intuition and avoiding the deficiencies of previous works, we propose a novel framework, hierarchical clause annotation (HCA), for capturing clausal structures of complex sentences, based on the linguistic research of clause hierarchy. With the HCA framework, we annotated a large HCA corpus to explore the potentialities of integrating HCA structural features into semantic parsing with complex sentences. Moreover, we decomposed HCA into two subtasks, i.e., clause segmentation and clause parsing, and provide neural baseline models for more-silver annotations. In evaluating the proposed models on our manually annotated HCA dataset, the performances of clause segmentation and parsing resulted in 91.3% F1-scores and 88.5% Parseval scores, respectively. Due to the same model architectures employed, the performance differences of the clause/discourse segmentation and parsing subtasks was reflected in our HCA corpus and compared discourse corpora, where our sentences contained more segment units and fewer interrelations than those in the compared corpora. [ABSTRACT FROM AUTHOR]

Bug reports vary in length, while some bug reports are lengthy, others are too brief to describe bugs in detail. In such a case, duplicate bug reports can serve as valuable resources for enriching bug descriptions. However, existing bug summarization methods mainly focused on summarizing a single bug report. In this paper, we focus on summarizing duplicate bug reports. By doing so, we aim to obtain an informative summary of bug reports while reducing redundant sentences in the summary. We apply several text summarization methods to duplicate bug reports. We then compare summarization results generated by different summarization methods and identify the most effective method for summarizing duplicate bug reports. Our comparative experiment reveals that the extractive multi-document method based on TF-IDF is the most effective in the summarization. This method successfully captures the relevant information from duplicate bug reports, resulting in comprehensive summaries. These results contribute to the advancement of bug summarization techniques, especially in summarizing duplicate bug reports. [ABSTRACT FROM AUTHOR]

With the rapid growth of social media platforms, digitization of official records, and digital publication of articles, books, magazines, and newspapers, lots of data are generated every day. This data is a foundation of information and contains a vast amount of text that may be complex, ambiguous, redundant, irrelevant, and unstructured. Therefore, we require tools and methods that can help us understand and automatically summarize the vast amount of generated text. There are mainly two types of approaches to perform text summarization: abstractive and extractive. In Abstractive Text Summarization, a concise summary is generated by including the salient features of the input documents and paraphrasing documents using new sentences and phrases. While in Extractive Text Summarization, a summary is produced by selecting and combining the most significant sentences and phrases from the source documents. The researchers have given numerous techniques for both kinds of text summarization. In this work, we classify Extractive Text Summarization approaches and review them based on their characteristics, techniques, and performance. We have discussed the existing Extractive Text Summarization approaches along with their limitations. We also classify and discuss evaluation measures and provide the research challenges faced in Extractive Text Summarization. [ABSTRACT FROM AUTHOR]

Copyright of Przeglad Elektrotechniczny is the property of Przeglad Elektrotechniczny and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Due to the exponential overflow of textual information in various fields of knowledge and on the internet, it is very challenging to extract important information or to generate a summary from some multi-document collection in a specific field. With such a gigantic amount of textual content, human text summarization becomes impractical since it is expensive and consumes a lot of time and effort. So, developing automatic text summarization (ATS) systems is becoming increasingly essential. ATS approaches are either extractive or abstractive. The extractive approach is simpler and faster than the abstractive approach. This work proposes an extractive ATS system that aims to extract a small subset of sentences from a large multi-document text. First, the whole text is preprocessed by applying some natural language processing techniques such as sentences segmentation, words tokenization, removal of stop-words, and stemming to provide a structured representation of the original document collection. Based on this structured representation, the ATS problem is formulated as a multi-objective optimization (MOO) problem that optimizes the extracted summary to maintain the coverage of the main text content while avoiding redundant information. Secondly, an evolutionary sparse multi-objective algorithm is developed to solve the formulated large-scale MOO. The output of this algorithm is a set of non-dominated summaries (Pareto front). A novel criterion is proposed to select the target summary from the Pareto front. The proposed ATS system has been examined using (DUC) datasets, and the output summaries have been evaluated using (ROUGE) metrics and compared with the literature. [ABSTRACT FROM AUTHOR]

In recent years, combining web scraping techniques with Natural Language Processing (NLP) has emerged as a powerful approach to unlock deeper insights from unstructured textual data. This research study presents a detailed exploration of web scraping using Natural Language Processing (NLP) techniques, demonstrating how these methodologies can be synergistically integrated to extract and analyze unstructured text from diverse web sources. This research study analyzes the challenges posed by unstructured data on the web and how NLP can play a pivotal role in converting this text into structured and actionable information. The first part of the paper covers an overview of web scraping methods, including rule-based parsing, XPath queries, and the use of web scraping libraries such as BeautifulSoup and Scrapy. The second part of this research work focuses on applying NLP techniques to process and analyze the extracted textual data. Further, the preprocessing steps such as tokenization, stemming, and stop word removal, are analyzed followed by more advanced techniques like Named Entity Recognition. [ABSTRACT FROM AUTHOR]

Automatic summarization extracts the core content of the text by compressing the source text information. At present, most abstractive summarization tasks use a sequence-to-sequence model based on attention mechanism, but the model decodes the generated summary with low semantic accuracy and high content repetition rate. So this paper proposes an automatic text summarization method combining pre-training and attention enhancement to improve the quality of the generated summary. This model is based on the PGN model with coverage mechanism. Firstly, the Transformer encoder pre-training text acquires the semantic relationship between characters. Then, it uses the attention enhancement mechanism to make the current moment attention distribution of the decoder refer to the historical moment attention distribution in the decoder of the sequence-to-sequence model. Finally, it optimizes beam search algorithm to suppress the model's decoder predictive output short sentences. The experimental evaluation index uses the ROUGE value. The experimental results on the public datasets of NLPCC2018 and LCSTS indicate that, compared with the PGN model training results with the coverage mechanism, ROUGE-1, ROUGE-2 and ROUGE-L indicators are all obtained improved, which fully verifies the advancement and effectiveness of the method proposed in this paper. [ABSTRACT FROM AUTHOR]

With the abundance of conversations happening everywhere, dialogue summarization plays an increasingly important role in the real world. However, dialogues inevitably involve many personal pronouns, which hinder the performance of existing dialogue summarization models. This work proposes a framework named WHORU to inject external personal pronoun resolution (PPR) information into abstractive dialogue summarization models. A simple and effective PPR method for the dialogue domain is further proposed to reduce time and space consumption. Experiments demonstrated the superiority of the proposed methods. More importantly, WHORU achieves new SOTA results on SAMSum and AMI datasets. [ABSTRACT FROM AUTHOR]

Text Summarization models facilitate biomedical clinicians and researchers in acquiring informative data from enormous domain-specific literature within less time and effort. Evaluating and selecting the most informative sentences from biomedical articles is always challenging. This study aims to develop a dual-mode biomedical text summarization model to achieve enhanced coverage and information. The research also includes checking the fitment of appropriate graph ranking techniques for improved performance of the summarization model. The input biomedical text is mapped as a graph where meaningful sentences are evaluated as the central node and the critical associations between them. The proposed framework utilizes the top k similarity technique in a combination of UMLS and a sampled probability-based clustering method which aids in unearthing relevant meanings of the biomedical domain-specific word vectors and finding the best possible associations between crucial sentences. The quality of the framework is assessed via different parameters like information retention, coverage, readability, cohesion, and ROUGE scores in clustering and non-clustering modes. The significant benefits of the suggested technique are capturing crucial biomedical information with increased coverage and reasonable memory consumption. The configurable settings of combined parameters reduce execution time, enhance memory utilization, and extract relevant information outperforming other biomedical baseline models. An improvement of 17% is achieved when the proposed model is checked against similar biomedical text summarizers. [ABSTRACT FROM AUTHOR]

As the popularity of large language models (LLMs) has risen over the course of the last year, led by GPT-3/4 and especially its productization as ChatGPT, we have witnessed the extensive application of LLMs to text summarization. However, LLMs do not intrinsically have the power to verify the correctness of the information they supply and generate. This research introduces a novel approach to abstractive summarization, aiming to address the limitations of LLMs in that they struggle to understand the truth. The proposed method leverages extracted knowledge graph information and structured semantics as a guide for summarization. Building upon BART, one of the state-of-the-art sequence-to-sequence pre-trained LLMs, multi-source transformer modules are developed as an encoder, which are capable of processing textual and graphical inputs. Decoding is performed based on this enriched encoding to enhance the summary quality. The Wiki-Sum dataset, derived from Wikipedia text dumps, is introduced for evaluation purposes. Comparative experiments with baseline models demonstrate the strengths of the proposed approach in generating informative and relevant summaries. We conclude by presenting our insights into utilizing LLMs with graph external information, which will become a powerful aid towards the goal of factually correct and verified LLMs. [ABSTRACT FROM AUTHOR]

Text summarization is a subtask of natural language processing referring to the automatic creation of a concise and fluent summary that captures the main ideas and topics from one or multiple documents. Earlier literature surveys focus on extractive approaches, which rank the top-n most important sentences in the input document and then combine them to form a summary. As argued in the literature, the summaries of these approaches do not have the same lexical flow or coherence as summaries that are manually produced by humans. Newer surveys elaborate abstractive approaches, which generate a summary with potentially new phrases and sentences compared to the input document. Generally speaking, contrary to the extractive approaches, the abstractive ones create summaries that are more similar to those produced by humans. However, these approaches still lack the contextual representation needed to form fluent summaries. Recent advancements in deep learning and pretrained language models led to the improvement of many natural language processing tasks, including abstractive summarization. Overall, these surveys do not present a comprehensive evaluation framework that assesses the aforementioned approaches. Taking the above into account, the contribution of this survey is fourfold: (i) we provide a comprehensive survey of the state-of-the-art approaches in text summarization; (ii) we conduct a comparative evaluation of these approaches, using well-known datasets from the related literature, as well as popular evaluation scores such as ROUGE-1, ROUGE-2, ROUGE-L, ROUGE-LSUM, BLEU-1, BLEU-2 and SACREBLEU; (iii) we report on insights gained on various aspects of the text summarization process, including existing approaches, datasets and evaluation methods, and we outline a set of open issues and future research directions; (iv) we upload the datasets and the code used in our experiments in a public repository, aiming to increase the reproducibility of this work and facilitate future research in the field. [ABSTRACT FROM AUTHOR]

This research suggests utilizing natural language processing to summarize the YouTube video transcript in a way that preserves important details. Around one-third of YouTube users in India who view videos on their mobile devices spend more than 48 hours each month on the platform, according to a Google study. It has gotten tougher to sit through these films, some of which may go longer than they should. Our efforts can even be in naught if we can't find the information we need in them. It can take a lot of time and effort to find movies that provide the information we are really seeking for. There are a lot of online videos, for instance, where the speaker wants to capture the audience's attention, but we won't understand the message unless we watch the entire video. The built-in model accepts links to YouTube videos and creates summaries for each one. It also gives the user the ability to choose the algorithm and the percentage of summaries they want to create. Another feature that the user has access to is the ability to transform the generated summary from text to speech and to translate it into the Hindi language. [ABSTRACT FROM AUTHOR]

Automatic Text Summarization (ATS) models yield outcomes with insufficient coverage of crucial details and poor degrees of novelty. The first issue resulted from the lengthy input, while the second problem resulted from the characteristics of the training dataset itself. This research employs the divide-and-conquer approach to address the first issue by breaking the lengthy input into smaller pieces to be summarized, followed by the conquest of the results in order to cover more significant details. For the second challenge, these chunks are summarized by models trained on datasets with higher novelty levels in order to produce more human-like and concise summaries with more novel words that do not appear in the input article. The results demonstrate an improvement in both coverage and novelty levels. Moreover, we defined a new metric to measure the novelty of the summary. Finally, the findings led us to conclude that the novelty levels are more significantly influenced by the training dataset itself, as in CNN/DM, than by other factors like the training model or its training objective, as in Pegasus. [ABSTRACT FROM AUTHOR]

Over the past few years, the explainable artificial intelligence (XAI) model receives a broad desire for investigation. The natural language processing (NLP) commune is reaching the fundamental change too – constructing a set of paradigms, which describe the preference on a few chief jobs devoid of influencing the execution. Abstractive Text Summarization (ATS) remains the job of building summary sentences by fusing factualities out of disparate source sentences and compressing them into a smaller portrayal when sustaining data content and comprehensive sense. This remains extremely arduous and long-drawn-out for people to physically summarize huge text documents. This study proffers Ontology-based Knowledge Aware Multi-focus Conditional Generative Adversarial Network (OKAM-CGAN) for novel documents. This could build novel sentences by analyzing many finer pieces than sentences, especially, semantic phrases. The proffered OKAM-CGAN comprises 3 prime portions – ontology aware knowledge-based document representation module, multitask and multi-focus learning unit, and an adversarial network unit. Experiential assessment is performed by correlating with advanced methodologies like RNN-W, CopyNet, GCU, Seq2Seq, and KESG concerning ROUGE scores. Consequently, it is observed that the proffered OKAM-CGAN attains 42.1% of ROUGE-L, 40% of accuracy, 45%of precision, and 53% of recall for the CNN/Daily Mail database and 45% of ROUGE-L, 4% of accuracy, 54% of precision, and 57% of recall for the Edmunds database. [ABSTRACT FROM AUTHOR]

Text summarization is a prominent task in natural language processing (NLP) that condenses lengthy texts into concise summaries. Despite the success of existing supervised models, they often rely on datasets of well-constructed text pairs, which can be insufficient for languages with limited annotated data, such as Chinese. To address this issue, we propose a semi-supervised learning method for text summarization. Our method is inspired by the cycle-consistent adversarial network (CycleGAN) and considers text summarization as a style transfer task. The model is trained by using a similar procedure and loss function to those of CycleGAN and learns to transfer the style of a document to its summary and vice versa. Our method can be applied to multiple languages, but this paper focuses on its performance on Chinese documents. We trained a T5-based model and evaluated it on two datasets, CSL and LCSTS, and the results demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Artificial Intelligence in the field of Law has introduced a lot of innovations to aid legal experts, and one of those are summarizing systems. Text summarization is the process of condensing text into a compact summary highlighting the important parts of the original text. As years pass by, numerous text summarization systems are being developed, and improvements keep getting made. These advancements have been helping humans in easing labor work with summarizing lengthy documents. With this, the researchers would like to propose a tool that will summarize legal documents, specifically cases obtained from the Philippine Supreme Court website. This includes extracting meanings and analyzation of words, and similarity of sentences from the original legal document obtained from the website of the Philippine Supreme Court. The tool will take the content of the original document, analyze the words, structure, and sentences according to context, and finally produce the summary. The proposed tool will be called SUMMIT, from the words "Summarize It". The method to be used to summarize the documents using the tool is called Latent Semantic Analysis (LSA). The researchers will use Python as their programming language in developing the summarizer tool. [ABSTRACT FROM AUTHOR]

The term 'executed linguistics' corresponds to an interdisciplinary domain in which the solutions are identified and provided for real-time language-related problems. The exponential generation of text data on the Internet must be leveraged to gain knowledgeable insights. The extraction of meaningful insights from text data is crucial since it can provide valueadded solutions for business organizations and end-users. The Automatic Text Summarization (ATS) process reduces the primary size of the text without losing any basic components of the data. The current study introduces an Applied Linguistics-based English Text Summarization using a Mixed Leader-Based Optimizer with Deep Learning (ALTS-MLODL) model. The presented ALTS-MLODL technique aims to summarize the text documents in the English language. To accomplish this objective, the proposed ALTSMLODL technique pre-processes the input documents and primarily extracts a set of features. Next, the MLO algorithm is used for the effectual selection of the extracted features. For the text summarization process, the Cascaded Recurrent Neural Network (CRNN) model is exploited whereas the Whale Optimization Algorithm (WOA) is used as a hyperparameter optimizer. The exploitation of the MLO-based feature selection and the WOA-based hyperparameter tuning enhanced the summarization results. To validate the performance of the ALTS-MLODL technique, numerous simulation analyses were conducted. The experimental results signify the superiority of the proposed ALTS-MLODL technique over other approaches. [ABSTRACT FROM AUTHOR]

The term "Medical Text summarization" refers to the process of extracting or collecting more useful information from medical articles in a concise manner. Every day, the count of medical publications increases continuously, and applying text summarization techniques can minimize the time needed to manually transform medical papers into a summarized version. This study's goal is to present a summary of recent works in medical text summarization from 2018 to 2022. It includes 15 papers covering different methodologies such as Clinical Context-Aware (CCA), Prognosis Quality Recognition (PQR), Bidirectional Encoder Representations From Transformers (BERT), Generative Adversarial Networks (GAN), Recurrent Neural Network (RNN), and Sequence-To-Sequence (seq-2-seq) model. Also, the paper describes the newest datasets (PubMed, arXiv, SUMPUBMED, Evidence-Based Medicine Summarization, COVID-19 Open Research, BioMed Central, Clinical Context-Aware, Biomedical Relation Extraction Dataset, Semantic Scholar Open Research Corpus, and Prognosis Quality Recognition) and evaluation metrics (Recall-Oriented Understudy for Gisting Evaluation (ROUGE), F1 Metric, Bilingual Evaluation Understudy (BLEU), BERTScore (BS), and Accuracy) used in medical text summarization. [ABSTRACT FROM AUTHOR]

Recent cross-lingual summarization research has pursued the use of a unified end-to-end model which has demonstrated a certain level of improvement in performance and effectiveness, but this approach stitches together multiple tasks and makes the computation more complex. Less work has focused on alignment relationships across languages, which has led to persistent problems of summary misordering and loss of key information. For this reason, we first simplify the multitasking by converting the translation task into an equal proportion of cross-lingual summary tasks so that the model can perform only cross-lingual summary tasks when generating cross-lingual summaries. In addition, we splice monolingual and cross-lingual summary sequences as an input so that the model can fully learn the core content of the corpus. Then, we propose a reinforced regularization method based on the model to improve its robustness, and build a targeted ABO mechanism to enhance the semantic relationship alignment and key information retention of the cross-lingual summaries. Ablation experiments are conducted on three datasets of different orders of magnitude to demonstrate the effective enhancement of the model by the optimization approach; they outperform the mainstream approaches on the cross-lingual summarization task and the monolingual summarization task for the full dataset. Finally, we validate the model's capabilities on a cross-lingual summary dataset of professional domains, and the results demonstrate its superior performance and ability to improve cross-lingual sequencing. [ABSTRACT FROM AUTHOR]

Huge quantities of audio and video material are available at universities and teaching institutions, but their use can be limited because of the lack of intelligent search tools. This paper describes a possible way to set up an indexing scheme that offers a smart search modality, that combines semantic analysis of video/audio transcripts with the exact time positioning of uttered words. The proposal leverages NLP methods for topic modeling with lexical analysis of lessons' transcripts and builds a semantic hierarchical index into the corpus of lessons analyzed. Moreover, using abstracting summarization, the system can offer short summaries on the subject semantically implied by the search carried out. [ABSTRACT FROM AUTHOR]

The rapid growth of text data on the Internet requires effective automatic text summarization techniques. This study proposes a hybrid text summarization approach that combines a Multi-hidden Recurrent Neural Network and a mayfly-harmony search algorithm. The neural network generates a feature vector for each sentence. The mayflyharmony search algorithm then optimizes the feature weights to extract the most relevant sentences for the summary. This manuscript capacity provides essential information and expertise that can be effectively summarised using Efficient Abstractive Text Summarising (EATS) techniques. This project aimed to extract informative summaries from various articles by utilizing regularly utilized handcrafted elements from literature. A Multi-hidden Recurrent Neural Network (MRNN) was used to generate a feature vector, and a new feature assortment strategy called Mayfly-Harmony Search (MHS) was applied for feature extraction. The number of sentences, word frequency, title similarity, term frequency-inverse sentence frequency, sentence location, sentence length, sentencesentence similarity, sentence phrases, proper nouns, n-gram co-occurrence, and document length were the features used. By taking diverse Mayfly Algorithm explanations found from other expanses of the search space and processing them with Harmony Search, the suggested hybrid of the Mayfly Algorithm and Harmony Search was employed to produce superior results. [ABSTRACT FROM AUTHOR]

A large variety of complaint reports reflect subjective information expressed by citizens. A key challenge of text summarization for complaint reports is to ensure the factual consistency of generated summary. Therefore, in this paper, a simple and weakly supervised framework considering factual consistency is proposed to generate a summary of city-based complaint reports without pre-labeled sentences/words. Furthermore, it considers the importance of entity in complaint reports to ensure factual consistency of summary. Experimental results on the customer review datasets (Yelp and Amazon) and complaint report dataset (complaint reports of Shenyang in China) show that the proposed framework outperforms state-of-the-art approaches in ROUGE scores and human evaluation. It unveils the effectiveness of our approach to helping in dealing with complaint reports. [ABSTRACT FROM AUTHOR]

Indonesian automatic text summarization research is developed rapidly. The quality, especially readability aspect, of text summary can be reached if the meaning of the text can be maintained properly. Therefore, this research aims to enhance the quality of extractive Indonesian automatic text summarization with considering the quality of structured representation of text. This research uses sequential pattern mining (SPM) to produce This research use SPM to produce sequence of words (SoW) as structured text representation using PrefixSpan algorithm. Then, SPM is combined with feature-based approach using sentence scoring method to produce summary. The experiment result using IndoSum dataset shows that even though the combination of SPM and sentence scoring can increase the precision value of recall-oriented understudy for gisting evaluation (ROUGE)-1, ROUGE-2, and ROUGE-L, from 0.68 to 0.76, 0.54 to 0.69, and 0.51 to 0.72. Especially, combination of SPM and Sentence Scoring can enhance precision, recall, and f-measure of ROUGE-L that consider the order of word occurance in measurement. SPM increases ROUGE-L f-measure value of sentence scoring from 0.32 to 0.36. Moreover, combination of sentence scoring and SPM is better than SumBasic that used as feature-based approach in the previous Indonesian text summarization research. [ABSTRACT FROM AUTHOR]

Abstract Due to the exponential overflow of textual information in various fields of knowledge and on the internet, it is very challenging to extract important information or to generate a summary from some multi-document collection in a specific field. With such a gigantic amount of textual content, human text summarization becomes impractical since it is expensive and consumes a lot of time and effort. So, developing automatic text summarization (ATS) systems is becoming increasingly essential. ATS approaches are either extractive or abstractive. The extractive approach is simpler and faster than the abstractive approach. This work proposes an extractive ATS system that aims to extract a small subset of sentences from a large multi-document text. First, the whole text is preprocessed by applying some natural language processing techniques such as sentences segmentation, words tokenization, removal of stop-words, and stemming to provide a structured representation of the original document collection. Based on this structured representation, the ATS problem is formulated as a multi-objective optimization (MOO) problem that optimizes the extracted summary to maintain the coverage of the main text content while avoiding redundant information. Secondly, an evolutionary sparse multi-objective algorithm is developed to solve the formulated large-scale MOO. The output of this algorithm is a set of non-dominated summaries (Pareto front). A novel criterion is proposed to select the target summary from the Pareto front. The proposed ATS system has been examined using (DUC) datasets, and the output summaries have been evaluated using (ROUGE) metrics and compared with the literature.

Efficient Transformer models typically employ local and global attention methods, or utilize hierarchical or recurrent architectures, to process long text inputs in natural language processing tasks. However, these models face challenges in terms of sacrificing either efficiency, accuracy, or compatibility to develop their application in longer sequences. To maintain both the accuracy of global attention and the efficiency of local attention, while keeping a good compatibility to be easily applied to an existing pre-trained model, in this paper, we propose multi-level local attention (Mulla attention), which is a hierarchical local attention that acts on both the input sequence and multiple pooling sequences of different granularity simultaneously, thus performing long-range modeling while maintaining linear or log-linear complexity. We apply Mulla attention to LongT5 and implement our LongT5-Mulla sequence-to-sequence model, without introducing new parameters except for positional embeddings. Experiments show that our model can surpass all baseline models, including two original variants of LongT5, in the 8~16k-input long text summarization task on the Multi-News, arXiv and WCEP-10 datasets, with improvements of at least +0.22, +0.01, +0.52 percentage points (pp) averaged Rouge scores respectively, while at the meantime being able to effectively process longer sequences that have 16~48k tokens with at least 52.6% lower memory consumption than LongT5-tglobal, and +0.56~1.62 pp averaged Rouge scores higher than LongT5-local. These results demonstrate that our proposed LongT5-Mulla model can effectively process long sequences and extend the maximum input length for long text tasks from 16k to 48k while maintaining accuracy and efficiency.

In recent years,pretrained language models have developed rapidly,pushing natural language processing into a whole new stage of development.To help researchers understand where and how the powerful pretrained language models can be applied in natural language processing,this paper surveys the state-of-the-art of its application.Specifically,we first briefly review typical pretrained language models,including monolingual,multilingual and Chinese pretrained models.Then,we discuss these pretrained language models' contributions to five different natural language processing tasks:information extraction,sentiment analysis,question answering,text summarization,and machine translation.Finally,we discuss some challenges faced by the applications of pretrained language models.

Since the introduction of neural networks,text summarization techniques continue to attract the attention of resear-chers.Similarly,generative adversarial networks(GANs)can be used for text summarization because they can generate text features or learn the distribution of the entire sample and produce correlated sample points.In this paper,we exploit the features of generative adversarial networks(GANs)and use them for abstractive text summarization tasks.The proposed generative adversa-rial model has three components:a generator,which encodes the input sentences into shorter representations;a readability discriminator,which forces the generator to create comprehensible summaries;and a similarity discriminator,which acts on the generator to curb the discorrelation between the outputted text summarization and the inputted text summarization.In addition,Chinese WordNet is used as an external knowledge base in the similarity discriminator to enhance the discriminator.The generator is optimized using policy gradient algorithm,converting the problem into reinforcement learning.Experimental results show that the proposed model gets high ROUGE evaluation scores.

This paper use text summarization techniques as a tool for empirical legal research, creating a summary of the decisions given the phrases predictive power with regards to the decision outcome. A dataset of habeas corpus decisions from various courts in Brazil is used that explicitly cite the COVID pandemic as a reason for requesting the release of the patients. A predictive model is created and through this analysis we propose to find the arguments most correlated with the outcome.

Recently, for the abstractive summarization task, most of proposed techniques have adopted the deep recurrent neural network (RNN)-based sequential auto-encoding architecture to effectively learn and generate meaningful summaries for different input documents. However, most of recent RNN-based models always suffer the challenges related to the involvement of much capturing high-frequency/reparative phrases in long documents during the training process which leads to the outcome of trivial and generic summaries are generated. In addition, the lack of thorough analysis on the sequential and long-range dependency relationships between words within different contexts while learning the textual representation also makes the achieved summaries unnatural and incoherent. In order to deal with these challenges, in this paper we proposed a novel semantic-enhanced generative adversarial network (GAN)-based approach for abstractive text summarization task, called as: SGAN4AbSum. We use an adversarial training strategy for our text summarization model which trains the generator and discriminator to simultaneously handle the summary generation and distinguishing the generated summary with the ground-truth one. The input of generator is the jointed rich-semantic and global structural latent representations of training documents which are achieved by applying a combined BERT and graph convolutional network textual embedding mechanism. Extensive experiments in benchmark datasets demonstrate the effectiveness of our proposed SGAN4AbSum which achieve the competitive ROUGE-based scores in comparing with state-of-the-art abstractive text summarization baselines. [ABSTRACT FROM AUTHOR]

Background: Biomedical ontologies are representations of biomedical knowledge that provide terms with precisely defined meanings. They play a vital role in facilitating biomedical research in a cross-disciplinary manner. Quality issues of biomedical ontologies will hinder their effective usage. One such quality issue is missing concepts. In this study, we introduce a logical definition-based approach to identify potential missing concepts in SNOMED CT. A unique contribution of our approach is that it is capable of obtaining both logical definitions and fully specified names for potential missing concepts. Method: The logical definitions of unrelated pairs of fully defined concepts in non-lattice subgraphs that indicate quality issues are intersected to generate the logical definitions of potential missing concepts. A text summarization model (called PEGASUS) is fine-tuned to predict the fully specified names of the potential missing concepts from their generated logical definitions. Furthermore, the identified potential missing concepts are validated using external resources including the Unified Medical Language System (UMLS), biomedical literature in PubMed, and a newer version of SNOMED CT. Results: From the March 2021 US Edition of SNOMED CT, we obtained a total of 30,313 unique logical definitions for potential missing concepts through the intersecting process. We fine-tuned a PEGASUS summarization model with 289,169 training instances and tested it on 36,146 instances. The model achieved 72.83 of ROUGE-1, 51.06 of ROUGE-2, and 71.76 of ROUGE-L on the test dataset. The model correctly predicted 11,549 out of 36,146 fully specified names in the test dataset. Applying the fine-tuned model on the 30,313 unique logical definitions, 23,031 total potential missing concepts were identified. Out of these, a total of 2,312 (10.04%) were automatically validated by either of the three resources. Conclusions: The results showed that our logical definition-based approach for identification of potential missing concepts in SNOMED CT is encouraging. Nevertheless, there is still room for improving the performance of naming concepts based on logical definitions. [ABSTRACT FROM AUTHOR]

Automatic text summarization is one of the most promising solutions to the ever-growing challenges of textual data as it produces a shorter version of the original document with fewer bytes, but the same information as the original document. Despite the advancements in automatic text summarization research, research involving the development of automatic text summarization methods for documents written in Hausa, a Chadic language widely spoken in West Africa by approximately 150,000,000 people as either their first or second language, is still in early stages of development. This study proposes a novel graph-based extractive single-document summarization method for Hausa text by modifying the existing PageRank algorithm using the normalized common bigrams count between adjacent sentences as the initial vertex score. The proposed method is evaluated using a primarily collected Hausa summarization evaluation dataset comprising of 113 Hausa news articles on ROUGE evaluation toolkits. The proposed approach outperformed the standard methods using the same datasets. It outperformed the TextRank method by 2.1%, LexRank by 12.3%, centroid-based method by 19.5%, and BM25 method by 17.4%. [ABSTRACT FROM AUTHOR]

Considering the ever-growing volume of electronic documents made available in our daily lives, the need for an efficient tool to capture their gist increases as well. Automatic text summarization, which is a process of shortening long text and extracting valuable information, has been of great interest for decades. Due to the difficulties of semantic understanding and the requirement of large training data, the development of this research field is still challenging and worth investigating. In this paper, we propose an automated text summarization approach with the adaptation of static and contextual representations based on an extractive approach to address the research gaps. To better obtain the semantic expression of the given text, we explore the combination of static embeddings from GloVe (Global Vectors) and the contextual embeddings from BERT (Bidirectional Encoder Representations from Transformer) and GPT (Generative Pre-trained Transformer) based models. In order to reduce human annotation costs, we employ policy gradient reinforcement learning to perform unsupervised training. We conduct empirical studies on the public dataset, Gigaword. The experimental results show that our approach achieves promising performance and is competitive with various state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

In today's world as the data on the web is increasing it becomes a challenge to identify the relevant information. Automatic text summarization (ATS) provides a significant answer to it. In this paper, fuzzy logic and shark smell optimization (SSO) based algorithm for extractive text summarization is proposed. Shark Smell Optimization has been used to assign a weight to eight different features to identify the less and more important text features of text summarization. Then, the Fuzzy Logic's inference system is utilized to generate fuzzy rules, and finally an automated summary is generated. The system generated summaries have been tested against the reference summaries from the DUC 2002, DUC 2003, DUC-2004 and TAC-11 dataset and ROUGE toolkit has been used for the evaluation of the proposed solution. Results of the proposed algorithm are compared against traditional methods and the rouge score suggested that the proposed algorithm generates better results than other methods. [ABSTRACT FROM AUTHOR]

In the area of text summarization, there have been significant advances recently. In the meantime, the current trend in text summarization is focused more on news summarization. Therefore, developing a synthesis approach capable of extracting, comparing, and ranking sentences is vital to create a summary of various news articles in the context of erroneous online data. It is necessary, however, for the news summarization system to be able to deal with multi-document summaries due to content redundancy. This paper presents a method for summarizing multi-document news web pages based on similarity models and sentence ranking, where relevant sentences are extracted from the original article. English-language articles are collected from five news websites that cover the same topic and event. According to our experimental results, our approach provides better results than other recent methods for summarizing news. [ABSTRACT FROM AUTHOR]

Long document summarization poses obstacles to current generative transformer-based models because of the broad context to process and understand. Indeed, detecting long-range dependencies is still challenging for today's state-of-the-art solutions, usually requiring model expansion at the cost of an unsustainable demand for computing and memory capacities. This paper introduces Emma, a novel efficient memory-enhanced transformer-based architecture. By segmenting a lengthy input into multiple text fragments, our model stores and compares the current chunk with previous ones, gaining the capability to read and comprehend the entire context over the whole document with a fixed amount of GPU memory. This method enables the model to deal with theoretically infinitely long documents, using less than 18 and 13 GB of memory for training and inference, respectively. We conducted extensive performance analyses and demonstrate that Emma achieved competitive results on two datasets of different domains while consuming significantly less GPU memory than competitors do, even in low-resource settings. [ABSTRACT FROM AUTHOR]

Eliciting informative user opinions from online reviews is a key success factor for innovative product design and development. The unstructured, noisy, and verbose nature of user reviews, however, often complicate large-scale need finding in a format useful for designers without losing important information. Recent advances in abstractive text summarization have created the opportunity to systematically generate opinion summaries from online reviews to inform the early stages of product design and development. However, two knowledge gaps hinder the applicability of opinion summarization methods in practice. First, there is a lack of formal mechanisms to guide the generative process with respect to different categories of product attributes and user sentiments. Second, the annotated training datasets needed for supervised training of abstractive summarization models are often difficult and costly to create. This article addresses these gaps by (1) devising an efficient computational framework for abstractive opinion summarization guided by specific product attributes and sentiment polarities, and (2) automatically generating a synthetic training dataset that captures various degrees of granularity and polarity. A hierarchical multi-instance attribute-sentiment inference model is developed for assembling a high-quality synthetic dataset, which is utilized to fine-tune a pretrained language model for abstractive summary generation. Numerical experiments conducted on a large dataset scraped from three major e-Commerce retail stores for apparel and footwear products indicate the performance, feasibility, and potentials of the developed framework. Several directions are provided for future exploration in the area of automated opinion summarization for user-centered design. [ABSTRACT FROM AUTHOR]

The increasing amount of legal information available online is overwhelming for both citizens and legal professionals, making it difficult and time-consuming to find relevant information and keep up with the latest legal developments. Automatic text summarization techniques can be highly beneficial as they save time, reduce costs, and lessen the cognitive load of legal professionals. However, applying these techniques to legal documents poses several challenges due to the complexity of legal documents and the lack of needed resources, especially in linguistically under-resourced languages, such as the Greek language. In this paper, we address automatic summarization of Greek legal documents. A major challenge in this area is the lack of suitable datasets in the Greek language. In response, we developed a new metadata-rich dataset consisting of selected judgments from the Supreme Civil and Criminal Court of Greece, alongside their reference summaries and category tags, tailored for the purpose of automated legal document summarization. We also adopted several state-of-the-art methods for abstractive and extractive summarization and conducted a comprehensive evaluation of the methods using both human and automatic metrics. Our results: (i) revealed that, while extractive methods exhibit average performance, abstractive methods generate moderately fluent and coherent text, but they tend to receive low scores in relevance and consistency metrics; (ii) indicated the need for metrics that capture better a legal document summary's coherence, relevance, and consistency; (iii) demonstrated that fine-tuning BERT models on a specific upstream task can significantly improve the model's performance. [ABSTRACT FROM AUTHOR]