Search

One of the important application of natural language processing (NLP) is Name Entity Recognition (NER). It automatically recognise and categorise named entities in a document. Named Entities can be the name of an individual, group, place, etc. It is crucial to the success of NER applications including text summarization, machine translation and information extraction and retrieval. It is one of the most useful application tools for a variety of topics and languages. Despite its widespread use and effectiveness in English, this field is currently under investigation for other Indian languages, such as Bodo. Due to the lack of resources and a high-quality dataset, NER in Bodo is a difficult task. In this research, a deep learning-based NER tagger is investigated for the Bodo language and NER tagged dataset is generated for Bodo language using Docanno and enlarge the dataset size by employing a data augmentation technique. As there is no Bodo NER baseline model to compared with, we employed several deep learning techniques for Bodo NER System and compared their results. We achieved an accuracy of 99.62%, precision of 99.75%, recall of 98.74% and F-score of 99.35% when employed with LSTM and character based. This study also highlights GRU and CNN based models performance in Bodo NER task. [ABSTRACT FROM AUTHOR]

Building an efficient summarization and search tool for research articles is a complex task that involves interdisciplinary expertise in NLP, database management, web development, and user experience design. With the rapid growth of the scientific content, manually reading and selecting important content of research articles became challenging. Thus, there is a need for a summarization tool to help scholars reading their content fast along with a search tool that will find important contents and keep them in organized way. To summarize the contents of different articles a summarization tool is proposed in this work that generates extractive and abstractive summaries. Along with summarization a search engine also been proposed in this work that save the searched results in a comma-separated value (CSV) format including the search queries and meta information of articles such as keyword, title, author name, URL, year of publication, abstracts and summaries. These CSVs help users to get idea about article contents in offline mode without reading or searching the whole text. The efficiency of the summarizer tool is evaluated in terms of precision(pr), recall(re) and F-measure(F-m) of Rouge-1(R1), Rouge-2(R2), Rouge_sum(R_sum) and Bertscore(BS) measures for ten research articles. The average pr, re and F-m obtained from BS are 42%, 42% and 42% for extractive summarization and 41%, 41% and 41% for abstractive summarization. This tool will be helpful to research scholars in the collection of literature and the preparation of related work for their research. [ABSTRACT FROM AUTHOR]

This scientific research study focuses on the advancements in Natural Language Processing (NLP) driven by large-scale parallel corpora and presents a comprehensive methodology for creating a parallel, multilingual corpus using NLP techniques and semantic technologies, with a particular focus on washing machine manuals. The study highlights the significant progress made in NLP through the utilization of large-scale parallel corpora and advanced NLP techniques. The successful creation of a parallel, multilingual corpus for washing machine manuals, coupled with the integration of semantic technologies and ontology modeling, demonstrates the broad applicability and potential of NLP in diverse domains.The research covers various aspects, including text extraction, segmentation, and the development of specialized pipelines for question-answering, translation, and text summarization tailored for washing machine manuals. Translation experiments using fine-tuned models demonstrated the feasibility of providing washing machine manuals in local languages, expanding accessibility and understanding for users worldwide. Additionally, the study explored text summarization using a powerful transformer-based model, which exhibited remarkable proficiency in generating concise and coherent summaries from complex input texts. The implementation of a question-answering pipeline showcased the effectiveness of various language models in handling question-answering tasks with high accuracy and effectiveness.Additionally, the article discusses the processes of data collection, information preparation, ontology creation, alignment strategies, and text analytics. Furthermore, the study addresses the challenges and potential future developments in this field, offering insights into the promising applications of NLP in the context of washing machine manuals. [ABSTRACT FROM AUTHOR]

In the present era, it is challenging for people to extract crucial information due to the ongoing evolution of textual data on the internet as well as in online resources. Thus, only a procedure known as Text Summarization (TS) can be used to acquire the required information. Therefore, the most common and practical method of summary is known as extractive text summarization, which involves selecting the most pertinent sentences from the original text material. In a relatively short period of time, it can gather the most priceless information. In order to extract the phrases that serve as a document's summary, this study applies scoring-based optimisation models employing Ant Colony Optimisation (ACO), Butterfly Optimisation (BO), and Particle Swarm Optimisation (PSO). The model is simulated on the DUC 2006 dataset, and the TOPSIS approach is used to validate the results for various DUC documents. The PSO-based summarizer, however, performs noticeably better than the other two summarizers based on rank. [ABSTRACT FROM AUTHOR]

The difficulty of accurately summarising Assamese text content is a significant barrier in natural language processing (NLP). Manually summarising lengthy Assamese texts is time-consuming and labor-intensive. As a result, automatic text summarization has developed as a critical NLP study topic. In this study, we integrate the Transformer and Self-Attention approaches to develop an abstract text summarization model. This Transformer-based technique uses self-attention approaches to successfully manage co-reference concerns in Assamese text, enhancing overall system understanding. This proposed approach improves the efficiency of text summarization greatly. We exhaustively evaluated the model using the Assamese dataset (AD-50), which contains human-produced summaries, to assess its performance. When compared to current state-of-the-art baseline models, our model outperformed them. On the AD-50 dataset, for example, our suggested model obtained a low training loss of 0.0022 during 20 training epochs, as well as an amazing model accuracy of 47.15 percentage. This research marks a substantial advancement in the field of Assamese abstractive text summarization, with intriguing implications for practical applications in NLP. [ABSTRACT FROM AUTHOR]

In the current era of internet, a huge volume of opinionated sentiment reviews is generated on a daily basis. Normally, such reviews are enriched with deep sentiments and express the opinions on various features of a product or service under consideration. Identification of a set of important domain features opens an opportunity to build automatic product and service summarization system. It is observed that the common phenomena is to consider the frequently appearing words as the domainb features. However, not all frequently appearing words can be domain-specific features. In this paper, a novel Domain Feature Miner (DOMINER) approach is proposed for robust extractive summarization. The entire domain feature set mining problem is modelled as a clustering problem. Firstly, the bond energy based clustering technique is employed to cluster the domain features based on their frequency and co-appearance counts. Later, relevant clusters are extracted for the final set of domain feature set retrieval. The Proposed DOMINER scheme is extensively evaluated against quantitative performance metrics such as precision, recall, and F-score for six diversified domains such as Cellphone, Camera, Laptop, Tablet, Television, and Hotel. Experimental results on benchmark data sets reveal that proposed DOMINER scheme mines high quality of domain feature from unstructured reviews evident from precision, recall, and F-score values as 78.10%, 64.21%, and 70.48%, respectively against state of the art existing schemes. The high quality domain features extraction from DOMINER help generate the robust extractive summaries for product and services. [ABSTRACT FROM AUTHOR]

Data augmentation entails artificially expanding the dataset's size by applying various transformations to the existing raw data. Enhancing the quality and quantity of the datasets with varying sizes by employing varieddata augmentation techniques has immense importance in the field on Natural Language Processing. Several notable applications for instance text classification, sentiment analysis, text summarization, etc. have proven to be benefitted immensely with the employment of text augmentation techniques. Hence, the paper focuses on efficient text classification using varied datasets of different sizes; small- 500 instances, medium-5564 instances and large-43934 instances.The work considers the standard DistilBERT model, a popular transformer-based language model and presents the impact on the performance of the modelafter employing different text augmentation techniques. The study specifically focuses on three augmentation methods: (a) Synonym augmentation:that involves replacing words with their synonyms to enhance vocabulary diversity and generalization, (b) Contextual word embeddings that enriches semantic understanding by leveraging pre-trained language models, and (c) Black translation that entails translating the text into a another different language and then translating it back, introducing variations in the data and capturing different linguistic patterns.Additionally,the work also discusses the combined effect of employing all three augmentation techniques simultaneously. Moreover, the study also aims compares the relation between the dataset sizes and the performance of the augmentation techniques. The study considers three standard datasets for the analysis and presents a comprehensive analysis using accuracy and F1 score as evaluation metrics. The results highlight the efficacy of each technique across small, medium, and large datasets, enabling a nuanced understanding of their benefits in different data scenarios. The findings indicate the varying degrees of improvement achieved through each augmentation technique.The enhancement achieved by applying text augmentation varied from around 2% on large datasets to 20% on smaller datasets. [ABSTRACT FROM AUTHOR]

In recent years, combining web scraping techniques with Natural Language Processing (NLP) has emerged as a powerful approach to unlock deeper insights from unstructured textual data. This research study presents a detailed exploration of web scraping using Natural Language Processing (NLP) techniques, demonstrating how these methodologies can be synergistically integrated to extract and analyze unstructured text from diverse web sources. This research study analyzes the challenges posed by unstructured data on the web and how NLP can play a pivotal role in converting this text into structured and actionable information. The first part of the paper covers an overview of web scraping methods, including rule-based parsing, XPath queries, and the use of web scraping libraries such as BeautifulSoup and Scrapy. The second part of this research work focuses on applying NLP techniques to process and analyze the extracted textual data. Further, the preprocessing steps such as tokenization, stemming, and stop word removal, are analyzed followed by more advanced techniques like Named Entity Recognition. [ABSTRACT FROM AUTHOR]

Artificial Intelligence in the field of Law has introduced a lot of innovations to aid legal experts, and one of those are summarizing systems. Text summarization is the process of condensing text into a compact summary highlighting the important parts of the original text. As years pass by, numerous text summarization systems are being developed, and improvements keep getting made. These advancements have been helping humans in easing labor work with summarizing lengthy documents. With this, the researchers would like to propose a tool that will summarize legal documents, specifically cases obtained from the Philippine Supreme Court website. This includes extracting meanings and analyzation of words, and similarity of sentences from the original legal document obtained from the website of the Philippine Supreme Court. The tool will take the content of the original document, analyze the words, structure, and sentences according to context, and finally produce the summary. The proposed tool will be called SUMMIT, from the words "Summarize It". The method to be used to summarize the documents using the tool is called Latent Semantic Analysis (LSA). The researchers will use Python as their programming language in developing the summarizer tool. [ABSTRACT FROM AUTHOR]

Text summarization is the process of reducing text size while preserving its key points. Thanks to this process, the reading time of the text is also reduced which contributes to reaching the desired information quickly, especially in today's world where time is much more important. In addition, summarization can be used to create a solution to extract outstanding information from the text. In this study, we focus on abstract summarization, which can draw more human like conclusions from the text. A summarization study was carried out on the data set that was collected from online Turkish news sources. Rouge and Bert-score performance metrics were used to compare the performance of this study using the text to text transfer transformer (T5) method. The precision values ​​of the Rouge-1, Rouge-2, Rouge-L and Bert-score performance metrics obtained in this study were found to be 0.6913, 0.6623, 0.7528 and 0.8718, respectively. Recall values ​​were 0.9210, 0.8917, 0.9183 and 0.9138, respectively. F measure values ​​were 0.7649, 0.7338, 0.8084 and 0.8913 respectively. Considering the success of the results obtained in the study, a method that can obtain successful results for Turkish text summarization is presented and the original dataset is made available to other researchers. [ABSTRACT FROM AUTHOR]

Studies in low-resource languages have become more challenging with the increasing volume of texts in today's digital era. Also, the lack of labeled data and text processing libraries in a language further widens the research gap between high and low-resource languages, such as English and Indonesian. This has led to the use of a transfer learning approach, which applies pre-trained models to solve similar problems, even in different languages by using bilingual or cross-lingual word embedding. Therefore, this study aims to investigate two bilingual word embedding methods, namely VecMap and BiVec, for Indonesian – English language and evaluates them for bilingual lexicon induction and text summarization tasks. The generated bilingual embedding was compared with MUSE (Multilingual Unsupervised and Supervised Embeddings) as the existing multilingual word created with the generative adversarial network method. Furthermore, the VecMap was improved by creating shared vocabulary spaces and mapping the unshared ones between languages. The result showed the embedding produced by the joint methods of BiVec, performed better in intrinsic evaluation, especially with CSLS (Cross-Domain Similarity Local Scaling) retrieval. Meanwhile, the improved VecMap outperformed the regular type by 16.6% without surpassing the BiVec evaluation score. These methods enabled model transfer between languages when applied to cross-lingual-based text summarization. Moreover, the ROUGE score outperformed classical text summarization by adding only 10% of the training dataset of the target language. [ABSTRACT FROM AUTHOR]

As free online encyclopedias with massive volumes of content, Wikipedia and Wikidata are key to many Natural Language Processing (NLP) tasks, such as information retrieval, knowledge base building, machine translation, text classification, and text summarization. In this paper, we introduce WikiDes, a novel dataset to generate short descriptions of Wikipedia articles for the problem of text summarization. The dataset consists of over 80k English samples on 6987 topics. We set up a two-phase summarization method — description generation (Phase I) and candidate ranking (Phase II) — as a strong approach that relies on transfer and contrastive learning. For description generation, T5 and BART show their superiority compared to other small-scale pre-trained models. By applying contrastive learning with the diverse input from beam search, the metric fusion-based ranking models outperform the direct description generation models significantly up to ≈ 22 ROUGE in topic-exclusive split and topic-independent split. Furthermore, the outcome descriptions in Phase II are supported by human evaluation in over 45.33% chosen compared to 23.66% in Phase I against the gold descriptions. In the aspect of sentiment analysis, the generated descriptions cannot effectively capture all sentiment polarities from paragraphs while doing this task better from the gold descriptions. The automatic generation of new descriptions reduces the human efforts in creating them and enriches Wikidata-based knowledge graphs. Our paper shows a practical impact on Wikipedia and Wikidata since there are thousands of missing descriptions. Finally, we expect WikiDes to be a useful dataset for related works in capturing salient information from short paragraphs. The curated dataset is publicly available at: https://github.com/declare-lab/WikiDes. • Wikipedia- and Wikidata-based dataset to generate short descriptions from paragraphs. • Two-phase summarization — candidate description generation and ranking. • Sentiment consistency analysis between paragraph and gold description. [ABSTRACT FROM AUTHOR]

Understanding the rules of grammar and linguistic features is essential to understanding the context of a language, which helps to understand that language. Similarly, for Natural Language processing, the linguistic feature allows understanding of the language. This paper introduced how Coreference, Word-sense, and Semantic knowledge (CWS) of linguistic features work. It would improve the Natural Language Understanding (NLU) and Natural Language Processing (NLP) tasks of any NLP model and NLP applications (either existing or new). This paper proposed a CWS pipeline method to enhance the efficiency and performance of NLP applications like text summarization, information retrieval, question-answer, machine reading comprehension, etc. The proposed CWS pipeline model used a pre-trained CoNLL-2012 coreference dataset extracted from the famous Ontonotes-5.0 dataset for the English language. The model implementation is done in Python language. The performance evaluation is done using the standard CoNLL-2012 coreference dataset for the English language. The coreference marked output is evaluated against the manually tagged gold standard dataset. The proposed CWS pipeline model gives 78.98% of the average F1 score on the MUC metric, 1.78% higher than the previous models' top result. CWS pipeline model performs better than existing models. [ABSTRACT FROM AUTHOR]

People generally turn to websites such as Yelp to find reviews for the food/restaurant before trying it first-hand. However, some reviews are so long and ambiguous that users get confused about whether the review appreciates or disparages the food. Here, websites might want to employ summarization techniques so that the crux of the study is well understood in just a sentence. The process of compressing the information/data while preserving its idea is text summarization. The present work's main objective is to empirically investigate the suitable algorithm for summarizing reviews so that users do not need to spend time reading through the entire study. The performance of the conducted investigations is analyzed with the metrics such as cosine similarity, the score of the bilingual evaluation understudy (precision), and recall-oriented understudy for gisting evaluation analysis. The experiments show that the text summarization technique, LexRank outperforms other methods with the precision and recall values of 0.586 and 0.346, respectively. [ABSTRACT FROM AUTHOR]

The summary of a large piece of text or a document is a concise description of the same thing. It must retain most of the important points from the document and remove any kind of verbosity. The text summarization task is an information processing scheme that given a document or a set of documents, mines the most essential content from the source by considering the user or task at hand and presents the summary in a well-formed and brief text. Summarization can be done not just on one document, it can be done on a set of documents as well, which is called multi-document summarization. The proposed approach considers and in-corporates user preferences for the summarization task. Depending on the current user or current task, the summary could differ essentially. For an instance, a summary on 'photosynthesis', would be very different for the kid in 4th Grade vs. a student doing a bachelor's in Science. Therefore, while writing a summary, the reading history of the user is employed as a source of personalizing text summarization tasks. The proposed method is experimentally evaluated in the domain of news articles and obtained better summaries capable of extracting important concepts based on user preferences explained in the document when considering the relevant domain terms in the process of multi-document text summarization. [ABSTRACT FROM AUTHOR]

Behavioral biology is one of the crucial and trending topics these days which needs proper attention by scholars for rapid development in this field. Therefore, the purpose of this paper is to ease the process of collecting all kinds of relevant and vital data available on the internet regarding this topic in different forms of media, such as news articles, research papers, and YouTube lecture videos, all into one place into a single document in a proper summarized form. For proper training of the LSTM model, the lengthy video and journal datasets are pre-processed using the T5 transformer model to generate a uniform training dataset. So, in this work, a comprehensive approach is proposed based on an abstractive form of text summarization using the seq2seq encoder-decoder model combined with a stacked LSTM layer with an attention mechanism and a T5 transformer model pre-processor. Therefore, a proper hybrid model, T5LSTM-RNN, is implemented to generate the summarized data. [ABSTRACT FROM AUTHOR]

Text summarization research is significant and challenging in the domain of natural language processing. Abstractive text summarization mainly uses the encoder-decoder framework, wherein the encoder component does not have a sufficient semantic comprehension of the input text, and there are exposure biases and semantic inconsistencies between the reference and generated summaries during the training process. We propose an improved encoder-decoder model that incorporates a hierarchical attention mechanism and multiobjective reinforcement learning. The encoder introduces a multihead self-attention mechanism to allow for the acquisition of more comprehensive semantic information from multiple angles and dimensions, while the decoder introduces a pointer-generator network to solve the out-of-vocabulary problem. Multiobjective reinforcement learning methods are constructed throughout the training process to optimize the model in terms of addressing exposure bias, maintaining semantic consistency, and enhancing readability. The results of the comparative experiments demonstrate that the proposed model significantly improved in terms of the ROUGE evaluation metric, and the generated summaries were semantically similar to the reference summaries. [ABSTRACT FROM AUTHOR]

The need for automatic text summarization is natural: there is a huge volume of information available online, which prompts for a widespread interest in extracting relevant information in a concise and understandable manner. Here, automated text summarization has been treated as an extractive single-document summarization problem in the proposed system. To solve this problem, a particle swarm optimisation (PSO) algorithm-based approach is suggested, with the goal of producing good summaries in terms of content coverage, informativeness, and readability. This paper introduces XSumm-PSO: a new approach based on PSO optimization technique in a supervised manner for extractive summarization. Further, this paper also contributes a new feature "incorrect word" that captures misspelled words in the candidate sentences. This feature is combined with nine existing features used by proposed model to generate error free summaries. As a result, the proposed XSumm-PSO framework produces superior performance achieving improvements of +2.7%, +0.8%, and +0.8% for ROUGE-1, ROUGE-2, and ROUGE-L scores, respectively, on DUC 2002 dataset, over state-of-the-art techniques. The corresponding improvements on the CNN/DailyMail dataset are +0.97%, +0.25%, and +0.49%. We also performed sample t-test, showing the proposed approach is statistically consistent across various runs. • A PSO-based technique optimized in a supervised manner using ROGUE-1 is proposed. • The suggested model solves a single-document extractive text summarization task. • A new feature "incorrect word" is also introduced in this work. • We evaluate our proposed model on DUC-2002 and CNN/DailyMail benchmark datasets. • The suggested model generalizes better and produces better accuracy than SOTA. [ABSTRACT FROM AUTHOR]

Cyberbullying poses a digital threat to society. In this survey, we explain what cyberbullying is and its various forms. We focus on social media platforms and instant messaging apps that are susceptible to cyberbullying, discussing how we can identify such behavior in these spaces. Moving on, we conduct a systematic review of publicly available datasets in different languages, exploring techniques for data preprocessing, feature representation, and methodologies used in textual analysis for cyberbullying detection. We specifically look at natural language-based and platform-specific preprocessing methods. We also cover popular feature representation techniques like sentiment analysis, user information, text summarization, symbols, images, and word embedding for detecting cyberbullying. Next, we categorize existing techniques, including machine learning and neural networks, highlighting research gaps. Additionally, we discuss the challenges associated with current datasets and methods. This survey aims to provide early researchers with insights into cyberbullying literature and guide them in exploring potential research directions. • Analysis of prior literature for cyberbullying detection. • Proposes possible future research directions based on identified research challenges. • Identifies the need for high-quality datasets with contextual information. • Develop mitigation systems relevant in real-time. [ABSTRACT FROM AUTHOR]

The adoption of Deep Neural Networks (DNNs) has greatly benefited Natural Language Processing (NLP) during the past decade. However, the demands of long document analysis are quite different from those of shorter texts, while the ever increasing size of documents uploaded online renders automated understanding of lengthy texts a critical issue. Relevant applications include automated Web mining, legal document review, medical records analysis, financial reports analysis, contract management, environmental impact assessment, news aggregation, etc. Despite the relatively recent development of efficient algorithms for analyzing long documents, practical tools in this field are currently flourishing. This article serves as an entry point into this dynamic domain and aims to achieve two objectives. First of all, it provides an introductory overview of the relevant neural building blocks, serving as a concise tutorial for the field. Secondly, it offers a brief examination of the current state-of-the-art in two key long document analysis tasks: document classification and document summarization. Sentiment analysis for long texts is also covered, since it is typically treated as a particular case of document classification. Consequently, this article presents an introductory exploration of document-level analysis, addressing the primary challenges, concerns, and existing solutions. Finally, it offers a concise definition of "long text/document", presents an original overarching taxonomy of common deep neural methods for long document analysis and lists publicly available annotated datasets that can facilitate further research in this area. [ABSTRACT FROM AUTHOR]

Mathematical teaching techniques encompass a diverse range of approaches for students to grasp mathematical concepts. Currently, various mathematical teaching techniques exist, and it is crucial to extract personalized teaching techniques based on students' knowledge level. Cognitive Computing systems in the learning domain open a substantial range of possible alternatives, thereby helping both students and teachers. These systems enhance the human cognitive capabilities by developing cognitive assistant, which is a flexible and adaptive learning software and a continuous learning system. In this paper, a personalized cognitive layered framework is proposed to map the student's reasoning level with the mathematical teaching techniques. The analysis is done with real time data collected from the School for Grades 1 to 6 and mathematical teaching techniques extracted from mathematical research articles. The Hypothesis Evidence Extraction (HypoEE) algorithm is proposed to extract optimal teaching techniques. The proposed system follows the pipeline of cognitive computing processes such as extraction of factors of importance, knowledge assertion graph construction, node prioritization, correlation extraction, text summarization, hypothesis identification, evidence extraction and scoring, hypothesis refinement, inference and decision formulation. The system is evaluated using performance metrics and cognitive metrics. The validation is done by three mathematical subject experts. The results showcase a Confidence Weighted Score (CWS) of 0.96, an accuracy of 0.994, 0.92 precision, 0.88 recall, 0.89 F1 score, 0.83 specificity and cognitive metric scores indicating a taskability of 0.89, 0.86 knowledge capacity and 0.89 knowledge utilization. [Display omitted] • A personalized cognitive layered framework for deriving the most effective and personalized mathematical teaching strategy. • A cognitive system driven by evidence-based inference to personalize education. • A text summarization algorithm crafted to produce concise summaries from large volumes of text. • Application of the Hypothesis Evidence Extraction (HypoEE) algorithm to extract optimal mathematical teaching techniques based on individual reasoning levels. • Analyzing performance through expert assessments and cognitive evaluation metrics. [ABSTRACT FROM AUTHOR]

Text summarization is a widely-researched problem among scholars in the field of natural language processing. Multiple techniques have been proposed to help tackle this problem, yet some of these methodologies may still exhibit limitations such as the requirements for large training datasets, which might not always be possible, but more importantly, the lack of interpretability or transparency of the model. In this paper, we propose using meta-learning algorithm to train a deep learning model for extractive text summarization and then using various explanatory techniques such as SHAP (Shapley, 1953), linear regression (Lederer, 2022), decision trees (Fürnkranz, 2010), and input modification to gain insights into the model's decision making process. To evaluate the effectiveness of our approach, we will compare it to other popular natural language processing models like BERT (Miller, 2019) or XLNET (Yang et al., 2020) using the ROUGE metrics (Lin, 2004). Overall, our proposed approach provides a promising solution to the limitations of existing methods and a framework for improving the explainability of deep learning models in other natural language processing tasks. • Bi-LSTM and Meta-learning produce better results compared to other popular models. • Meta-learning can be utilized to improve performance in data-scarce scenarios. • Use SHAP, regression, decision tree, and text mutation to enhance interpretability. [ABSTRACT FROM AUTHOR]

In the rapidly evolving field of natural language processing, the demand for efficient automated text summarization systems that not only distill extensive documents but also capture their nuanced thematic elements has never been greater. This paper introduces the FuzzyTP-BERT framework, a novel approach in extractive text summarization that synergistically combines Fuzzy Topic Modeling (FuzzyTM) with the advanced capabilities of Bidirectional Encoder Representations from Transformers (BERT). Unlike traditional extractive methods, FuzzyTP-BERT integrates fuzzy logic to refine topic modeling, enhancing the semantic sensitivity of summaries by allowing a more nuanced representation of word-topic relationships. This integration results in summaries that are not only coherent but also thematically rich, addressing a significant gap in current summarization technology. Extensive evaluations on benchmark datasets demonstrate that FuzzyTP-BERT significantly outperforms existing models in terms of ROUGE scores, effectively balancing topical relevance with semantic coherence. Our findings suggest that incorporating fuzzy logic into deep learning frameworks can markedly improve the quality of automated text summaries, potentially benefiting a wide range of applications in the information overload age. [ABSTRACT FROM AUTHOR]

Text summarization automatically produces a summary containing important sentences and includes all relevant important information from the original document. One of the main approaches, when viewed from the summary results, are extractive and abstractive. An extractive summary is heading towards maturity and now research has shifted towards abstractive summation and real-time summarization. Although there have been so many achievements in the acquisition of datasets, methods, and techniques published, there are not many papers that can provide a broad picture of the current state of research in this field. This paper provides a broad and systematic review of research in the field of text summarization published from 2008 to 2019. There are 85 journal and conference publications which are the results of the extraction of selected studies for identification and analysis to describe research topics/trends, datasets, preprocessing, features, techniques, methods, evaluations, and problems in this field of research. The results of the analysis provide an in-depth explanation of the topics/trends that are the focus of their research in the field of text summarization; provide references to public datasets, preprocessing and features that have been used; describes the techniques and methods that are often used by researchers as a comparison and means for developing methods. At the end of this paper, several recommendations for opportunities and challenges related to text summarization research are mentioned. [ABSTRACT FROM AUTHOR]

The scientific research process generally starts with the examination of the state of the art, which may involve a vast number of publications. Automatically summarizing scientific articles would help researchers in their investigation by speeding up the research process. The automatic summarization of scientific articles differs from the summarization of generic texts due to their specific structure and inclusion of citation sentences. Most of the valuable information in scientific articles is presented in tables, figures, and algorithm pseudocode. These elements, however, do not usually appear in a generic text. Therefore, several approaches that consider the particularity of a scientific article structure were proposed to enhance the quality of the generated summary, resulting in ad hoc automatic summarizers. This paper provides a comprehensive study of the state of the art in this field and discusses some future research directions. It particularly presents a review of approaches developed during the last decade, the corpora used, and their evaluation methods. It also discusses their limitations and points out some open problems. The conclusions of this study highlight the prevalence of extractive techniques for the automatic summarization of single monolingual articles using a combination of statistical, natural language processing, and machine learning techniques. The absence of benchmark corpora and gold standard summaries for scientific articles remains the main issue for this task. [ABSTRACT FROM AUTHOR]

Mandarin Chinese, a globally prevalent language, boasts an abundance of regularly refreshed short news texts accessible online. Consequently, devising concise summaries of these texts has emerged as a pivotal challenge for enhancing information dissemination and comprehension efficiency. To tackle this issue, we introduce DMSeqNet-mBART, an innovative model grounded in the mBART framework, positioning it as a state-of-the-art solution for Chinese short news summarization. DMSeqNet-mBART incorporates the Adaptive-DropMessage technique, an innovative methodology that intelligently discards or retains information contingent upon the attention mechanism's output. Furthermore, the model integrates several enhanced technologies, such as dynamic convolutional layers, gated residual connections, customized feed-forward networks enhanced with batch normalization, self-attention, and cross-attention, all aimed at bolstering the performance and robustness of Chinese short news summarization. Rigorous comparative experiments, conducted across six recognized Chinese short news summary datasets, demonstrate that DMSeqNet-mBART significantly surpasses industry-leading models like T5, MLC, PLCC, and GPT-4 in terms of fluency, completeness, robustness, and accuracy. These results, as validated by benchmarks including BERTScore, BLEU, and ROUGE metrics, underscore the model's superiority across diverse evaluation standards. • Introduces DMSeqNet-mBART for Chinese text summarization. • Integrates Adaptive-DropMessage, enhancing key info extraction. • Achieves SOTA in six key Chinese news summarization datasets. • Sheds light on complex Chinese text processing challenges. [ABSTRACT FROM AUTHOR]

Summarizing trending topics in large collections of Facebook posts is particularly relevant to profile social user activities and interests. However, automatically generating these summaries poses significant challenges due to the high heterogeneity of the input data, the limited fluency of extractive summaries, and the absence of abstractive summarization methods capable of handling multiple posts simultaneously. Existing abstractive models are either not suited to handle large post collections or disregard topic-level text relations. In this work, we present TASP , a novel tool for trending topic detection and summarization from English-written Facebook posts. It trains abstractive summarization models on multi-post collections by leveraging a shortlist of authoritative posts published by renowned newspapers. At inference time, TASP first creates clusters of semantically similar social posts, each one representing a distinct topic, using pre-trained transformer-based language models. Then, it generates abstractive summaries of the clusters for which authoritative information is missing. To the best of our knowledge, TASP is the first available tool suited to abstractive multi-post summarization. We test our approach on a large-scale dataset of real Facebook posts. The results show (1) The higher effectiveness of transformer-based approaches in generating topic-specific post clusters compared to traditional methods. (2) The importance of attending long pieces of text in multi-post abstractive summary generation. • A tool for abstractive summarization of multi-post collections is proposed. • It exploits topic modeling techniques to handle post heterogeneity. • It overcomes the limitations of existing approaches summarizing one post at a time. • It leverages transformers for both topic modeling and summarization. • It reports a showcase of a real large-scale data collection of UK Facebook posts. [ABSTRACT FROM AUTHOR]

Opinion mining aims to detect and extract relevant information from large quantity of customer reviews. Automatic opinion summarization then seeks to create a consensual point of view often oriented toward the main sentiment of clients to render their experience. Although factual information is valuable for companies to understand what works or not in their products, summarization approaches that convey objectivity, and constructive feedback from customer reviews have yet to be explored. We propose an adversarial multi-task learning model for document summarization to address this new issue. Our algorithm combines an autoencoder for document summarization with a gradient reversal layer to learn independent representations of subjective and sentiment-based material. We assess and compare our method on the Amazon product review dataset where we introduce an original evaluation dataset for objective summarization. We further completed the analysis with neutrality and objectivity metrics. This study demonstrates that the generated summaries carry out relevant and objective content but also emphasize the importance of various processes and layers in multi-task learners to control the information effectively. • Objective opinion summarization of customer reviews. • Unsupervised autoencoder for summarization. • Sentiment disentanglement with gradient reversal layer. • Dataset and metrics to assess neutrality and objectivity of summaries. [ABSTRACT FROM AUTHOR]

Factual inconsistency with source documents in automatically generated summaries can lead to misinformation or pose risks. Existing factual consistency (FC) metrics are constrained by their performance, efficiency, and explainability. Recent advances in Large language models (LLMs) have demonstrated remarkable potential in text evaluation but their effectiveness in assessing FC in summarization remains underexplored. Prior research has mostly focused on proprietary LLMs, leaving essential factors that affect their assessment capabilities unexplored. Additionally, current FC evaluation benchmarks are restricted to news articles, casting doubt on the generality of the FC methods tested on them. In this paper, we first address the gap by introducing TreatFact —a dataset of LLM-generated summaries of clinical texts, annotated for FC by domain experts. Moreover, we benchmark 11 LLMs for FC evaluation across news and clinical domains and analyse the impact of model size, prompts, pre-training and fine-tuning data. Our findings reveal that despite proprietary models prevailing on the task, open-source LLMs lag behind. Nevertheless, there is potential for enhancing the performance of open-source LLMs through increasing model size, expanding pre-training data, and developing well-curated fine-tuning data. Experiments on TreatFact suggest that both previous methods and LLM-based evaluators are unable to capture factual inconsistencies in clinical summaries, posing a new challenge for FC evaluation. • TreatFact: a clinical summary dataset annotated for factual consistency. • Benchmarking 11 LLMs across news and clinical domains for FC evaluation. • Analysis of key factors impacting LLM performance on factual consistency evaluation. • Proprietary LLMs outperform open-source counterparts. • Existing FC evaluation struggles with clinical summary inconsistencies. [ABSTRACT FROM AUTHOR]

Automatic Text Summarization (ATS) involves estimating the salience of information and creating coherent summaries that include all relevant and important information from the original text. Extensive research has been carried out on ATS since 1958, gradually evolving from simple to advanced techniques, including machine learning-based, neural network-based, and deep learning-based techniques. Progress has been made in both extractive and abstractive methods throughout this development. Despite numerous surveys on ATS, there remains a notable absence of a comprehensive literature review encompassing the latest advancements in deep learning techniques for text summarization. Therefore, this paper provides a Systematic Literature Review (SLR) of deep learning-based text summarization in both types (extractive and abstractive) between 2014 and 2023. To the best of our knowledge, this is the first SLR that offers a comprehensive overview of extractive and abstractive text summarization techniques based on Deep Learning models. According to the defined inclusion and exclusion criteria, 73 deep learning-based text summarization studies are selected for further investigation. The structure of the review is organized as follows. Firstly, it identifies and examines the deep learning models employed in both extractive and abstractive text summarization. Then, the input text's representation methods are identified and discussed clearly. Next, training strategies used in supervised extractive summarization techniques are identified. Furthermore, mechanisms that improve the abstractive summarization process are identified. Additionally, the most commonly used datasets and their advantages and disadvantages are discussed. The most commonly used evaluation metrics are also identified. Finally, the challenges and possible solutions to guide future research in the field are discussed. • Analyze the latest deep learning-based text summarization techniques. • Introduce a new taxonomy of deep learning models for text summarization. • Identify methods for representing input text clearly. • Identify training strategies for supervised extractive summarization techniques. • Identify the mechanisms used in abstractive summarization techniques. [ABSTRACT FROM AUTHOR]

Assigning International Classification of Diseases (ICD) codes to clinical texts is a common and crucial practice in patient classification, hospital management, and further statistics analysis. Current auto-coding methods mainly transfer this task to a multi-label classification problem. Such solutions are suffering from high-dimensional mapping space and excessive redundant information in long clinical texts. To alleviate such a situation, we introduce text summarization methods to the ICD coding regime and apply text matching to select ICD codes. We focus on the tenth revision of the ICD (ICD-10) coding and design a novel summarization-based approach (SuM) with an end-to-end strategy to efficiently assign ICD-10 code to clinical texts. In this approach, a knowledge-guided pointer network is purposed to distill and summarize key information in clinical texts precisely. Then a matching model with matching-aggregation architecture follows to align the summary result with code, tuning the one-vs-all scenario to one-vs-one matching so that the large-label-space obstacle laid in classification approaches would be avoided. The 12,788 ICD-10 coded discharge summaries from a Chinese hospital were collected to evaluate the proposed approach. Compared with existing methods, the purposed model achieves the greatest coding results with Micro AUC of 0.9548, MRR@10 of 0.7977, Precision@10 of 0.0944, and Recall@10 of 0.9439 for the TOP-50 Dataset. Results on the FULL-Dataset remain consistent. Also, the proposed knowledge encoder and applied end-to-end strategy are proven to facilitate the whole model to gain efficacy in selecting the most suitable code. The proposed automatic ICD-10 code assignment approach via text summarization can effectively capture critical messages in long clinical texts and improve the performance of ICD-10 coding of clinical texts. • The text summarization method is used for ICD-10 coding with better distillation of key information in long clinical texts. • With the guidance of the main diagnosis, the summary of the long clinical text is precise and applicable for code matching. • The text matching method facilitates ICD-10 coding with interactive features and the one-vs-one operation. • A novel end-to-end model of the summarization module and text matching module is proposed. [ABSTRACT FROM AUTHOR]

The ever-increasing volume of research literature poses challenges for researchers in keeping up with related works in their fields. Automating the generation of related work section holds promise for saving time and effort. However, current models often fall short of producing coherent and logically correct related work with multiple sentences, a phenomenon we refer to as rhetorical structure chaos. Rhetorical structure describes how adjacent spans of units are connected to each other, and logically correct rhetorical structure is essential for a well-structured related work. Hence, to tackle the rhetorical structure chaos issue, this paper explicitly incorporates rhetorical structure information into related work generation. Firstly, we conduct the first rhetorical structure analysis for related work section, which provides insights into understanding the organization and arrangement of contents within related work. Then, based on two preliminary studies on rhetorical structure, we present a novel related work generation model called RSGen, which incorporates rhetorical structure at both the encoding and decoding stages. The encoding stage is facilitated with a rhetorical structure-based graph encoder, while the decoding process is guided by a rhetorical plan — ordered sequence of rhetorical functions of the related work. We conduct extensive experiments on three related work generation datasets to evaluate the performance of our model. The results show that our approach achieves state-of-the-art performance on ROUGE metrics. An ablation study and more analyses further highlight the remarkable efficacy of introducing rhetorical structure into both the encoding and decoding stages. • The first rhetorical structure analysis for related work section. • A model enhanced with rhetorical structure at both encoding and decoding stages. • Superior performance that exceeds all baselines and achieves state-of-the-art. [ABSTRACT FROM AUTHOR]

Text summarization in medical domain is one of the most crucial chores as it deals with the critical human information. Consequently the proper summarization and key point extraction from medical deeds using pre-trained Language models is now the key figure to be focused on for the researchers. But due to the considerable amount of real-world data and enormous amount of memory requirement to train the Large Language Models (LLMs), research on these models become challenging. To overcome these challenges multiple prompting and tuning techniques are being used. In this paper, effectiveness of prompt engineering and parameter efficient fine tuning is being studied to summarize the Hospital Discharge Summary (HDS) papers effectively, so that these models can accurately interprete medical terminologies and contexts, generate brief but compact summaries, and draw out concentrated themes, which opens new approaches for the application of LLMs in healthcare and making HDS more patient-friendly. In this research LLaMA 2 (Large Language Model Meta AI) has been considered as the base model. Also, the model has been fine-tuned using QLoRA (Quantized Low Rank Adapters), which can bring down the memory usage of LLMs without compromising the data quality. This study explores the way to use LLMs on HDS datasets without the hassle of memory usage using QLoRA, into electronic health record systems to further streamline the handling and retrieval of healthcare information. • Presents a method to summarize HDS by parameter-efficient fine-tuning to a LLM. • It utilizes QLoRA fine-tuning on the LLAMA 2 to minimize the memory requirements. • A comparative analysis of prompt engineering and parameter-efficient fine-tuning is performed. • The effectiveness of the approach is evaluated using BLEU and ROGUE-L scores. • Medical professionals have conducted a quality assessment of the summarization. [ABSTRACT FROM AUTHOR]

There are different textual content summarization processes available in natural Language Processing. Amongst them abstractive textual content summarization is one of the challenging problems in natural language processing and that too, with very little research done in regional languages. Unlike other summarization techniques, which reuses the words and phrases from the source text, abstractive text summarization builds a short and concise precis of a huge text document built from the underlying message of the text not necessarily using the same words and phrases from the source. The objective of the proposed work is to create a brief and understandable abstractive summary of a Malayalam document. Malayalam is one of the 22 scheduled languages of India spoken by over 34 million people and is designated as a Classical Language in India. Being a Classical language, Malayalam has a very unique syntactic and semantic rules which makes this work more important. The proposed work attempts to create an attention mechanism to generate the summary of the source document. In this work, the goal was to compare the efficiency of Attention model with sequence to sequence baseline model of Malayalam text and thereby implementing a better abstractive text summarizer for a malayalam document. [ABSTRACT FROM AUTHOR]

With the advent of transformer-based architectures, the contextual representation of text data has leveraged the query and the document to be represented in low-dimensional dense vector space. These vectors are learned embeddings of fixed sizes, resulting in deeper text understanding. In this study, we designed a pipeline for effectively retrieving documents from a large search space by combining the deeper text understanding capabilities of the transformer-based BERT model and a phrase embedding-based query expansion model. To learn the contextual representations, we fine-tuned a deep semantic matching model by separately encoding the document and the query. The encoder model is based on the Sentence BERT (SBERT) architecture, which separately generates dense vector representations of documents and queries. The study has also addressed the maximum token length limitation of transformer-based models through the summarization of lengthy documents. In addition, to improve the clarity and completeness of short queries and reduce the semantic gap, a phrase embedding-based query expansion model is employed. The documents and their dense vectors are indexed using the Elasticsearch engine, and matched them with query vectors for retrieving query-specific documents. Finally, the BERT-based cross-encoder model is used to re-rank the relevant records for each query. It performs full self-attention over the inputs, and yields richer text interactions to produce the final results. To assess performance, experiments are conducted on two well-known datasets, TREC-CDS-2014 and OHSUMED. A comparative analysis is carried out, which clearly demonstrates that the proposed framework produced competitive retrieval results. • Transformer-based contextual text representation approach is proposed for IR. • Bi-Encoder architecture is applied to learn the representations of documents & query. • BERT-based summarization is applied to address the limitation on sequence length. • Phrase Embedding-based QE technique is employed to expand short and unclear queries. • Re-ranking of the retrieval results is performed using the cross encoder model. [ABSTRACT FROM AUTHOR]

The Transformer-based summarization models rely solely on the attention mechanism for document encoding, making it difficult to accurately capture long-range dependencies in long documents due to the presence of attention redundancy. To address this issue, we propose an extractive summarization framework guided by a topic model (TopicSum) that utilizes a heterogeneous graph neural network to leverage the topic information as document-level features during the sentence selection process, thereby capturing the long-range dependencies among sentences. The sentence-level features in this topic model align with the basic unit of the extractive summarization task. Additionally, a memory mechanism is employed to dynamically store and update the memory module, reducing the potential of repetitive information guiding sentence selection. We evaluated the model on three large document datasets, namely Pubmed, arXiv, and GovReport, and achieved significantly higher Rouge scores than previous works, including extractive and abstractive models. Furthermore, our experiments demonstrate that recent highly regarded large language models such as ChatGPT are insufficient to handle the long document summarization task directly. The proposed approach in this paper exhibits sufficient competitiveness in terms of both generation quality and deployment conditions. [ABSTRACT FROM AUTHOR]

Development of effective automatic summarization approaches for legal documents suffer from several challenges like extremely long document-summary pairs, lack of large scale training datasets with tractable document-summary token lengths, etc. In this work, we deal with the problem of legal document summarization by building a modified abstractive dataset from the original dataset. This ensures that the length of each document-summary pair is manageable and can be processed by the state of the art summarization approaches (such as BART). Secondly, we deal with the data scarcity problem by creating more number of training samples, from each of the original document-summary pair. This is done by creating multiple extractive summaries from each sample in the original dataset, following which ground-truth summary sentences are assigned to each of the extractive summary to generate new training samples. This results in a larger training dataset that can be utilized for fine-tuning summarization models. Our proposed approach has been evaluated on two different legal datasets- BillSum and Forum of Information Retrieval Evaluation (FIRE). With respect to the ROUGE metrics, the proposed approach is able to outperform pre-trained BART model fine-tuned on original dataset by (3 − 8) % for FIRE test sets, and by (1 − 3) % for the BillSum test sets. Considering the BERTScore metrics, the proposed approach obtains (1 − 2) % improvements on the FIRE test sets, while for the BillSum test sets (3 − 8) % improvements are observed. Such improvements suggest that the proposed dataset building approach can help achieve improved abstractive summarization of lengthy legal documents. • Lengthy nature and data scarcity are the two main challenges with legal documents. • Lengthy legal document summarization is handled by the proposed approach. • The data scarcity problem is handled by creating a feasible abstractive dataset. • A novel Extract-Then-Assign (ETA) approach is proposed. • ETA approach can greatly improve abstractive summarization of legal documents. [ABSTRACT FROM AUTHOR]

Automatically generating a survey draft is a challenge to text summarization research because it needs to select important sentences from important references in a large set of candidate papers for composing sections that are in line with section titles and different sections discuss the most relevant reference papers of different number, which are beyond the capability of previous text summarization approaches as they assume that all candidate papers should be included into one summary. This paper proposes an approach to generating survey draft according to a pattern consisting of sections with titles given by the user who requests the survey. The problem of generating each section can be divided into the following sub-problems: (1) rank the input scientific documents (in short documents) according to the title of a section, (2) determine the number of documents that are most relevant to the title, and (3) rank and select sentences from the selected documents according to the title. A position closeness distance of key word is proposed to rank a set of documents by measuring how closely two key words within section title are distributed within each document, which is used to rank the documents. The rationale is that the positions of the neighboring key words of a section title should be closer in more relevant documents than other words. As different sections have different number of selected documents, a method is proposed to determine the number of documents to be included into the current section based on the slope shape of the sorted rank curve of documents according to the section title. Based on the duality property of the closeness, ranks of sentences within a document can be directly obtained when the document is ranked according to the title of section, and both the importance and coherence of selected sentences can be reflected without extra calculation for ranking sentences. Experiments and manual evaluation show that the proposed methods achieve significant improvements compared with other approaches. The proposed approach is significant in applications as different surveys can be generated according to different patterns given by different users. [ABSTRACT FROM AUTHOR]

In Natural Language Processing (NLP) domain, the majority of automatic text summarization approaches depend on a prior knowledge of the language and/or the domain of the text being summarized. Such approaches requires language dependent part-of-speech taggers, parsers, databases, pre-structured lexicons, etc. In this research, we propose a novel automatic text summarization model, Text Documents - Language Agnostic Summarization Model (TxLASM), which is able to perform extractive text summarization task in language/domain agnostic manner. TxLASM depends on specific characteristics of the major elements of the text being summarized rather than its domain, context, or language and thus rules out the need for language dependent pre-processing tools, taggers, parsers, lexicons or databases. Within TxLASM , we present a novel technique for encoding the shapes of major text elements (paragraphs, sentences, n-grams and words); moreover, we present language independent preprocessing algorithms to normalize words and perform relative stemming or lemmatization. Those algorithms and its Shape-Coding technique enable the TxLASM to extract intrinsic features of text elements and score them statistically, and subsequently extract a representative summary that is independent of the text language, domain and context. TxLASM was applied on an English and Portuguese benchmark datasets, and the results were compared to twelve state-of-the-art approaches presented in recent literature. In addition, the model was applied on French and Spanish news datasets, and the results were compared to those obtained by standard commercial summarization tools. TxLASM has outperformed all the SOTA approaches as well as the commercial tools in all four languages while maintaining its language and domain agnostic nature. [ABSTRACT FROM AUTHOR]

With the development of pre-trained language models and large-scale datasets, automatic text summarization has attracted much attention from the community of natural language processing, but the progress of automatic summarization evaluation has stagnated. Although there have been efforts to improve automatic summarization evaluation, ROUGE has remained one of the most popular metrics for nearly 20 years due to its competitive evaluation performance. However, ROUGE is not perfect, there are studies have shown that it is suffering from inaccurate evaluation of abstractive summarization and limited diversity of generated summaries, both caused by lexical bias. To avoid the bias of lexical similarity, more and more meaningful embedding-based metrics have been proposed to evaluate summaries by measuring semantic similarity. Due to the challenge of accurately measuring semantic similarity, none of them can fully replace ROUGE as the default automatic evaluation toolkit for text summarization. To address the aforementioned problems, we propose a compromise evaluation framework (ROUGE-SEM) for improving ROUGE with semantic information, which compensates for the lack of semantic awareness through a semantic similarity module. According to the differences in semantic similarity and lexical similarity, summaries are classified into four categories for the first time, including good-summary, pearl-summary, glass-summary, and bad-summary. In particular, the back-translation technique is adopted to rewrite pearl-summary and glass-summary that are inaccurately evaluated by ROUGE to alleviate lexical bias. Through this pipeline framework, summaries are first classified by candidate summary classifier, then rewritten by categorized summary rewriter, and finally scored by rewritten summary scorer, which are efficiently evaluated in a manner consistent with human behavior. When measured using Pearson, Spearman, and Kendall rank coefficients, our proposal achieves comparable or higher correlations with human judgments than several state-of-the-art automatic summarization evaluation metrics in dimensions of coherence, consistency, fluency, and relevance. This also suggests that improving ROUGE with semantics is a promising direction for automatic summarization evaluation. [ABSTRACT FROM AUTHOR]

• The select-then-extract framework can better alleviate class imbalance. • Jointing the Extractive Summarization task contributes to the ECPE task. • Multi-granularity cooperative graph can better capture inter-clause relations. • The global keyword helps highlight causal cues implied in distant clauses. • Integrated inference predictor improves the efficiency of reasoning. The class imbalance problem arises in the current end-to-end models for the emotion-cause pair extraction (ECPE) task, and a large number of negative clause pairs interfere model to depict the characteristics of emotion and cause. Most existing methods adopt a window-constrained strategy to limit the relative distance between candidate clauses to a smaller range to filter negative pairs. However, those methods ignore the semantic association between distant clauses, and using an artificial window size is imperfect due to the inherent ambiguity and delicacy of emotion. Inspired by the centrality principle of discourse, in this paper we propose a select-then-extract framework for the ECPE task, where the core clauses are first selected from the original document as candidate emotion and cause clauses, and then emotions, causes and emotion-cause pairs are jointly extracted from the candidate clauses. Specifically, our model includes two main components: core clause selector and emotion-cause pairs extractor. For the Core Clause Selector , we introduce the extractive document summarization (EDS) task and present a multi-granularity semantic awareness cooperative graph model (MGCOG) to extract core clauses from documents. Compared to the previous methods, the core clause selector is more effective for alleviating the category imbalance because the global causal cues and context can be captured by learning global keywords and inter-clause relationships, which determines more efficient candidate clauses. For Emotion-Cause Pairs Extractor , we put forward a multi-task learning model to jointly extract emotions, causes and emotion-cause pairs from the selected core clauses. Here, a multi-head attention is used to further model the relationship between candidate clauses, and a co-predictor is designed for assigning scores to all possible emotion-cause pairs. We further investigate the pipeline and the joint model under the select-then-extract framework, and show that the experimental results on benchmark datasets are consistently superior to the comparative baseline models. Extensive ablation experiments also verify the effectiveness of each component. [ABSTRACT FROM AUTHOR]

The central challenge in Automatic Text Summarization (ATS) involves efficiently generating machine-generated text summaries through optimization algorithms. An ATS is a critical component for systems dealing with textual information processing. However, the current approach faces a significant hurdle due to the computational intensity of the process, particularly when employing complex optimization techniques like swarm intelligence optimization alongside a costly ATS repair operator. While the current approach yields impressive Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics for the generated summary, it struggles with inefficiencies, mainly attributed to the substantial optimization time consumed by the ATS repair operator scheme. In order to address this, a novel solution called Decomposition-based Multi-Objective Differential Evolution (MODE/D) is proposed. It is built upon the foundation of Differential Evolution for Multi-Objective Optimization (DEMO) and the weighted sum method (WS), coupled with an innovative ATS repair operator scheme. Through experimentation on Document Understanding Conferences (DUC) datasets, the novel approach of MODE/D – WS is validated by evaluating the results using ROUGE metrics. The outcomes are twofold: a remarkable reduction in serial execution time and a noteworthy enhancement over existing techniques in the scholarly domain, as evidenced by improved ROUGE-1, ROUGE-2, and ROUGE-L scores. [Display omitted] • The adoption of MODE/D and the WS method to effectively tackle the optimization problem of extractive multi-document ATS. • Employment of ROUGE metrics to gauge the quality of the proposed approach's summaries relative to alternative methodologies. • The introduction of an enhanced ATS repair operator scheme to reduce the computational demands of the repair operator. • Establish performance benchmarks by comparing MODE/D's execution times and word generation rates with other approaches in serial execution. [ABSTRACT FROM AUTHOR]

• Mapping large-scale, and high spatial resolution ecological quality (EQ) by GEE. • Overall change of EQ in Central Asia is slight inverted U-shaped. • Driving paths for EQ: urbanization, agriculture, resource, climate and policy. • Land use intensity and agricultural water use efficiency are main factors for EQ. • Interaction between human and natural factors are increasingly important for EQ. With the frequent occurrence of worldwide extreme climate events, human-induced ecosystem degradation has seriously threatened the realization of the Sustainable Development Goals (SDGs), especially in arid ecologically fragile areas. Macro-scale ecological quality (EQ) monitoring and exploration of its driving mechanisms have become research hotspots. However, this research field still lacks a method framework with strong comparability, adaptability and transferability, which significantly restricts the applicability of research results. In this study, a method framework for exploring spatial and temporal changes in EQ and their driving mechanisms based on text summarization and information extraction is constructed. Taking Central Asia as a typical case, this study outlines its EQ driving paths, explores the influencing mechanisms of representative drivers, and verifies the effectiveness of this method framework based on the comparison of the spatial and temporal evolution of EQ at multiple scales. The results indicate that the overall EQ in Central Asia exhibited a slight inverted U-shaped trend, with its driving paths falling into five categories: urban expansion, agricultural development, resource extraction, climate change, and ecological protection. The fragmentation of areas with high EQ is the main landscape characteristic in Central Asia. Furthermore, land use intensity and agricultural water use efficiency are significant factors in Central Asia's EQ evolution. Over time, the interaction between anthropogenic and natural factors has played an essential role in EQ evolution in Central Asia, with interactions between altitude, climate aridity, agricultural water use efficiency, and land use intensity gradually intensifying. This study has an implication for the construction of method framework for EQ-related study in ecologically fragile areas at the macro scale. [ABSTRACT FROM AUTHOR]

Abstractive Text Summarization (ATS) is a task of capturing information from different sources and condense it such that, content is represented well and there is no loss of information. It has been an active area of research for quiet sometime now. ATS is more closer to human generated summaries and have the capability of representing and combining multiple information. With advent of deep learning architectures, many tasks relating to natural language processing have achieved persistent and comparable high performances. It has proven advantageous and showed promising results in machine translation, speech recognition, image captioning and many others using sequence to sequence models. Language tools such as Part of Speech taggers, Named Entity Recognizer for Indian languages are not very competitive and hence, language specific techniques do not perform very well for Indian languages. Deep learning techniques are language agnostic and hence can overcome these shortcomings. In this paper, Generative Adversarial Networks(GAN(s)) are assimilated to create gist for longer piece of text in conjunction to paraphrase detection. [ABSTRACT FROM AUTHOR]

• Statistical feature based extractive approach for multi-document summarization. • Fuzzy logic system is used for sentence scoring. • Cosine similarity measure is used to tackle redundancy from summary. Nowadays abundant amount of information is available on Internet which makes it difficult for the users to locate desired information. Automatic methods are needed to efficiently sieve and scavenge useful information from the Internet. Text summarization is identified and accepted as one of the solutions to find desired contents from one or more documents. The objective of proposed multi-document summarization is to gain good content coverage with information diversity. The proposed statistical feature based model utilizes the fuzzy model to deal with the imprecise and uncertainty of feature weight. Redundancy removal using cosine similarity is presented as enrichment to proposed work. The proposed approach is compared with DUC (Document Understanding Conference) participant systems and other summarization systems such as TexLexAn, ItemSum, Yago Summarizer, MSSF and PatSum using ROUGE measure on dataset DUC 2004. The experimental results show that our proposed work achieves a significant performance improvement over the other summarizers. [ABSTRACT FROM AUTHOR]

• A hybrid approach to learn and represent users' preference knowledge. • Users' preference knowledge representation supplements ratings prediction. • Extending latent factor model to include acquired preference knowledge. • Experiments on Amazon products datasets shows performance improvements. Some existing preference prediction methods have utilized users' review texts to learn additional knowledge to support the prediction task. Such methods determine and represent users' preference knowledge by conducting user sentiment, aspect sentiment and topic analysis as recognized in the review texts. However, the discovered item topics from topic-based methods may not fit the preferences of most users while the discovered users' opinions and aspects' sentiments from sentiment based methods may not reflect each user's opinion. This paper proposes a hybrid approach to learn and represent users' preference knowledge from review texts and utilize the acquired representation to support rating prediction. Our approach assumes that user preferences are affected by relevant item aspects and majority preference which can be captured through proper summarization and representation of users' review texts. Thus, two deep learning practices are established: the recurrent neural network - Long Short-Term Memory (RNN-LSTM) architecture to learn users' preference knowledge along with item aspects which influence preferences and the Doc2Vec algorithm to convert the acquired knowledge to a suitable representation. The approach extends probabilistic matrix factorization (PMF) model by strengthening its latent factors predictions with the acquired preference knowledge which is used to regulate the predictions. Our experiments on the Amazon products dataset have revealed the capability of learning a suitable representation for users' preference knowledge and its impact on rating prediction as our proposed approach beats alternative methods. [ABSTRACT FROM AUTHOR]

Recently, large language models have shown great success on various abstractive summarization datasets. These datasets consist of numerous data that are enough to train a large number of parameters. However, for a new domain, there is a lack of labeled data to train those parameters and the model is easily overfitted to a small amount of data. In addition, because annotating document-summary pairs is too expensive and transfer learning using high-resource datasets causes a domain shifting problem, a low-resource abstractive summarization task is becoming necessary. Herein, we propose an efficient framework for low-resource abstractive summarization using a pointer-generator network and a meta-learning technique to address the above problems. Meta-learning using existing high-resource datasets enables our model to rapidly adapt to a new domain using limited data to solve the domain shifting problem. In addition, we explore the copy mechanism using a pointer-generator network that can copy words from a source document when generating a summary. The experimental results on 11 different datasets show that the proposed model outperforms the previous state-of-the-art models in low-resource abstractive summarization on most of the datasets. [ABSTRACT FROM AUTHOR]

The primary challenge faced in extractive text summarization is related to the scoring of sentences, with the critical factor for scoring being the manner in which the sentence representation is conducted. This study aims to investigate this hypothesis and to perform a detailed analysis of the impact of sentence representation techniques that have been used both semantically and syntactically. The study initially evaluated the empirical impact of individual syntactic and semantic features on the accuracy of summarization. To examine syntactic usage, a comprehensive list of 40 syntactic features was developed, while semantic representation was accomplished using sentence embeddings. Subsequently, an improved feature set was proposed that jointly utilizes syntactic and semantic features. To assess the impact of this feature set on the resulting summaries, the proposed sentence representation was tested on three distinct summarization corpora consisting of lengthy scientific documents across diverse domains. The assessment of summary evaluation and classification performance evaluation metrics was conducted to evaluate the quality of the resulting summaries. The findings of the experiments indicated that the summaries generated by the proposed feature set performed better than not only those obtained using individual features but even summaries produced by state-of-the-art methods. • In literature, text summarization has been handled by syntactic or semantic approach. • A new enhanced sentence representation was proposed to score sentences. • The sentence scoring model was a random forest fed by this enhanced representation. • The proposed model remarkably outperformed the baseline and state-of-the-art models. [ABSTRACT FROM AUTHOR]

Extractive text summarization involves selecting and combining key sentences directly from the original text, rather than generating new content. While various methods, both statistical and graph-based, have been explored for this purpose, accurately capturing the intended meaning remains a challenge. To address this, researchers are investigating innovative techniques that harness deep learning models like BERT (Bidirectional Encoder Representations from Transformers). However, BERT has limitations in summarizing lengthy documents due to input length constraints. To find a more effective solution, we propose a novel approach. This approach combines the power of BERT, a transformer network pre-trained on extensive self-supervised datasets, with BiGRU (Bidirectional Gated Recurrent Units), a recurrent neural network that captures sequential dependencies within the text for extracting salient information. Our method involves using BERT to generate sentence-level embeddings, which are then fed into the BiGRU network. This allows us to achieve a comprehensive understanding of the complete document's context. In experimental analysis conducted on arXiv and PubMed datasets, the proposed approach outperformed several state-of-the-art models. It achieved remarkable ROUGE-F scores of (46.7, 19.4, 35.4) and (47.0, 21.3, 39.7) on these datasets respectively. The proposed fusion of BERT and BiGRU significantly enhances extractive text summarization. It shows promising potential for summarizing lengthy documents and benefiting various domains that require concise and informative summaries. [ABSTRACT FROM AUTHOR]

• We proposed a novel summary method by using code vector of autoencoder. • Unsupervised abstractive summarization using Adversarially regularized autoencoder. • Conditional model is proposed and enhanced performance for review summarization. Abstractive summarization is challenging problem, especially abstractive summarization based on unsupervised learning because it must generate whole, unique sentences. In the real world, companies use abstractive summarization to understand customer feedbacks. In many cases, this work is done by humans and so is expensive in terms of time and money. Therefore, there is an increasing demand for machine learning-based abstractive summarization systems. However, most previous abstractive summarization studies were of supervised models. In this paper, we proposed novel abstractive summarization methods that can be trained unsupervisedly. One of the proposed methods is based on Adversarially Regularized Autoencoder(ARAE) model, but abstractive summary generation method for each cluster of similar customers' reviews, is newly proposed. We further proposed Conditional Adversarially Regularized Autoencoder(CARAE) model which is similar to the ARAE model but with the addition of condition nodes so that additional information about the cluster can be used during summarization. We first performed summary experiments based on Korean and additionally performed experiments on English. In the experiments, we set up some comparison models and used ROUGE and BLEU to evaluate our proposed models' performance. Overall, our proposed models outperformed the comparison models and CARAE model performed better than the ARAE model. [ABSTRACT FROM AUTHOR]