Search

Text summarization research is significant and challenging in the domain of natural language processing. Abstractive text summarization mainly uses the encoder-decoder framework, wherein the encoder component does not have a sufficient semantic comprehension of the input text, and there are exposure biases and semantic inconsistencies between the reference and generated summaries during the training process. We propose an improved encoder-decoder model that incorporates a hierarchical attention mechanism and multiobjective reinforcement learning. The encoder introduces a multihead self-attention mechanism to allow for the acquisition of more comprehensive semantic information from multiple angles and dimensions, while the decoder introduces a pointer-generator network to solve the out-of-vocabulary problem. Multiobjective reinforcement learning methods are constructed throughout the training process to optimize the model in terms of addressing exposure bias, maintaining semantic consistency, and enhancing readability. The results of the comparative experiments demonstrate that the proposed model significantly improved in terms of the ROUGE evaluation metric, and the generated summaries were semantically similar to the reference summaries. [ABSTRACT FROM AUTHOR]

Text summarization is a widely-researched problem among scholars in the field of natural language processing. Multiple techniques have been proposed to help tackle this problem, yet some of these methodologies may still exhibit limitations such as the requirements for large training datasets, which might not always be possible, but more importantly, the lack of interpretability or transparency of the model. In this paper, we propose using meta-learning algorithm to train a deep learning model for extractive text summarization and then using various explanatory techniques such as SHAP (Shapley, 1953), linear regression (Lederer, 2022), decision trees (Fürnkranz, 2010), and input modification to gain insights into the model's decision making process. To evaluate the effectiveness of our approach, we will compare it to other popular natural language processing models like BERT (Miller, 2019) or XLNET (Yang et al., 2020) using the ROUGE metrics (Lin, 2004). Overall, our proposed approach provides a promising solution to the limitations of existing methods and a framework for improving the explainability of deep learning models in other natural language processing tasks. • Bi-LSTM and Meta-learning produce better results compared to other popular models. • Meta-learning can be utilized to improve performance in data-scarce scenarios. • Use SHAP, regression, decision tree, and text mutation to enhance interpretability. [ABSTRACT FROM AUTHOR]

With the advent of transformer-based architectures, the contextual representation of text data has leveraged the query and the document to be represented in low-dimensional dense vector space. These vectors are learned embeddings of fixed sizes, resulting in deeper text understanding. In this study, we designed a pipeline for effectively retrieving documents from a large search space by combining the deeper text understanding capabilities of the transformer-based BERT model and a phrase embedding-based query expansion model. To learn the contextual representations, we fine-tuned a deep semantic matching model by separately encoding the document and the query. The encoder model is based on the Sentence BERT (SBERT) architecture, which separately generates dense vector representations of documents and queries. The study has also addressed the maximum token length limitation of transformer-based models through the summarization of lengthy documents. In addition, to improve the clarity and completeness of short queries and reduce the semantic gap, a phrase embedding-based query expansion model is employed. The documents and their dense vectors are indexed using the Elasticsearch engine, and matched them with query vectors for retrieving query-specific documents. Finally, the BERT-based cross-encoder model is used to re-rank the relevant records for each query. It performs full self-attention over the inputs, and yields richer text interactions to produce the final results. To assess performance, experiments are conducted on two well-known datasets, TREC-CDS-2014 and OHSUMED. A comparative analysis is carried out, which clearly demonstrates that the proposed framework produced competitive retrieval results. • Transformer-based contextual text representation approach is proposed for IR. • Bi-Encoder architecture is applied to learn the representations of documents & query. • BERT-based summarization is applied to address the limitation on sequence length. • Phrase Embedding-based QE technique is employed to expand short and unclear queries. • Re-ranking of the retrieval results is performed using the cross encoder model. [ABSTRACT FROM AUTHOR]

The Transformer-based summarization models rely solely on the attention mechanism for document encoding, making it difficult to accurately capture long-range dependencies in long documents due to the presence of attention redundancy. To address this issue, we propose an extractive summarization framework guided by a topic model (TopicSum) that utilizes a heterogeneous graph neural network to leverage the topic information as document-level features during the sentence selection process, thereby capturing the long-range dependencies among sentences. The sentence-level features in this topic model align with the basic unit of the extractive summarization task. Additionally, a memory mechanism is employed to dynamically store and update the memory module, reducing the potential of repetitive information guiding sentence selection. We evaluated the model on three large document datasets, namely Pubmed, arXiv, and GovReport, and achieved significantly higher Rouge scores than previous works, including extractive and abstractive models. Furthermore, our experiments demonstrate that recent highly regarded large language models such as ChatGPT are insufficient to handle the long document summarization task directly. The proposed approach in this paper exhibits sufficient competitiveness in terms of both generation quality and deployment conditions. [ABSTRACT FROM AUTHOR]

Development of effective automatic summarization approaches for legal documents suffer from several challenges like extremely long document-summary pairs, lack of large scale training datasets with tractable document-summary token lengths, etc. In this work, we deal with the problem of legal document summarization by building a modified abstractive dataset from the original dataset. This ensures that the length of each document-summary pair is manageable and can be processed by the state of the art summarization approaches (such as BART). Secondly, we deal with the data scarcity problem by creating more number of training samples, from each of the original document-summary pair. This is done by creating multiple extractive summaries from each sample in the original dataset, following which ground-truth summary sentences are assigned to each of the extractive summary to generate new training samples. This results in a larger training dataset that can be utilized for fine-tuning summarization models. Our proposed approach has been evaluated on two different legal datasets- BillSum and Forum of Information Retrieval Evaluation (FIRE). With respect to the ROUGE metrics, the proposed approach is able to outperform pre-trained BART model fine-tuned on original dataset by (3 − 8) % for FIRE test sets, and by (1 − 3) % for the BillSum test sets. Considering the BERTScore metrics, the proposed approach obtains (1 − 2) % improvements on the FIRE test sets, while for the BillSum test sets (3 − 8) % improvements are observed. Such improvements suggest that the proposed dataset building approach can help achieve improved abstractive summarization of lengthy legal documents. • Lengthy nature and data scarcity are the two main challenges with legal documents. • Lengthy legal document summarization is handled by the proposed approach. • The data scarcity problem is handled by creating a feasible abstractive dataset. • A novel Extract-Then-Assign (ETA) approach is proposed. • ETA approach can greatly improve abstractive summarization of legal documents. [ABSTRACT FROM AUTHOR]

Automatically generating a survey draft is a challenge to text summarization research because it needs to select important sentences from important references in a large set of candidate papers for composing sections that are in line with section titles and different sections discuss the most relevant reference papers of different number, which are beyond the capability of previous text summarization approaches as they assume that all candidate papers should be included into one summary. This paper proposes an approach to generating survey draft according to a pattern consisting of sections with titles given by the user who requests the survey. The problem of generating each section can be divided into the following sub-problems: (1) rank the input scientific documents (in short documents) according to the title of a section, (2) determine the number of documents that are most relevant to the title, and (3) rank and select sentences from the selected documents according to the title. A position closeness distance of key word is proposed to rank a set of documents by measuring how closely two key words within section title are distributed within each document, which is used to rank the documents. The rationale is that the positions of the neighboring key words of a section title should be closer in more relevant documents than other words. As different sections have different number of selected documents, a method is proposed to determine the number of documents to be included into the current section based on the slope shape of the sorted rank curve of documents according to the section title. Based on the duality property of the closeness, ranks of sentences within a document can be directly obtained when the document is ranked according to the title of section, and both the importance and coherence of selected sentences can be reflected without extra calculation for ranking sentences. Experiments and manual evaluation show that the proposed methods achieve significant improvements compared with other approaches. The proposed approach is significant in applications as different surveys can be generated according to different patterns given by different users. [ABSTRACT FROM AUTHOR]

In Natural Language Processing (NLP) domain, the majority of automatic text summarization approaches depend on a prior knowledge of the language and/or the domain of the text being summarized. Such approaches requires language dependent part-of-speech taggers, parsers, databases, pre-structured lexicons, etc. In this research, we propose a novel automatic text summarization model, Text Documents - Language Agnostic Summarization Model (TxLASM), which is able to perform extractive text summarization task in language/domain agnostic manner. TxLASM depends on specific characteristics of the major elements of the text being summarized rather than its domain, context, or language and thus rules out the need for language dependent pre-processing tools, taggers, parsers, lexicons or databases. Within TxLASM , we present a novel technique for encoding the shapes of major text elements (paragraphs, sentences, n-grams and words); moreover, we present language independent preprocessing algorithms to normalize words and perform relative stemming or lemmatization. Those algorithms and its Shape-Coding technique enable the TxLASM to extract intrinsic features of text elements and score them statistically, and subsequently extract a representative summary that is independent of the text language, domain and context. TxLASM was applied on an English and Portuguese benchmark datasets, and the results were compared to twelve state-of-the-art approaches presented in recent literature. In addition, the model was applied on French and Spanish news datasets, and the results were compared to those obtained by standard commercial summarization tools. TxLASM has outperformed all the SOTA approaches as well as the commercial tools in all four languages while maintaining its language and domain agnostic nature. [ABSTRACT FROM AUTHOR]

With the development of pre-trained language models and large-scale datasets, automatic text summarization has attracted much attention from the community of natural language processing, but the progress of automatic summarization evaluation has stagnated. Although there have been efforts to improve automatic summarization evaluation, ROUGE has remained one of the most popular metrics for nearly 20 years due to its competitive evaluation performance. However, ROUGE is not perfect, there are studies have shown that it is suffering from inaccurate evaluation of abstractive summarization and limited diversity of generated summaries, both caused by lexical bias. To avoid the bias of lexical similarity, more and more meaningful embedding-based metrics have been proposed to evaluate summaries by measuring semantic similarity. Due to the challenge of accurately measuring semantic similarity, none of them can fully replace ROUGE as the default automatic evaluation toolkit for text summarization. To address the aforementioned problems, we propose a compromise evaluation framework (ROUGE-SEM) for improving ROUGE with semantic information, which compensates for the lack of semantic awareness through a semantic similarity module. According to the differences in semantic similarity and lexical similarity, summaries are classified into four categories for the first time, including good-summary, pearl-summary, glass-summary, and bad-summary. In particular, the back-translation technique is adopted to rewrite pearl-summary and glass-summary that are inaccurately evaluated by ROUGE to alleviate lexical bias. Through this pipeline framework, summaries are first classified by candidate summary classifier, then rewritten by categorized summary rewriter, and finally scored by rewritten summary scorer, which are efficiently evaluated in a manner consistent with human behavior. When measured using Pearson, Spearman, and Kendall rank coefficients, our proposal achieves comparable or higher correlations with human judgments than several state-of-the-art automatic summarization evaluation metrics in dimensions of coherence, consistency, fluency, and relevance. This also suggests that improving ROUGE with semantics is a promising direction for automatic summarization evaluation. [ABSTRACT FROM AUTHOR]

• The select-then-extract framework can better alleviate class imbalance. • Jointing the Extractive Summarization task contributes to the ECPE task. • Multi-granularity cooperative graph can better capture inter-clause relations. • The global keyword helps highlight causal cues implied in distant clauses. • Integrated inference predictor improves the efficiency of reasoning. The class imbalance problem arises in the current end-to-end models for the emotion-cause pair extraction (ECPE) task, and a large number of negative clause pairs interfere model to depict the characteristics of emotion and cause. Most existing methods adopt a window-constrained strategy to limit the relative distance between candidate clauses to a smaller range to filter negative pairs. However, those methods ignore the semantic association between distant clauses, and using an artificial window size is imperfect due to the inherent ambiguity and delicacy of emotion. Inspired by the centrality principle of discourse, in this paper we propose a select-then-extract framework for the ECPE task, where the core clauses are first selected from the original document as candidate emotion and cause clauses, and then emotions, causes and emotion-cause pairs are jointly extracted from the candidate clauses. Specifically, our model includes two main components: core clause selector and emotion-cause pairs extractor. For the Core Clause Selector , we introduce the extractive document summarization (EDS) task and present a multi-granularity semantic awareness cooperative graph model (MGCOG) to extract core clauses from documents. Compared to the previous methods, the core clause selector is more effective for alleviating the category imbalance because the global causal cues and context can be captured by learning global keywords and inter-clause relationships, which determines more efficient candidate clauses. For Emotion-Cause Pairs Extractor , we put forward a multi-task learning model to jointly extract emotions, causes and emotion-cause pairs from the selected core clauses. Here, a multi-head attention is used to further model the relationship between candidate clauses, and a co-predictor is designed for assigning scores to all possible emotion-cause pairs. We further investigate the pipeline and the joint model under the select-then-extract framework, and show that the experimental results on benchmark datasets are consistently superior to the comparative baseline models. Extensive ablation experiments also verify the effectiveness of each component. [ABSTRACT FROM AUTHOR]

Recently, large language models have shown great success on various abstractive summarization datasets. These datasets consist of numerous data that are enough to train a large number of parameters. However, for a new domain, there is a lack of labeled data to train those parameters and the model is easily overfitted to a small amount of data. In addition, because annotating document-summary pairs is too expensive and transfer learning using high-resource datasets causes a domain shifting problem, a low-resource abstractive summarization task is becoming necessary. Herein, we propose an efficient framework for low-resource abstractive summarization using a pointer-generator network and a meta-learning technique to address the above problems. Meta-learning using existing high-resource datasets enables our model to rapidly adapt to a new domain using limited data to solve the domain shifting problem. In addition, we explore the copy mechanism using a pointer-generator network that can copy words from a source document when generating a summary. The experimental results on 11 different datasets show that the proposed model outperforms the previous state-of-the-art models in low-resource abstractive summarization on most of the datasets. [ABSTRACT FROM AUTHOR]

The primary challenge faced in extractive text summarization is related to the scoring of sentences, with the critical factor for scoring being the manner in which the sentence representation is conducted. This study aims to investigate this hypothesis and to perform a detailed analysis of the impact of sentence representation techniques that have been used both semantically and syntactically. The study initially evaluated the empirical impact of individual syntactic and semantic features on the accuracy of summarization. To examine syntactic usage, a comprehensive list of 40 syntactic features was developed, while semantic representation was accomplished using sentence embeddings. Subsequently, an improved feature set was proposed that jointly utilizes syntactic and semantic features. To assess the impact of this feature set on the resulting summaries, the proposed sentence representation was tested on three distinct summarization corpora consisting of lengthy scientific documents across diverse domains. The assessment of summary evaluation and classification performance evaluation metrics was conducted to evaluate the quality of the resulting summaries. The findings of the experiments indicated that the summaries generated by the proposed feature set performed better than not only those obtained using individual features but even summaries produced by state-of-the-art methods. • In literature, text summarization has been handled by syntactic or semantic approach. • A new enhanced sentence representation was proposed to score sentences. • The sentence scoring model was a random forest fed by this enhanced representation. • The proposed model remarkably outperformed the baseline and state-of-the-art models. [ABSTRACT FROM AUTHOR]

• Statistical feature based extractive approach for multi-document summarization. • Fuzzy logic system is used for sentence scoring. • Cosine similarity measure is used to tackle redundancy from summary. Nowadays abundant amount of information is available on Internet which makes it difficult for the users to locate desired information. Automatic methods are needed to efficiently sieve and scavenge useful information from the Internet. Text summarization is identified and accepted as one of the solutions to find desired contents from one or more documents. The objective of proposed multi-document summarization is to gain good content coverage with information diversity. The proposed statistical feature based model utilizes the fuzzy model to deal with the imprecise and uncertainty of feature weight. Redundancy removal using cosine similarity is presented as enrichment to proposed work. The proposed approach is compared with DUC (Document Understanding Conference) participant systems and other summarization systems such as TexLexAn, ItemSum, Yago Summarizer, MSSF and PatSum using ROUGE measure on dataset DUC 2004. The experimental results show that our proposed work achieves a significant performance improvement over the other summarizers. [ABSTRACT FROM AUTHOR]

• A hybrid approach to learn and represent users' preference knowledge. • Users' preference knowledge representation supplements ratings prediction. • Extending latent factor model to include acquired preference knowledge. • Experiments on Amazon products datasets shows performance improvements. Some existing preference prediction methods have utilized users' review texts to learn additional knowledge to support the prediction task. Such methods determine and represent users' preference knowledge by conducting user sentiment, aspect sentiment and topic analysis as recognized in the review texts. However, the discovered item topics from topic-based methods may not fit the preferences of most users while the discovered users' opinions and aspects' sentiments from sentiment based methods may not reflect each user's opinion. This paper proposes a hybrid approach to learn and represent users' preference knowledge from review texts and utilize the acquired representation to support rating prediction. Our approach assumes that user preferences are affected by relevant item aspects and majority preference which can be captured through proper summarization and representation of users' review texts. Thus, two deep learning practices are established: the recurrent neural network - Long Short-Term Memory (RNN-LSTM) architecture to learn users' preference knowledge along with item aspects which influence preferences and the Doc2Vec algorithm to convert the acquired knowledge to a suitable representation. The approach extends probabilistic matrix factorization (PMF) model by strengthening its latent factors predictions with the acquired preference knowledge which is used to regulate the predictions. Our experiments on the Amazon products dataset have revealed the capability of learning a suitable representation for users' preference knowledge and its impact on rating prediction as our proposed approach beats alternative methods. [ABSTRACT FROM AUTHOR]

• We proposed a novel summary method by using code vector of autoencoder. • Unsupervised abstractive summarization using Adversarially regularized autoencoder. • Conditional model is proposed and enhanced performance for review summarization. Abstractive summarization is challenging problem, especially abstractive summarization based on unsupervised learning because it must generate whole, unique sentences. In the real world, companies use abstractive summarization to understand customer feedbacks. In many cases, this work is done by humans and so is expensive in terms of time and money. Therefore, there is an increasing demand for machine learning-based abstractive summarization systems. However, most previous abstractive summarization studies were of supervised models. In this paper, we proposed novel abstractive summarization methods that can be trained unsupervisedly. One of the proposed methods is based on Adversarially Regularized Autoencoder(ARAE) model, but abstractive summary generation method for each cluster of similar customers' reviews, is newly proposed. We further proposed Conditional Adversarially Regularized Autoencoder(CARAE) model which is similar to the ARAE model but with the addition of condition nodes so that additional information about the cluster can be used during summarization. We first performed summary experiments based on Korean and additionally performed experiments on English. In the experiments, we set up some comparison models and used ROUGE and BLEU to evaluate our proposed models' performance. Overall, our proposed models outperformed the comparison models and CARAE model performed better than the ARAE model. [ABSTRACT FROM AUTHOR]

Highlights • Word2vec representation improves the summarization task compared to bag of words. • Feature learning using unsupervised neural networks improves the summarization task. • Unsupervised neural networks trained on word2vec vectors gives promising results. • Ensemble learning with word2vec representation obtains the best results. Abstract The vast amounts of data being collected and analyzed have led to invaluable source of information, which needs to be easily handled by humans. Automatic Text Summarization (ATS) systems enable users to get the gist of information and knowledge in a short time in order to make critical decisions quickly. Deep neural networks have proven their ability to achieve excellent performance in many real-world Natural Language Processing and computer vision applications. However, it still lacks attention in ATS. The key problem of traditional applications is that they involve high dimensional and sparse data, which makes it difficult to capture relevant information. One technique for overcoming these problems is learning features via dimensionality reduction. On the other hand, word embedding is another neural network technique that generates a much more compact word representation than a traditional Bag-of-Words (BOW) approach. In this paper, we are seeking to enhance the quality of ATS by integrating unsupervised deep neural network techniques with word embedding approach. First, we develop a word embedding based text summarization, and we show that Word2Vec representation gives better results than traditional BOW representation. Second, we propose other models by combining word2vec and unsupervised feature learning methods in order to merge information from different sources. We show that unsupervised neural networks models trained on Word2Vec representation give better results than those trained on BOW representation. Third, we also propose three ensemble techniques. The first ensemble combines BOW and word2vec using a majority voting technique. The second ensemble aggregates the information provided by the BOW approach and unsupervised neural networks. The third ensemble aggregates the information provided by Word2Vec and unsupervised neural networks. We show that the ensemble methods improve the quality of ATS, in particular the ensemble based on word2vec approach gives better results. Finally, we perform different experiments to evaluate the performance of the investigated models. We use two kind of datasets that are publically available for evaluating ATS task. Results of statistical studies affirm that word embedding-based models outperform the summarization task compared to those based on BOW approach. In particular, ensemble learning technique with Word2Vec representation surpass all the investigated models. [ABSTRACT FROM AUTHOR]

Sentiment classification aims to predict the sentiment label for a given text. Recently, several research efforts have been devoted to incorporate matching clues between text words and class labels into the learning process of text representation. However, these methods heavily rely on the availability of label content. Moreover, they simply capture the label-specific signals to measure each word's contribution by either implicitly employing a learnable label representation or explicitly leveraging the interaction between text words and labels via the interaction mechanism. To deal with these issues, in this paper, we propose a novel framework called Label-Guided Dual-view Sentiment Classifier (LGDSC). We first introduce a new strategy for generating an effective label description and then design a novel Dual-Channel Label-guided Attention Network (DLAN) to learn a text representation via two different channels. DLAN will be further leveraged to learn label-guided text representations from two different views. Extensive experimental results on four real-world datasets demonstrate that LGDSC consistently outperforms the state-of-the-art baseline methods. • Propose an inverse label entropy based strategy for generating effective label descriptions. • Design a dual-channel label-guided attention network to learn text representation via two different channels. • Extensive experiments conducted on four widely used datasets demonstrate the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

In this paper, we propose DeepSumm, a novel method based on topic modeling and word embeddings for the extractive summarization of single documents. Recent summarization methods based on sequence networks fail to capture the long range semantics of the document which are encapsulated in the topic vectors of the document. In DeepSumm, our aim is to utilize the latent information in the document estimated via topic vectors and sequence networks to improve the quality and accuracy of the summarized text. Each sentence is encoded through two different recurrent neural networks based on probabilistic topic distributions and word embeddings, and then a sequence to sequence network is applied to each sentence encoding. The outputs of the encoder and the decoder in the sequence to sequence networks are combined after weighting using an attention mechanism and converted into a score through a multi-layer perceptron network. We refer to the score obtained through the topic model as Sentence Topic Score (STS) and to the score generated through word embeddings as Sentence Content Score (SCS). In addition, we propose Sentence Novelty Score (SNS) and Sentence Position Score (SPS) and perform a weighted fusion of the four scores for each sentence in the document to compute a Final Sentence Score (FSS). The proposed DeepSumm framework was evaluated on the standard DUC 2002 benchmark and CNN/DailyMail datasets. Experimentally, it was demonstrated that our method captures both the global and the local semantic information of the document and essentially outperforms existing state-of-the-art approaches for extractive text summarization with ROUGE-1, ROUGE-2, and ROUGE-L scores of 53.2, 28.7 and 49.2 on DUC 2002 and 43.3, 19.0 and 38.9 on CNN/DailyMail dataset. • DeepSumm for extractive summarization based on seq2seq networks. • Sentence encoded with probabilistic topic distributions and word embeddings RNNs. • Proposed Sentence Novelty Score, based on sentence content and topic embeddings. • Seq2seq attention networks extract content and topic Scores. • Summary generated using four sentence scores: Content, Topic, Novelty and Position. [ABSTRACT FROM AUTHOR]

Text summarization and classification are core techniques to analyze a huge amount of text data in the big data environment. Moreover, as the need to read texts on smart phones, tablets and television as well as personal computers continues to grow, text summarization and classification techniques become more important and both of them do essential processes for text analysis in many applications. Traditional text summarization and classification techniques have individually been considered as different research fields in this literature. However, we find out that they can help each other as text summarization makes use of category information from text classification and text classification does summary information from text summarization. Therefore, we propose an effective integrated learning framework using both of summary and category information in this paper. In this framework, the feature-weighting method for text summarization utilizes a language model to combine feature distributions in each category and text, and one for text classification does the sentence importance scores estimated from the text summarization. In the experiments, the performances of the integrated framework are better than ones of individual text summarization and classification. In addition, the framework has some advantages of easy implementation and language independence because it is based on only simple statistical approaches and POS tagger. [ABSTRACT FROM AUTHOR]

Text summarization is either extractive or abstractive. Extractive summarization is to select the most salient pieces of information (words, phrases, and/or sentences) from a source document without adding any external information. Abstractive summarization allows an internal representation of the source document so as to produce a faithful summary of the source. In this case, external text can be inserted into the generated summary. Because of the complexity of the abstractive approach, the vast majority of work in text summarization has adopted an extractive approach. In this work, we focus on concepts fusion and generalization, i.e. where different concepts appearing in a sentence can be replaced by one concept which covers the meanings of all of them. This is one operation that can be used as part of an abstractive text summarization system. The main goal of this contribution is to enrich the research efforts on abstractive text summarization with a novel approach that allows the generalization of sentences using semantic resources. This work should be useful in intelligent systems more generally since it introduces a means to shorten sentences by producing more general (hence abstractions of the) sentences. It could be used, for instance, to display shorter texts in applications for mobile devices. It should also improve the quality of the generated text summaries by mentioning key (general) concepts. One can think of using the approach in reasoning systems where different concepts appearing in the same context are related to one another with the aim of finding a more general representation of the concepts. This could be in the context of Goal Formulation, expert systems, scenario recognition, and cognitive reasoning more generally. We present our methodology for the generalization and fusion of concepts that appear in sentences. This is achieved through (1) the detection and extraction of what we define as generalizable sentences and (2) the generation and reduction of the space of generalization versions. We introduce two approaches we have designed to select the best sentences from the space of generalization versions. Using four NLTK 1 1 The Natural Language Tool Kit ( http://nltk.org/ ) corpora, the first approach estimates the “acceptability” of a given generalization version. The second approach is Machine Learning-based and uses contextual and specific features. The recall, precision and F1-score measures resulting from the evaluation of the concept generalization and fusion approach are presented. [ABSTRACT FROM AUTHOR]

• Automatic and accurate evaluation of summary is a key problem of text summarization research. • A framework for evaluating summary from fidelity, conciseness, and coherence. • An evaluation method based on the semantic link network representation of text. • The semantic link network plays an important role in representing and analyzing texts. Automatic evaluation of summary is one of the basic problems of automatic text summarization research. The commonly used approaches evaluate the informativeness, conciseness and coherence of summary separately based on different representations with weak semantics such as n-gram and vector of words. By transforming text into semantic link network, this paper proposes a general framework for automatically evaluating summary from the following aspects: (1) fidelity , the extent of a summary that conveys the core of its source text; (2) conciseness , the extent of non-redundancy within summary and, (3) coherence , the extent of relatedness between representations within summary, which are based on a uniform semantic representation of text. Experiments on the evaluation of the summarization of scientific papers and news show that the proposed framework achieves comparable or higher correlations with human judgments than the popular evaluation models. This research also verifies the role of the semantic link network in representing and analysing texts. [ABSTRACT FROM AUTHOR]

In this paper, we propose Ranksum, an approach for extractive text summarization of single documents based on the rank fusion of four multi-dimensional sentence features extracted for each sentence: topic information, semantic content, significant keywords, and position. The Ranksum obtains the sentence saliency rankings corresponding to each feature in an unsupervised way followed by the weighted fusion of the four scores to rank the sentences according to their significance. The scores are generated in completely unsupervised way, and a labeled document set is required to learn the fusion weights. Since we found that the fusion weights can generalize to other datasets, we consider the Ranksum as an unsupervised approach. To determine topic rank, we employ probabilistic topic models whereas semantic information is captured using sentence embeddings. To derive rankings using sentence embeddings, we utilize Siamese networks to produce abstractive sentence representation and then we formulate a novel strategy to arrange them in their order of importance. A graph-based strategy is applied to find the significant keywords and related sentence rankings in the document. We also formulate a sentence novelty measure based on bigrams, trigrams, and sentence embeddings to eliminate redundant sentences from the summary. The ranks of all the sentences – computed for each feature – are finally fused to get the final score for each sentence in the document. We evaluate our approach on publicly available summarization datasets — CNN/DailyMail and DUC 2002. Experimental results show that our approach outperforms other existing state-of-the-art summarization methods. [Display omitted] • A unified summarization framework with multi-dimensional sentence features. • Novel method to rank sentences using probabilistic topic vectors. • Sentences ranked with a new technique exploiting embeddings. • Novelty parameter utilizing bigrams, trigrams and sentence embeddings. • Rank-based fusion strategy for extractive summarization. [ABSTRACT FROM AUTHOR]

Automatic text summarization is a topic of great interest in many fields of knowledge. Particularly, query-oriented extractive multi-document text summarization methods have increased their importance recently, since they can automatically generate a summary according to a query given by the user. One way to address this problem is by multi-objective optimization approaches. In this paper, a memetic algorithm, specifically a Multi-Objective Shuffled Frog-Leaping Algorithm (MOSFLA) has been developed, implemented, and applied to solve the query-oriented extractive multi-document text summarization problem. Experiments have been conducted with datasets from Text Analysis Conference (TAC), and the obtained results have been evaluated with Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics. The results have shown that the proposed approach has achieved important improvements with respect to the works of scientific literature. Specifically, 25.41%, 7.13%, and 30.22% of percentage improvements in ROUGE-1, ROUGE-2, and ROUGE-SU4 scores have been respectively reached. In addition, MOSFLA has been applied to medicine texts from the Topically Diverse Query Focus Summarization (TD-QFS) dataset as a case study. • The query-oriented text summarization problem is addressed. • The criteria of query relevance and redundancy reduction are optimized simultaneously. • A multi-objective memetic algorithm (MOSFLA) is designed and developed. • The experiments use the Text Analysis Conference datasets and the ROUGE metrics. • MOSFLA outperforms the results of other 12 approaches from the scientific literature. [ABSTRACT FROM AUTHOR]

Concept maps are graphical representations of knowledge that show important concepts as nodes and the relationships between the concepts as links. An automatically generated concept map, depicting the overview of a specific knowledge domain or documents, can facilitate the learner's understanding of the content. Constructing a good concept map from the given text requires correct selection of concept words and identification of meaningful relationships between the concepts. We propose a new method for generating the concept map from a single document on the basis of bursts of words and the temporal relationships between the bursts. Its strength is examined in comparison with the widely used co-occurrence-based approach. The proposed method differs in that it detects where a word becomes highly topical and the places where two words share such intense periods in the discourse indicating that they form a topic together. This is more advantageous for grasping the flow of a story than the co-occurrence method that simply relates frequently accompanying words throughout the text. Also the method is less affected by high frequency words so that more meaningful relationships can be included in the top links that are limitedly accommodated in a map. For sections in a document, the burst analysis is separately performed for the entire document and for each section not to miss the background knowledge or the foreground relationships that are specific to the part. When evaluated against the human-created concept map, the proposed method shows comparable or better performance than the co-occurrence method in the recall measure and more robustness to frequent unimportant words. [ABSTRACT FROM AUTHOR]

The Web 2.0 has resulted in a shift as to how users consume and interact with the information, and has introduced a wide range of new textual genres, such as reviews or microblogs, through which users communicate, exchange, and share opinions. The exploitation of all this user-generated content is of great value both for users and companies, in order to assist them in their decision-making processes. Given this context, the analysis and development of automatic methods that can help manage online information in a quicker manner are needed. Therefore, this article proposes and evaluates a novel concept-level approach for ultra-concise opinion abstractive summarization. Our approach is characterized by the integration of syntactic sentence simplification, sentence regeneration and internal concept representation into the summarization process, thus being able to generate abstractive summaries, which is one the most challenging issues for this task. In order to be able to analyze different settings for our approach, the use of the sentence regeneration module was made optional, leading to two different versions of the system (one with sentence regeneration and one without). For testing them, a corpus of 400 English texts, gathered from reviews and tweets belonging to two different domains, was used. Although both versions were shown to be reliable methods for generating this type of summaries, the results obtained indicate that the version without sentence regeneration yielded to better results, improving the results of a number of state-of-the-art systems by 9%, whereas the version with sentence regeneration proved to be more robust to noisy data. [ABSTRACT FROM AUTHOR]

People affected by Autism Spectrum Disorders (ASD) have impairments in social interaction because they lack an adequate theory of mind. A significant percentile has inadequate reading comprehension skills. We present a multilingual tool called Open Book (OB) that applies Human Language Technologies (HLT) in order to identify reading comprehension obstacles in text documents and propose more simple alternatives with the aim of assisting the reading comprehension of users. OB involves several text transformations at lexical, syntactic and semantic level. In this paper we focus on three challenging components of the OB tool: the image retrieval component, the idiom detection component and the summarization module. There are very few studies that involve simplification by showing images associated to difficult concepts. In addition, the treatment of figurative language such as idioms or metaphors is one of the most challenging areas in Natural Language Processing (NLP). Finally, although text summarization is a more widely studied field in NLP, its application to text simplification remains as an open research issue. Thus, we focus on the integration of these three modules in our OB tool. We present the motivation for building these components and we describe how they are integrated in the whole system. Moreover, the usability and the usefulness of OB have been evaluated and analysed showing that the tool helps to produce texts easier to understand for autistic people. [ABSTRACT FROM AUTHOR]

Highlights: [•] The task of multi-lingual tweet generation is analyzed. [•] Headlines may not be the most suitable way to communicate information via Twitter. [•] Current tweets delivered by news providers are informative, but not interesting. [•] Automatic tweets were found to be indicative as well as interesting. [•] Overall, COMPENDIUM and the Language-specific summarizer were the best systems. [ABSTRACT FROM AUTHOR]

Abstract: Automatic Text Summarization has been shown to be useful for Natural Language Processing tasks such as Question Answering or Text Classification and other related fields of computer science such as Information Retrieval. Since Geographical Information Retrieval can be considered as an extension of the Information Retrieval field, the generation of summaries could be integrated into these systems by acting as an intermediate stage, with the purpose of reducing the document length. In this manner, the access time for information searching will be improved, while at the same time relevant documents will be also retrieved. Therefore, in this paper we propose the generation of two types of summaries (generic and geographical) applying several compression rates in order to evaluate their effectiveness in the Geographical Information Retrieval task. The evaluation has been carried out using GeoCLEF as evaluation framework and following an Information Retrieval perspective without considering the geo-reranking phase commonly used in these systems. Although single-document summarization has not performed well in general, the slight improvements obtained for some types of the proposed summaries, particularly for those based on geographical information, made us believe that the integration of Text Summarization with Geographical Information Retrieval may be beneficial, and consequently, the experimental set-up developed in this research work serves as a basis for further investigations in this field. [Copyright &y& Elsevier]

Abstract: Concept map is a graphical tool for representing knowledge. They have been used in many different areas, including education, knowledge management, business and intelligence. Constructing of concept maps manually can be a complex task; an unskilled person may encounter difficulties in determining and positioning concepts relevant to the problem area. An application that recommends concept candidates and their position in a concept map can significantly help the user in that situation. This paper gives an overview of different approaches to automatic and semi-automatic creation of concept maps from textual and non-textual sources. The concept map mining process is defined, and one method suitable for the creation of concept maps from unstructured textual sources in highly inflected languages such as the Croatian language is described in detail. Proposed method uses statistical and data mining techniques enriched with linguistic tools. With minor adjustments, that method can also be used for concept map mining from textual sources in other morphologically rich languages. [Copyright &y& Elsevier]

Abstract: In paper, we propose an unsupervised text summarization model which generates a summary by extracting salient sentences in given document(s). In particular, we model text summarization as an integer linear programming problem. One of the advantages of this model is that it can directly discover key sentences in the given document(s) and cover the main content of the original document(s). This model also guarantees that in the summary can not be multiple sentences that convey the same information. The proposed model is quite general and can also be used for single- and multi-document summarization. We implemented our model on multi-document summarization task. Experimental results on DUC2005 and DUC2007 datasets showed that our proposed approach outperforms the baseline systems. [Copyright &y& Elsevier]

Abstract: The rises of the life index quality together with the medical technology improvement lead to a longer life expectancy. Thus a better health care program, especially for elderly, is needed. The common health problems facing those senior citizens are changed from acute diseases to chronic diseases, such as diabetes, hypertension, etc. Along with these changes, medical tourism is becoming the trend of the future. In this paper, we propose a decision support systems, the Ubiquitous Context-aware Healthcare Service System (UCHS), which uses micro sensors integrate RFID to sense user’s life vital signal, such as electrocardiogram (ECG/EKG), heart rate (HR), respiratory rate (RR), blood pressure (BP), blood sugar (BS), and temperature and light. The UCHS is composted of Situation-Aware Medical Tourism Service Search Subsystem (SAMTS3), Healthy-life Map Guiding Subsystem (HMGS), Intelligent Curative Food Decision Support Subsystem (ICFDSS), and 4D Emergency Indication and Ambulance Dispatch Subsystem (4DEIADS) to provide relevant nature medicine recommendations to its user. The UCHS built upon an integrated service platform in which medical experts’ knowledge and all position and negative influence of the proposed therapy are inferred by using semantic network. [ABSTRACT FROM AUTHOR]

Abstract: Product reviews possess critical information regarding customers’ concerns and their experience with the product. Such information is considered essential to firms’ business intelligence which can be utilized for the purpose of conceptual design, personalization, product recommendation, better customer understanding, and finally attract more loyal customers. Previous studies of deriving useful information from customer reviews focused mainly on numerical and categorical data. Textual data have been somewhat ignored although they are deemed valuable. Existing methods of opinion mining in processing customer reviews concentrates on counting positive and negative comments of review writers, which is not enough to cover all important topics and concerns across different review articles. Instead, we propose an automatic summarization approach based on the analysis of review articles’ internal topic structure to assemble customer concerns. Different from the existing summarization approaches centered on sentence ranking and clustering, our approach discovers and extracts salient topics from a set of online reviews and further ranks these topics. The final summary is then generated based on the ranked topics. The experimental study and evaluation show that the proposed approach outperforms the peer approaches, i.e. opinion mining and clustering-summarization, in terms of users’ responsiveness and its ability to discover the most important topics. [Copyright &y& Elsevier]

• A comprehensive introduction of rhetorical structure theory (RST). • A snapshot and some statistical data of RST-DT. • An elaboration of the classical and latest RST parsing methods. • Applications of RST, especially in expert and intelligent systems. • Comparisons with other discourse structure theories. Rhetorical structure theory (RST) is a significant theory about discourse organization. With an increasing number of research interests focus on RST, many novel parsing approaches have been proposed and motivated many brand new applications, such as chatbots and other expert and intelligent systems. However, the work on RST dates back many years and there remains a lack of comprehensive literature reviews. The aim of this study is therefore to provide a comprehensive overview of RST, parsing methods and applications. In this paper, we first give a detailed introduction to RST. Then the commonly used discourse treebank: RST-DT is elaborated. We propose a new taxonomy to divide the RST parsing methods into different categories. With a focus on the classical and latest methods that have recently been developed, we review the pros and cons of these approaches, along with the performance analysis of them. We then summarize the applications of RST across various domains. Moreover, we present a comparative study of RST with other discourse structure theories. Finally, we discuss some implications of our findings and outline future research directions in this challenging and fast-growing field. [ABSTRACT FROM AUTHOR]

• Paper proposes using semantics as feature for text summarization. • Results prove that using semantics as feature improves system performance. • Evaluation on standard dataset shows competitive superiority. Automatic text summarization essentially condenses a long document into a shorter format while preserving its information content and overall meaning. It is a potential solution to the information overload. Several automatic summarizers exist in the literature capable of producing high-quality summaries, but they do not focus on preserving the underlying meaning and semantics of the text. In this paper, we capture and preserve the semantics of text as the fundamental feature for summarizing a document. We propose an automatic summarizer using the distributional semantic model to capture semantics for producing high-quality summaries. We evaluated our summarizer using ROUGE on DUC-2007 dataset and compare our results with other four different state-of-the-art summarizers. Our system outperforms the other reference summarizers leading us to the conclusion that usage of semantic as a feature for text summarization provides improved results and helps to further reduce redundancies from the input source. [ABSTRACT FROM AUTHOR]