Search

Automatic text summarisation is obtaining a subset that accurately represents the main text. A quality summary should contain the maximum amount of information while avoiding redundant information. Redundancy is a severe deficiency that causes unnecessary repetition of information within sentences and should not occur in summarisation studies. Although many optimisation-based text summarisation methods have been proposed in recent years, there exists a lack of research on the simultaneous optimisation of scope and redundancy. In this context, this study presents an approach in which maximum coverage and minimum redundancy, which form the two key features of a rich summary, are modelled as optimisation targets. In optimisation-based text summarisation studies, different conflicting objectives are generally weighted or formulated and transformed into single-objective problems. However, this transformation can directly affect the quality of the solution. In this study, the optimisation goals are met simultaneously without transformation or formulation. In addition, the multi-objective saplings growing-up algorithm (MO-SGuA) is implemented and modified for text summarisation. The presented approach, called Pareto optimal, achieves an optimal solution with simultaneous optimisation. Experimentation with the MO-SGuA method was tested using open-access (document understanding conference; DUC) data sets. Performance success of the MO-SGuA approach was calculated using the recall-oriented understudy for gisting evaluation (ROUGE) metrics and then compared with the competitive practices used in the literature. Testing achieved a 26.6% summarisation result for the ROUGE-2 metric and 65.96% for ROUGE-L, which represents an improvement of 11.17% and 20.54%, respectively. The experimental results showed that good-quality summaries were achieved using the proposed approach. [ABSTRACT FROM AUTHOR]

Automatic Text Summarization (ATS) is gaining popularity as there is a growing demand for a system capable of processing extensive textual content and delivering a concise, yet meaningful, relevant, and useful summary. Manual summarization is both expensive and time-consuming, making it impractical for humans to handle vast amounts of data. Consequently, the need for ATS systems has become evident. These systems encounter challenges such as ensuring comprehensive content coverage, determining the appropriate length of the summary, addressing redundancy, and maintaining coherence in the generated summary. Researchers are actively addressing these challenges by employing Natural Language Processing (NLP) techniques. While traditional methods exist for generating summaries, they often fall short of addressing multiple aspects simultaneously. To overcome this limitation, recent advancements have introduced multi-objective evolutionary algorithms for ATS. This study proposes an enhancement to the performance of ATS through the utilization of an improved version of the Binary Multi-Objective Grey Wolf Optimizer (BMOGWO), incorporating mutation. The performance of this enhanced algorithm is assessed by comparing it with state-of-the-art algorithms using the DUC2002 dataset. Experimental results demonstrate that the proposed algorithm significantly outperforms the compared approaches. [ABSTRACT FROM AUTHOR]

This study presents a pioneering approach that leverages advanced sensing technologies and data processing techniques to enhance the process of clinical documentation generation during medical consultations. By employing sophisticated sensors to capture and interpret various cues such as speech patterns, intonations, or pauses, the system aims to accurately perceive and understand patient–doctor interactions in real time. This sensing capability allows for the automation of transcription and summarization tasks, facilitating the creation of concise and informative clinical documents. Through the integration of automatic speech recognition sensors, spoken dialogue is seamlessly converted into text, enabling efficient data capture. Additionally, deep models such as Transformer models are utilized to extract and analyze crucial information from the dialogue, ensuring that the generated summaries encapsulate the essence of the consultations accurately. Despite encountering challenges during development, experimentation with these sensing technologies has yielded promising results. The system achieved a maximum ROUGE-1 metric score of 0.57, demonstrating its effectiveness in summarizing complex medical discussions. This sensor-based approach aims to alleviate the administrative burden on healthcare professionals by automating documentation tasks and safeguarding important patient information. Ultimately, by enhancing the efficiency and reliability of clinical documentation, this innovative method contributes to improving overall healthcare outcomes. [ABSTRACT FROM AUTHOR]

The explosive growth of dialogue data has aroused significant interest among scholars in abstractive dialogue summarization. In this paper, we propose a novel sequence-to-sequence framework called DS-SS (Dialogue Summarization with Factual-Statement Fusion and Dialogue Segmentation) for summarizing dialogues. The novelty of the DS-SS framework mainly lies in two aspects: 1) Factual statements are extracted from the source dialogue and combined with the source dialogue to perform the further dialogue encoding; and 2) A dialogue segmenter is trained and used to separate a dialogue to be encoded into several topic-coherent segments. Thanks to these two aspects, the proposed framework may better encode dialogues, thereby generating summaries exhibiting higher factual consistency and informativeness. Experimental results on two large-scale datasets SAMSum and DialogSum demonstrate the superiority of our framework over strong baselines, as evidenced by both automatic evaluation metrics and human evaluation. [ABSTRACT FROM AUTHOR]

The automatic generation of natural language is a complex and essential task in text processing. This study proposes a novel approach to address this fundamental problem by leveraging an improved version of DST_BERT, a model that converts input text into a vector representation. Our key contribution lies in the joint optimization of two models, NLU (Natural Language Under-standing) and NLG (Natural Language Generation), which enables us to obtain variable representations within a hidden space. This integration enhances the capabilities of both NLU and NLG in generating coherent and contextually appropriate language. The NLU and NLG models are seamlessly integrated with the hidden variable space, forming a generative representation model. To assess the effectiveness of our proposed approach, we conducted extensive experiments on the E2E and Weather datasets. The results highlight the state-of-the-art performance achieved by our model in generating natural language. [ABSTRACT FROM AUTHOR]

In recent years, deep learning has revolutionized natural language processing (NLP) by enabling the development of models that can learn complex representations of language data, leading to significant improvements in performance across a wide range of NLP tasks. Deep learning models for NLP typically use large amounts of data to train deep neural networks, allowing them to learn the patterns and relationships in language data. This is in contrast to traditional NLP approaches, which rely on hand-engineered features and rules to perform NLP tasks. The ability of deep neural networks to learn hierarchical representations of language data, handle variable-length input sequences, and perform well on large datasets makes them well-suited for NLP applications. Driven by the exponential growth of textual data and the increasing demand for condensed, coherent, and informative summaries, text summarization has been a critical research area in the field of NLP. Applying deep learning to text summarization refers to the use of deep neural networks to perform text summarization tasks. In this survey, we begin with a review of fashionable text summarization tasks in recent years, including extractive, abstractive, multi-document, and so on. Next, we discuss most deep learning-based models and their experimental results on these tasks. The paper also covers datasets and data representation for summarization tasks. Finally, we delve into the opportunities and challenges associated with summarization tasks and their corresponding methodologies, aiming to inspire future research efforts to advance the field further. A goal of our survey is to explain how these methods differ in their requirements as understanding them is essential for choosing a technique suited for a specific setting. This survey aims to provide a comprehensive review of existing techniques, evaluation methodologies, and practical applications of automatic text summarization. [ABSTRACT FROM AUTHOR]

Purpose: The conclusion section of a radiology report is crucial for summarizing the primary radiological findings in natural language and essential for communicating results to clinicians. However, creating these summaries is time-consuming, repetitive, and prone to variability and errors among different radiologists. To address these issues, we evaluated a fine-tuned Text-To-Text Transfer Transformer (T5) model for abstractive summarization to automatically generate conclusions for neuroradiology MRI reports in a low-resource language. Methods: We retrospectively applied our method to a dataset of 232,425 neuroradiology MRI reports in Spanish. We compared various pre-trained T5 models, including multilingual T5 and those newly adapted for Spanish. For precise evaluation, we employed BLEU, METEOR, ROUGE-L, CIDEr, and cosine similarity metrics alongside expert radiologist assessments. Results: The findings are promising, with the models specifically fine-tuned for neuroradiology MRI achieving scores of 0.46, 0.28, 0.52, 2.45, and 0.87 in the BLEU-1, METEOR, ROUGE-L, CIDEr, and cosine similarity metrics, respectively. In the radiological experts' evaluation, they found that in 75% of the cases evaluated, the conclusions generated by the system were as good as or even better than the manually generated conclusions. Conclusion: The methods demonstrate the potential and effectiveness of customizing state-of-the-art pre-trained models for neuroradiology, yielding automatic MRI report conclusions that nearly match expert quality. Furthermore, these results underscore the importance of designing and pre-training a dedicated language model for radiology report summarization. [ABSTRACT FROM AUTHOR]

Document summarization is the task of generating a shorter form of document with important information content. Automatic text summarization has been developed for this process and is still widely used. It is divided into two main parts as extractive summarization and abstractive summarization. In this study, we used sentence ranking methods for extractive summarization for Turkish news text within the scope of the experimental study. We used different summarization rates, 20%, 30%, 40%, 50% and 60%. Summarization results were evaluated with the ROUGE ve BLEU metrics. We proposed new methods based on major vowel harmony and minor vowel harmony features. We obtained high evaluation results in both ROUGE ve BLEU metrics with major vowel harmony and minor vowel harmony features. Additionally, we studied a hybrid model using major vowel harmony and minor vowel harmony rules together. We obtained the best results with major vowel harmony, minor vowel harmony, and hybrid model (major vowel harmony and minor vowel harmony together). We compared the three proposed methods with the BERTurk model prepared for Turkish based on Google BERT. The results obtained gave very close results to this state-of-the-art method and showed that it is worth developing. [ABSTRACT FROM AUTHOR]

Due to the enormous content of text available online through emails, social media, and news articles, it has become complicated to summarize the textual information from multiple documents. Text summarization automatically creates a comprehensive description of the document that retains its informative contents through the keywords, where Multi-Document Summarization (MDS) is a productive tool for data accumulation that creates a concise and informative summary from the documents. In order to extract the relevant information from the documents, Graph neural networks (GNNs) is the neural structure that detains the interrelation of the graph by progressing the messages between the graphical nodes. In the current years, the advanced version of GNNs, such as graph attention network (GAN), graph recurrent network, and graph convolutional network (GCN) provides a remarkable performance in text summarization with the advantage of deep learning techniques. Hence, in this survey, graph approaches for text summarization has been analyzed and discussed, where the recent text summarization model based on Deep learning techniques are highlighted. Further, the article provides the taxonomy to abstract the design pattern of Neural Networks and conducts a comprehensive of the existing text summarization model. Finally, the review article enlists the future direction of the researcher, which would motivate the enthusiastic and novel contributions in text summarizations. [ABSTRACT FROM AUTHOR]

In this digital era, there is a tremendous increase in the volume of data, which adds difficulties to the person who utilizes particular applications, such as websites, email, and news. Text summarization targets to reduce the complexity of obtaining statistics from the websites as it compresses the textual document to a short summary without affecting the relevant information. The crucial step in multi-document summarization is obtaining a relationship between the cross-sentence. However, the conventional methods fail to determine the inter-sentence relationship, especially in long documents. This research develops a graph-based neural network to attain an inter-sentence relationship. The significant step in the proposed multi-document text summarization model is forming the weighted graph embedding features. Furthermore, the weighted graph embedding features are utilized to evaluate the relationship between the document's words and sentences. Finally, the bidirectional long short-term memory (BiLSTM) classifier is utilized to summarize the multi-document text summarization. The experimental analysis uses the three standard datasets, the Daily Mail dataset, Document Understanding Conference (DUC) 2002, and Document Understanding Conference (DUC) 2004 dataset. The experimental outcome demonstrates that the proposed weighted graph embedding feature + BiLSTM model exceeds all the conventional methods with Precision, Recall, and F1 score of 0.5352, 0.6296, and 0.5429, respectively. [ABSTRACT FROM AUTHOR]

Text summarizing (TS) takes key information from a source text and condenses it for the user while retaining the primary material. When it comes to text summaries, the most difficult problem is to provide broad topic coverage and diversity in a single summary. Overall, text summarization addresses the fundamental need to distill large volumes of information into more manageable and digestible forms, making it a crucial technology in the era of information abundance. It benefits individuals, businesses, researchers, and various other stakeholders by enhancing efficiency and comprehension in dealing with textual data. In this paper, proposed a novel Modified Generative adversarial network (MGAN) for summarize the text. The proposed model involves three stages namely pre-processing, Extractive summarization, and summary generation. In the first Phase, the Text similarity dataset is pre-processed using Lowering Casing, Tokenization, Lemmatization, and, Stop Word Removal. In the second Phase, the Extractive summarization is done in three steps Generating similarity metrics, Sentence Ranking, and Sentence Extractive. In the third stage, a generative adversarial network (GAN) employs summary generation to jointly train the discriminative model D and the generative model G. To classify texts and annotate their syntax, Generative Model G employs a convolutional neural network called Bidirectional Gated Recursive Unit (CNN-BiGRU). The performance analysis of the proposed MGAN is calculated based on the parameters like accuracy, specificity, Recall, and Precision metrics. The proposed MGAN achieves an accuracy range of 99%. The result shows that the proposed MGAN improves the overall accuracy better than 9%, 6.5% and 5.4% is DRM, LSTM, and CNN respectively. [ABSTRACT FROM AUTHOR]

Automatic text summarization is the task of creating concise and fluent summaries without human intervention while preserving the meaning of the original text document. To increase the readability of the languages, a summary should be generated. In this paper, a novel Nesterov-accelerated Adaptive Moment Estimation Optimization based on Long Short-Term Memory [NADAM-LSTM] has been proposed to summarize the text. The proposed NADAM-LSTM model involves three stages namely pre-processing, summary generation, and parameter tuning. Initially, the Giga word Corpus dataset is pre-processed using Tokenization, Word Removal, Stemming, Lemmatization, and Normalization for removing irrelevant data. In the summary generation phase, the text is converted to the word-to-vector method. Further, the text is fed to LSTM to summarize the text. The parameter of the LSTM is then tuned using NADAM Optimization. The performance analysis of the proposed NADAM-LSTM is calculated based on parameters like accuracy, specificity, Recall, Precision, and F1 score. The suggested NADAM-LSTM achieves an accuracy range of 99.5%. The result illustrates that the proposed NADAM-LSTM enhances the overall accuracy better than 12%, 2.5%, and 1.5% in BERT, CNN-LSTM, and RNN respectively. [ABSTRACT FROM AUTHOR]

Two of the most crucial issues for text summarization to generate faithful summaries are to make use of knowledge beyond text and to make use of cross-sentence relations in text. Intuitive ways for the two issues are knowledge graph (KG) and graph neural network (GNN), respectively. Entities are semantic units in text and in KG. This paper focuses on both issues by leveraging entities mentioned in text to connect GNN and KG for summarization. Firstly, entities are leveraged to construct a sentence-entity graph with weighted multi-type edges to model sentence relations, and a relational heterogeneous GNN for summarization is proposed to calculate node encodings. Secondly, entities are leveraged to link the graph to KG to collect knowledge. Thirdly, entities guide a two-step summarization framework defining a multitask selector to select salient sentences and entities, and using an entity-focused abstractor to compress the sentences. GNN is connected with KG by constructing sentence-entity graphs where entity–entity edges are built based on KG, initializing entity embeddings on KG, and training entity embeddings using entity–entity edges. The relational heterogeneous GNN utilizes both edge weights and edge types in GNN to calculate graphs with weighted multi-type edges. Experiments show the proposed method outperforms extractive baselines including the HGNN-based HGNNSum and abstractive baselines including the entity-driven SENECA on CNN/DM, and outperforms most baselines on NYT50. Experiments on sub-datasets show the density of sentence-entity edges greatly influences the performance of the proposed method. The greater the density, the better the performance. Ablations show effectiveness of the method. [ABSTRACT FROM AUTHOR]

Pretrained models in abstractive summarization enable the summarization model to generate coherent summaries flexibly, but it also introduces unfactual hallucinations when generating novel words. To circumvent this drawback, we propose A Guided Tensor Product framework for abstractive Summarization (GTPSum). GTPSum utilizes highlighted sentence guidance signal and role embedding in the form of guided tensor product representations (GTPRs) to enhance the generation capacity for short document text. The guidance signal assists the model in focusing on crucial information, while the role embedding aids GTPSum in capturing the syntactic structure relationships present in the sentences. By employing this approach, the framework can not only focus on the essential information from the source document but also grasp the inherent sentence structure, resulting in generated summaries that closely align with the original content. The experimental results demonstrate the strong performance of our approach on the CNN/Daily Mail (44.28/20.88/41.08 ROUGE-1/-2/-L) and performed well on the other three datasets. Moreover, the comparison between our model generated summaries and those generated by the baseline models through human evaluation shows that our summaries contain more critical information, have better readability, and still maintain succinctness. [ABSTRACT FROM AUTHOR]

The biomedical field specialists often face difficulties in summarization of available online literature. The biomedical archives like PubMed contain large set of lengthy documents, which needs to be shortened for further analysis of relevant data. In this work, we have introduced Similarity-specific Teaching and Learning-based optimization (Sim-TLBO) framework, which is combined with controlled stochastic selection of most relevant sentences for effective summarization. This method of utilizing metaheuristic teaching–learning approach along with multiple objective functions analysis provided an agile and robust solution. The output summary from the proposed solution is evaluated and compared based on ROUGE performance and various objective parameters including cohesion, non-redundancy, coverage, and readability. The hybrid text similarity (HTS) based on BERT similarity technique resulted in optimum ROUGE-1 values of 0.82 and 0.79 at 50% and 30% compression rates, respectively. The results from the presented framework when compared to other benchmark and baseline summarization models demonstrated significant performance improvement. [ABSTRACT FROM AUTHOR]

This paper presents a query-based extractive text summarization approach by using sense-oriented semantic relatedness measure. To find the query relevant sentences, we have to find semantic relatedness measure between query and input text sentences. To find the relatedness score, we need to know the exact sense of the words present in query and input text sentences. Word sense disambiguation (WSD) finds the actual meaning of a word according to its context of the sentence. We have proposed a WSD technique to extract query relevant sentences which is used to find a sense-oriented sentence semantic relatedness score between the query and input text sentence. Here, a feature-based method is presented to find semantic relatedness score between query and input text sentence. Finally the proposed query-based text summary method uses relevant and redundancy-free features to form cluster. There is a high probability that same featured cluster may contain redundant sentences. Therefore, a redundancy removal method is proposed to get redundancy-free sentences. In the end, redundancy-free query relevant sentences are obtained with an information rich summary. We have evaluated our proposed WSD technique with other existing methods by using Senseval and SemEval datasets and proposed Sense-Oriented Sentence Semantic Relatedness Score by using Li et al. dataset. We compare our proposed query-based extractive text summarization method with other methods participated in Document Understanding Conference and as well as with current methods. Evaluation and comparison state that the proposed query-based extractive text summarization method outperforms many existing and recent methods. [ABSTRACT FROM AUTHOR]

Fine-tuning a pre-trained sequence-to-sequence-based language model has significantly advanced the field of abstractive summarization. However, the early models of abstractive summarization were limited by the gap between training and inference, and they did not fully utilize the potential of the language model. Recent studies have introduced a two-stage framework that allows the second-stage model to re-rank the candidate summary generated by the first-stage model, to resolve these limitations. In this study, we point out that the supervision method performed in the existing re-ranking model of the two-stage abstractive summarization framework cannot learn detailed and complex information of the data. In addition, we present the problem of positional bias in the existing encoder–decoder-based re-ranking model. To address these two limitations, this study proposes a hierarchical supervision method that jointly performs summary and sentence-level supervision. For sentence-level supervision, we designed two sentence-level loss functions: intra- and inter-intra-sentence ranking losses. Compared to the existing abstractive summarization model, the proposed method exhibited a performance improvement for both the CNN/DM and XSum datasets. The proposed model outperformed the baseline model under a few-shot setting. [ABSTRACT FROM AUTHOR]

Text summarization (TS) plays a crucial role in natural language processing (NLP) by automatically condensing and capturing key information from text documents. Its significance extends to diverse fields, including engineering, healthcare, and others, where it offers substantial time and resource savings. However, manual summarization is a laborious task, prompting the need for automated text summarization systems. In this paper, we propose a novel strategy for extractive summarization that leverages a generative adversarial network (GAN)-based method and Bidirectional Encoder Representations from Transformers (BERT) word embedding. BERT, a transformer-based architecture, processes sentence bidirectionally, considering both preceding and following words. This contextual understanding empowers BERT to generate word representations that carry a deeper meaning and accurately reflect their usage within specific contexts. Our method adopts a generator and discriminator within the GAN framework. The generator assesses the likelihood of each sentence in the summary while the discriminator evaluates the generated summary. To extract meaningful features in parallel, we introduce three dilated convolution layers in the generator and discriminator. Dilated convolution allows for capturing a larger context and incorporating long-range dependencies. By introducing gaps between filter weights, dilated convolution expands the receptive field, enabling the model to consider a broader context of words. To encourage the generator to explore diverse sentence combinations that lead to high-quality summaries, we introduce various noises to each document within our proposed GAN. This approach allows the generator to learn from a range of sentence permutations and select the most suitable ones. We evaluate the performance of our proposed model using the CNN/Daily Mail dataset. The results, measured using the ROUGE metric, demonstrate the superiority of our approach compared to other tested methods. This confirms the effectiveness of our GAN-based strategy, which integrates dilated convolution layers, BERT word embedding, and a generator-discriminator framework in achieving enhanced extractive summarization performance. [ABSTRACT FROM AUTHOR]

Recently, large pretrained language models (PLMs) have led a revolution in the information retrieval community. In most PLMs‐based retrieval frameworks, the ranking performance broadly depends on the model structure and the semantic complexity of the input text. Sequence‐to‐sequence generative models for question answering or text generation have proven to be competitive, so we wonder whether these models can improve ranking effectiveness by enhancing input semantics. This article introduces SE‐BERT, a semantically enhanced bidirectional encoder representation from transformers (BERT) based ranking framework that captures more semantic information by modifying the input text. SE‐BERT utilizes a pretrained generative language model to summarize both sides of the candidate passage and concatenate them into a new input sequence, allowing BERT to acquire more semantic information within the constraints of the input sequence's length. Experimental results from two Text Retrieval Conference datasets demonstrate that our approach's effectiveness increasing as the length of the input text increases. [ABSTRACT FROM AUTHOR]

The purpose of government news summarization is to extract the most important information from official government news reports. It is important for readers to be able to understand government news quickly in the age of information overload. Compared with other types of news, government news reports have more detailed content and possess a more normative format, resulting in greater length. To resolve the contradiction between the length of these reports and the trend of fragmented reading, this research proposes an automatic text summarization model based on multiple features. First, the features are extracted using the TF–IDF algorithm and word vector embedding method based on the bidirectional encoder representation from the transformers model. Second, we score the sentences based on the position, keywords, and similarity features. Finally, the top-ranked sentence is selected to form the summarization. To verify the effectiveness and superiority of the proposed method, Edmundson and ROUGE were adopted. First, based on the Edmundson evaluation criteria, the summarization results of various methods were scored. The score differences between ATS summarization based on the proposed method and manual summarization were minimal across consistency, grammaticality, time sequence, conciseness, and readability, with values of 0.14, 0.18, 0.12, 0.10, and 0.16, respectively. This suggests that our summarization exhibits the highest similarity to manual summarization. Second, we evaluated the summarization results using the ROUGE evaluation criteria. The results indicate that the proposed method achieved significantly higher scores compared to other models. Specifically, for character-level ROUGE-1, the P, R, and F scores reached 0.84, 0.93, and 0.88, respectively. At the word-level ROUGE-1, the P, R, and F scores were 0.81, 0.89, and 0.85, respectively, which demonstrates a noticeable improvement over other models. Furthermore, compared to manual methods, the proposed method has advantages in assisting the reader to obtain important information rapidly. [ABSTRACT FROM AUTHOR]

ChatGPT has drawn considerable attention from both the general public and domain experts with its remarkable text generation capabilities. This has subsequently led to the emergence of diverse applications in the field of biomedicine and health. In this work, we examine the diverse applications of large language models (LLMs), such as ChatGPT, in biomedicine and health. Specifically, we explore the areas of biomedical information retrieval, question answering, medical text summarization, information extraction and medical education and investigate whether LLMs possess the transformative power to revolutionize these tasks or whether the distinct complexities of biomedical domain presents unique challenges. Following an extensive literature survey, we find that significant advances have been made in the field of text generation tasks, surpassing the previous state-of-the-art methods. For other applications, the advances have been modest. Overall, LLMs have not yet revolutionized biomedicine, but recent rapid progress indicates that such methods hold great potential to provide valuable means for accelerating discovery and improving health. We also find that the use of LLMs, like ChatGPT, in the fields of biomedicine and health entails various risks and challenges, including fabricated information in its generated responses, as well as legal and privacy concerns associated with sensitive patient data. We believe this survey can provide a comprehensive and timely overview to biomedical researchers and healthcare practitioners on the opportunities and challenges associated with using ChatGPT and other LLMs for transforming biomedicine and health. [ABSTRACT FROM AUTHOR]

Large language models (LLMs) have substantially pushed artificial intelligence (AI) research and applications in the last few years. They are currently able to achieve high effectiveness in different natural language processing (NLP) tasks, such as machine translation, named entity recognition, text classification, question answering, or text summarization. Recently, significant attention has been drawn to OpenAI's GPT models' capabilities and extremely accessible interface. LLMs are nowadays routinely used and studied for downstream tasks and specific applications with great success, pushing forward the state of the art in almost all of them. However, they also exhibit impressive inference capabilities when used off the shelf without further training. In this paper, we aim to study the behavior of pre‐trained language models (PLMs) in some inference tasks they were not initially trained for. Therefore, we focus our attention on very recent research works related to the inference capabilities of PLMs in some selected tasks such as factual probing and common‐sense reasoning. We highlight relevant achievements made by these models, as well as some of their current limitations that open opportunities for further research. This article is categorized under:Fundamental Concepts of Data and Knowledge > Key Design Issues in Data MiningTechnologies > Artificial Intelligence [ABSTRACT FROM AUTHOR]

Query‐focused multi‐document summarization (Qf‐MDS) is a sub‐task of automatic text summarization that aims to extract a substitute summary from a document cluster of the same topic and based on a user query. Unlike other summarization tasks, Qf‐MDS has specific research challenges including the differences and similarities across related document sets, the high degree of redundancy inherent in the summaries created from multiple related sources, relevance to the given query, topic diversity in the produced summary and the small source‐to‐summary compression ratio. In this work, we propose a semantic diversity feature based query‐focused extractive summarizer (SDbQfSum) built on powerful text semantic representation techniques underpinned with Wikipedia commonsense knowledge in order to address the query‐relevance, centrality, redundancy and diversity challenges. Specifically, a semantically parsed document text is combined with knowledge‐based vectorial representation to extract effective sentence importance and query‐relevance features. The proposed monolingual summarizer is evaluated on a standard English dataset for automatic query‐focused summarization tasks, that is, the DUC2006 dataset. The obtained results show that our summarizer outperforms most state‐of‐the‐art related approaches on one or more ROUGE measures achieving 0.418, 0.092 and 0.152 in ROUGE‐1, ROUGE‐2, and ROUGE‐SU4 respectively. It also attains competitive performance with the slightly outperforming system(s), for example, the difference between our system's result and best system in ROUGE‐1 is just 0.006. We also found through the conducted experiments that our proposed custom cluster merging algorithm significantly reduces information redundancy while maintaining topic diversity across documents. [ABSTRACT FROM AUTHOR]

Representation transfer is a widely used technique in natural language processing. We propose methods of cleaning the dominant dataset of text simplification (TS) WikiLarge in multi-views to remove errors that impact model training and fine-tuning. The results show that our method can effectively refine the dataset. We propose to take the pre-trained text representations from a similar task (e.g., text summarization) to text simplification to conduct a continue-fine-tuning strategy to improve the performance of pre-trained models on TS. This approach will speed up the training and make the model convergence easier. Besides, we also propose a new decoding strategy for simple text generation. It is able to generate simpler and more comprehensible text with controllable lexical simplicity. The experimental results show that our method can achieve good performance on many evaluation metrics. • Using sentence representations to refine the dataset. • Fine-tuning by taking text representations from similar tasks. • A new decoding strategy for simple text generation. [ABSTRACT FROM AUTHOR]

Nowadays, the Internet is a structure that people can access easily and at the same time produce content easily and without control. In parallel with this situation, the ability of extract information from the raw data that makes up big data has become more complex. The fact that the headlines of the contents contain uncontrolled and misleading elements makes it difficult to reach the right information. The headlines of the contents are important for people to reach the information they want in their limited time. In this study, it is aimed to produce headlines suitable for the content instead of headlines that may be misleading for news. For this purpose, an application that produces headlines for Turkish news with deep learning method has been developed. SuDer news corpus is used as dataset. For the training of the model, it is aimed to obtain more humanoid results in the production of news headlines by using the Transformer architecture, which is frequently preferred in natural language studies today and the abstract summarization method. In this study, in order to compare the performance of the Transformer model, models are prepared and trained with Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) architectures. At the end of 25 epochs of training with LSTM, GRU and Transformer architectures on the corpus, the values of loss are 1.03, 0.55 and 2.49 respectively. In the experiments performed on the validation data, measurements are made with ROUGE-1, ROUGE-2 and ROUGE-L metrics. As a result of the measurements, it is observed that the Transformer architecture is partially good, based on the metric values produced. In addition, when the headlines produced with these architectures are examined, it is observed that the headline obtained with the Transformer architecture produce headlines that are partially more suitable for the news content compared to other architectures. [ABSTRACT FROM AUTHOR]

The article titled "Timely need for navigating the potential and downsides of LLMs in healthcare and biomedicine" provides a comprehensive exploration of the opportunities and challenges associated with large language models (LLMs) in the field of biomedicine and healthcare. The authors discuss the transformative potential of LLMs in areas such as biomedical information retrieval, question answering, medical text summarization, information extraction, and medical education. They also highlight the need for caution regarding data privacy, ethical considerations, and the mitigation of biases. The article presents a list of popular LLMs in the health domain and identifies limitations and new challenges, along with mitigation strategies. Additionally, it suggests various applications of LLMs in healthcare, including personalized treatment recommendations, predictive health analytics, and automated medical literature review. The authors emphasize the importance of proceeding with ethical principles, inclusivity, and collaboration in integrating LLMs into healthcare. While the article provides valuable insights, it could have further explored futuristic challenges and applications of LLMs in the biomedical domain. Overall, this work contributes to the ongoing discourse on the role of AI in healthcare and sets the stage for future research and development in this field. [Extracted from the article]

PURPOSE: Manual extraction of case details from patient records for cancer surveillance is a resource-intensive task. Natural Language Processing (NLP) techniques have been proposed for automating the identification of key details in clinical notes. Our goal was to develop NLP application programming interfaces (APIs) for integration into cancer registry data abstraction tools in a computer-assisted abstraction setting. METHODS: We used cancer registry manual abstraction processes to guide the design of DeepPhe-CR, a web-based NLP service API. The coding of key variables was performed through NLP methods validated using established workflows. A container-based implementation of the NLP methods and the supporting infrastructure was developed. Existing registry data abstraction software was modified to include results from DeepPhe-CR. An initial usability study with data registrars provided early validation of the feasibility of the DeepPhe-CR tools. RESULTS: API calls support submission of single documents and summarization of cases across one or more documents. The container-based implementation uses a REST router to handle requests and support a graph database for storing results. NLP modules extract topography, histology, behavior, laterality, and grade at 0.79-1.00 F1 across multiple cancer types (breast, prostate, lung, colorectal, ovary, and pediatric brain) from data of two population-based cancer registries. Usability study participants were able to use the tool effectively and expressed interest in the tool. CONCLUSION: The DeepPhe-CR system provides an architecture for building cancer-specific NLP tools directly into registrar workflows in a computer-assisted abstraction setting. Improved user interactions in client tools may be needed to realize the potential of these approaches. We have developed Natural Language Processing tools for augmenting cancer registry data abstraction. [ABSTRACT FROM AUTHOR]

The rapid development of technology has resulted in a surge in the volume of digital data available. This situation creates a problem for users who need assistance in locating specific information within this massive collection of data, resulting in a time-consuming process. Automatic Text Summarization systems have been developed as a more effective solution than traditional summarization techniques to address this problem and improve user access to relevant information. It is well known that researchers in the health sciences find it difficult to keep up with the latest literature due to their busy schedules. This study aims to produce comprehensive abstracts of Turkish-language scientific papers in the field of health sciences. Although abstracts of scientific papers are already available, more thorough summaries are still needed. To the best of our knowledge, no previous attempt has been made to automatically summarize Turkish language health science papers. For this purpose, a dataset of 105 Turkish papers was collected from DergiPark. Term Frequency, Term Frequency-Inverse Document Frequency, Latent Semantic Analysis, TextRank, and Latent Dirichlet Allocation algorithms were chosen as extractive text summarization methods due to their frequent use in this field. The performance of the text summarization models was evaluated using recall, precision, and F-score metrics, and the algorithms gave satisfactory results for Turkish. [ABSTRACT FROM AUTHOR]

Natural Language Processing (NLP) is the computerized approach to analyzing text using both structured and unstructured data. NLP is a simple, empirically powerful, and reliable approach. It achieves state-of-the-art performance in language processing tasks like Semantic Search (SS), Machine Translation (MT), Text Summarization (TS), Sentiment Analyzer (SA), Named Entity Recognition (NER) and Emotion Detection (ED). NLP is expected to be the technology of the future, based on current technology deployment and adoption. The primary question is: What does NLP have to offer in terms of reality, and what are the prospects? There are several problems to be addressed with this developing method, as it must be compatible with future technology. In this paper, the benefits, challenges and limitations of this innovative paradigm along with the areas open to do research are shown. [ABSTRACT FROM AUTHOR]

The rapid growth of electronic health records (EHRs) has led to unprecedented biomedical data. Clinician access to the latest patient information can improve the quality of healthcare. However, clinicians have difficulty finding information quickly and easily due to the sheer data mining volume. Biomedical information retrieval (BIR) systems can help clinicians find the information required by automatically searching EHRs and returning relevant results. However, traditional BIR systems cannot understand the complex relationships between EHR entities. Transformers are a new type of neural network that is very effective for natural language processing (NLP) tasks. As a result, transformers are well suited for tasks such as machine translation and text summarization. In this paper, we propose a new BIR system for EHRs that uses transformers for predicting cancer treatment from EHR. Our system can understand the complex relationships between the different entities in an EHR, which allows it to return more relevant results to clinicians. We evaluated our system on a dataset of EHRs and found that it outperformed state-of-the-art BIR systems on various tasks, including medical question answering and information extraction. Our results show that Transformers are a promising approach for BIR in EHRs, reaching an accuracy and an F1-score of 86.46%, and 0.8157, respectively. We believe that our system can help clinicians find the information they need more quickly and easily, leading to improved patient care. [ABSTRACT FROM AUTHOR]

Nowadays, individuals can be overwhelmed by a huge number of documents being present in daily life. Capturing the necessary details is often a challenge. Therefore, it is rather important to summarize documents to obtain the main information quickly. There currently exist automatic approaches to this task, but their quality is often not properly assessed. State-of-the-art metrics rely on human-generated summaries as a reference for the evaluation. If no reference is given, the assessment will be challenging. Therefore, in the absence of human-generated reference summaries, we investigated an alternative approach to how machine-generated summaries can be evaluated. For this, we focus on the original text or document to retrieve a metric that allows a direct evaluation of automatically generated summaries. This approach is particularly helpful in cases where it is difficult or costly to find reference summaries. In this paper, we present a novel metric called Summary Score without Reference—SUSWIR—which is based on four factors already known in the text summarization community: Semantic Similarity, Redundancy, Relevance, and Bias Avoidance Analysis, overcoming drawbacks of common metrics. Therefore, we aim to close a gap in the current evaluation environment for machine-generated text summaries. The novel metric is introduced theoretically and tested on five datasets from their respective domains. The conducted experiments yielded noteworthy outcomes, employing the utilization of SUSWIR. [ABSTRACT FROM AUTHOR]

Peer reviews form an essential part of scientific communication. Scholarly peer review is probably the most accepted way to evaluate research papers by involving multiple experts to review the concerned research independently. Usually, the area chair, the program chair, or the editor takes a call weighing the reviewer's judgments. It communicates the decision to the author via writing a meta-review by summarizing the review comments. With the exponential rise in research paper submissions and the corresponding rise in the reviewer pool, it becomes stressful for the chairs/editors to manage conflicts, arrive at a consensus, and also write an informative meta-review. Here in this work, we propose a novel deep neural network-based approach for generating meta-reviews in an unsupervised fashion. To generate consistent meta-reviews, we use a generic template where the task is like to slot-fill the template with the generated meta-review text. We consider the setting where only peer reviews with no summaries or meta-reviews are provided and propose an end-to-end neural network model to perform unsupervised opinion-based abstractive summarization. We first use an aspect-based sentiment analysis model, which classifies the review sentences with the corresponding aspects (e.g., novelty, substance, soundness, etc.) and sentiment. We then extract opinion phrases from reviews for the corresponding aspect and sentiment labels. Next, we train a transformer model to reconstruct the original reviews from these extraction. Finally, we filter the selected opinions according to their aspect and/or sentiment at the time of summarization. The selected opinions of each aspect are used as input to the trained Transformer model, which uses them to construct an opinion summary. The idea is to give a concise meta-review that maximizes information coverage by focusing on aspects and sentiment present in the review, coherence, readability, and redundancy. We evaluate our model on the human written template-based meta-reviews to show that our framework outperforms competitive baselines. We believe that the template-based meta-review generation focusing on aspect and sentiment will help the editor/chair in decision-making and assist the meta-reviewer in writing better and more informative meta-reviews. We make our codes available at https://github.com/sandeep82945/Unsupervised-meta-review-generation. [ABSTRACT FROM AUTHOR]

A discourse containing one or more sentences describes daily issues and events for people to communicate their thoughts and opinions. As sentences are normally consist of multiple textsegments, correct understanding of the theme of a discourse should take into consideration of the relations in between text segments. Although sometimes a connective exists in raw texts for conveying relations, it is more often the cases that no connective exists in between two text segments but some implicit relation does exist in between them. The task of implicit discourse relation recognition (IDRR) is to detect implicit relation and classify its sense between two text segments without a connective. Indeed, the IDRR task is important to diverse downstream natural language processing tasks, such as text summarization, machine translation, and so on. This article provides a comprehensive and up-to-date survey for the IDRR task. We first summarize the task definition and data sources widely used in the field. We categorize the main solution approaches for the IDRR task from the viewpoint of its development history. In each solution category, we present and analyze the most representative methods, including their origins, ideas, strengths, and weaknesses. We also present performance comparisons for those solutions experimented on a public corpus with standard data processing procedures. Finally, we discuss future research directions for discourse relation analysis. [ABSTRACT FROM AUTHOR]

Natural Language Generation (NLG) has improved exponentially in recent years thanks to the development of sequence-to-sequence deep learning technologies such as Transformer-based language models. This advancement has led to more fluent and coherent NLG, leading to improved development in downstream tasks such as abstractive summarization, dialogue generation, and data-to-text generation. However, it is also apparent that deep learning based generation is prone to hallucinate unintended text, which degrades the system performance and fails to meet user expectations in many real-world scenarios. To address this issue, many studies have been presented in measuring and mitigating hallucinated texts, but these have never been reviewed in a comprehensive manner before. In this survey, we thus provide a broad overview of the research progress and challenges in the hallucination problem in NLG. The survey is organized into two parts: (1) a general overview of metrics, mitigation methods, and future directions, and (2) an overview of task-specific research progress on hallucinations in the following downstream tasks, namely abstractive summarization, dialogue generation, generative question answering, data-to-text generation, and machine translation. This survey serves to facilitate collaborative efforts among researchers in tackling the challenge of hallucinated texts in NLG. [ABSTRACT FROM AUTHOR]

Texts used in L2 classrooms have traditionally been simplified manually, but recent technological advances allow us to investigate whether automatic text simplification (ATS) software can help L2 learners comprehend texts in second and foreign languages. Participants were divided into low and high L2 reading proficiency groups and assigned to read either the authentic or automatically simplified version of a text and completed a free recall task and MC comprehension test. The results did not show any significant correlations among the variables of topic knowledge, topic interest, and MC comprehension, but there were correlations among L2 reading comprehension, MC comprehension, and free recall results. Results also showed that the automatically simplified text facilitated the comprehension of the more proficient readers but not the less proficient readers according to their performance on the free recall assessment. Implications are that L2 teachers cannot blindly use whatever text they want with ATS, and ATS software designers may need to reconsider the current conservative approach to simplification that many ATS tools use. [ABSTRACT FROM AUTHOR]

The task of generating news text summarizations aims to solve the problems of wasting time and read- ing fatigue caused by users'inability to quickly grasp the key points of the content when reading news. At present, the best text summarization model for Chinese is the T5 PEGASUS model, but there are few researches on this model. In this study, the Chinese word segmentation of the T5 PEGASUS model is improved, and the Pkuseg word segmentation method, which is more suitable for news field, is used for processing, and its effectiveness is verified on three public datasets with different news lengths: NLPCC2017, LCSTS and SogouCS. It is found that the Pkuseg method is more suitable for the T5 PEGASUS model. The ROUGE value of T5 Pegasus model generated summaries is positively correlated with the length of news text, and the loss value of training set and the decline speed of loss value are negatively correlated with the length of news text. In the face of a small number of training sets, the model can get a high ROUGE score, so the model has a strong few - shot learning ability. [ABSTRACT FROM AUTHOR]

Many enterprise systems are document-intensive and require extensive manual verification. The verification process has challenge in terms of time and remaining bugs. A general automatic or semi-automatic document verification system would be useful. However, as the nature of the natural language, the context is an important factor. In this research, the target context is selected to be the financial documents, which have been highly interested recently. An automatic document verification model based on only entities (mostly faced within financial documents) was experimented. The summary report was verified with original documents, such that entities in the summary were searched for matching in the original documents. Verification process success was evaluated by comparison of the named entity algorithms in the literature. The special Kaggle data set ready for this purpose was used for entity matching from the summary within the original documents. The average document verification accuracy of named entity finding algorithms for only financial type documents was 85.36%, where the proposed entity recognition algorithm reached 88.80%. On the other hand, the average document verification time of the experimented algorithms and the developed algorithm is 2.43 and 2.48 s respectively. As a conclusion, when both the BERT-base-cased classification model and rule-based approaches are applied specific to the context, it enhances the entity verification process with an insignificant time cost. Consequently, even we used limited data and rules, it is seen that there exists opportunity to automatize the document verification process with the support of both the BERT-base-cased classification model and rule-based approaches. [ABSTRACT FROM AUTHOR]

Today, most newspapers utilize social media to disseminate news. On the one hand, this results in an overload of related articles for social media users. On the other hand, since social media tends to form echo chambers around their users, different opinions and information may be hidden. Enabling users to access different information (possibly outside of their echo chambers, without the burden of reading entire articles, often containing redundant information) may be a step forward in allowing them to form their own opinions. To address this challenge, we propose a system that integrates Transformer neural models and text summarization models along with decision rules. Given a reference article already read by the user, our system first collects articles related to the same topic from a configurable number of different sources. Then, it identifies and summarizes the information that differs from the reference article and outputs the summary to the user. The core of the system is the sentence classification algorithm, which classifies sentences in the collected articles into three classes based on similarity with the reference article: sentences classified as dissimilar are summarized by using a pre-trained abstractive summarization model. We evaluated the proposed system in two steps. First, we assessed its effectiveness in identifying content differences between the reference article and the related articles by using human judgments obtained through crowdsourcing as ground truth. We obtained an average F1 score of 0.772 against average F1 scores of 0.797 and 0.676 achieved by two state-of-the-art approaches based, respectively, on model tuning and prompt tuning, which require an appropriate tuning phase and, therefore, greater computational effort. Second, we asked a sample of people to evaluate how well the summary generated by the system represents the information that is not present in the article read by the user. The results are extremely encouraging. Finally, we present a use case. [ABSTRACT FROM AUTHOR]

Aiming at the problems of traditional text abstract extraction algorithms for processing long text of Chinese patents and unsatisfactory results of long abstract generation, the PatBertSum algorithm is proposed, which enables the algorithm to process long (more than 1500 words) patent text with high efficiency and generate high-quality long (more than 200 words) text summaries. The method is based on the improved BertSum algorithm model, using the new CLTPDS patented text dataset, processing long texts by Head-Tail, transforming Chinese input representations, generating sentence vectors using a pre-trained model, and capturing internal text features and text structure features to extract summaries. Experimentally, this paper demonstrates that the method has improved the recall and F-value of ROUGE by more than 8 percentage points compared with existing methods. [ABSTRACT FROM AUTHOR]

Unsupervised opinion summarization is the technique of automatically generates summaries without gold reference, and the summaries that reflects aspects of information about the entity. Although there are more mature studies on unsupervised opinion summarazaiton, but these studies focus more on unsupervised training methods and ignore the extraction of information by the model. In this paper, we propose JointSum, an unsupervised opinion summarization method based on variational autoencoder model. JointSum first extracts aspect and sentiment information in reviews by joint attention and dictionary learning, respectively. Joint attention consists of text attention and auxiliary attention, which can extract key information in the input text from different fine-grained levels. Then we calculate the variance and mean of the Gaussian distribution in variational autoencoder model using aspect and sentiment information. In addition, we added the review score prediction subtask to increase the robustness of the model. Finally, in generation phase, we adopt pointer-generator network because it includes copy and coverage mechanism that can solve problems in text generation. Experiments on Amazon and Yelp datasets, the results show that the model has good performance in both automatic and human evaluation, the ROUGE-L value on the Yelp dataset gets 20.83. [ABSTRACT FROM AUTHOR]

The discussion in this paper is related to extracting a single lemma from different morphological variants related to a particular dictionary root word. The existing popular online lemmatizers like the Stanford LemmaProcessor, Spacy Lemmatizer, LemmaGen, MorphAdorner, etc. generate the correct lemmas for all singular-plural nouns and all verbal words existing in different tenses, but all these lemmatizers are not able to derive the correct lemma for any type of derived words; specially for nominalized derived words. The proposed lemmatizer – 'LemmaQuest' is designed and implemented to overcome these limitations. LemmaQuest first creates distinct groups for all allied morphed words like singular-plural nouns, verbs in all tenses, and nominalized words. These groups are created based on a combination of different statistical distance measures considering all possible pairs of input words. After that, lemmas are generated for each group. The main objective of this proposed model is to extract the correct lemma for a set of a large number of input words in an optimized time, which leads to a vast improvement in text simplification, keyword extraction, text summarization and other text mining applications. [ABSTRACT FROM AUTHOR]