Search

The object of the research is the task of automatic abstracting of natural language texts. The importance of these tasks is determined by the existing problem of creating essays that would adequately reflect the content of the original text and highlight key information. This task requires the ability of models to deeply analyze the context of the text, which complicates the abstracting process. Results are presented that demonstrate the effectiveness of using generative models based on neural networks, text semantic analysis methods, and deep learning for automatic creation of abstracts. The use of models showed a high level of adequacy and informativeness of abstracts. GPT (Generative Pre-trained Transformer) generates text that looks like it was written by a human, which makes it useful for automatic essay generation. For example, the GPT model generates abbreviated summaries based on input text, while the BERT model is used for summarizing texts in many areas, including search engines and natural language processing. This allows for short but informative abstracts that retain the essential content of the original and provides the ability to produce high-quality abstracts that can be used for abstracting web pages, emails, social media, and other content. Compared to traditional abstracting methods, artificial intelligence provides such advantages as greater accuracy, informativeness and the ability to process large volumes of text more efficiently, which facilitates access to information and improves productivity in text processing. Automatic abstracting of texts using artificial intelligence models allows to significantly reduce the time required for the analysis of large volumes of textual information. This is especially important in today’s information environment, where the amount of available data is constantly growing. The use of these models promotes efficient use of resources and increases overall productivity in a variety of fields, including scientific research, education, business and media [ABSTRACT FROM AUTHOR]

A long-term objective of artificial intelligence is to design an abstractive text summarization (ATS) system that can produce condensed, adequate, and realistic summaries for the source documents. Deep learning approaches have contributed significantly to recent advancements in ATS, taking the state-of-the-art to new heights. Despite the considerable success of earlier approaches, producing high-quality and more human-like abstractive summaries continues to be a challenging task in reality. In this paper, we present an adversarial framework for abstractive text summarization, in which a generative model G and a proposed deep discriminative model D are trained in an adversarial manner. The goal of the generator is to generate summaries that are hard to differentiate from real summaries, whereas the discriminator's role is to estimate the probability that a summary came from the training data rather than the generator to direct the generative model's training. Experimental results on a benchmark CNN/Daily Mail dataset demonstrate that the proposed model achieves Rouge-1 and Rouge-L scores of 41.58 and 38.96 respectively which are better than the ones reported by various other methods (e.g. the base paper [1] achieved Rouge-1 and Rouge-L score of 39.92 and 36.71, while one of the recent works ACGT [2] achieved Rouge-1 score of 40.49 and Rouge-L score of 37.41). The manuscript's unique characteristic is qualitative evaluation, which, along with quantitative evaluation, shows that the proposed model is superior. [ABSTRACT FROM AUTHOR]

Medical imaging description and disease diagnosis are vitally important yet time-consuming. Automated diagnosis report generation (DRG) from medical imaging description can reduce clinicians' workload and improve their routine efficiency. To address this natural language generation task, fine-tuning a pre-trained large language model (LLM) is cost-effective and indispensable, and its success has been witnessed in many downstream applications. However, semantic inconsistency of sentence embeddings has been massively observed from undesirable repetitions or unnaturalness in text generation. To address the underlying issue of anisotropic distribution of token representation, in this study, a contrastive learning penalized cross-entropy (CLpCE) objective function is implemented to enhance the semantic consistency and accuracy of token representation by guiding the fine-tuning procedure towards a specific task. Furthermore, to improve the diversity of token generation in text summarization and to prevent sampling from unreliable tail of token distributions, a diversity contrastive search (DCS) decoding method is designed for restricting the report generation derived from a probable candidate set with maintained semantic coherence. Furthermore, a novel metric named the maximum of token repetition ratio (maxTRR) is proposed to estimate the token diversity and to help determine the candidate output. Based on the LLM of a generative pre-trained Transformer 2 (GPT-2) of Chinese version, the proposed CLpCE with DCS (CLpCEwDCS) decoding framework is validated on 30,000 desensitized text samples from the "Medical Imaging Diagnosis Report Generation" track of 2023 Global Artificial Intelligence Technology Innovation Competition. Using four kinds of metrics evaluated from n-gram word matching, semantic relevance, and content similarity as well as the maxTRR metric extensive experiments reveal that the proposed framework effectively maintains semantic coherence and accuracy (BLEU-1, 0.4937; BLEU-2, 0.4107; BLEU-3, 0.3461; BLEU-4, 0.2933; METEOR, 0.2612; ROUGE, 0.5182; CIDER, 1.4339) and improves text generation diversity and naturalness (maxTRR, 0.12). The phenomenon of dull or repetitive text generation is common when fine-tuning pre-trained LLMs for natural language processing applications. This study might shed some light on relieving this issue by developing comprehensive strategies to enhance semantic coherence, accuracy and diversity of sentence embeddings. [ABSTRACT FROM AUTHOR]

The challenges faced by the massive increase in scientific publications draw parallels to the Larsen effect, where an amplified sound loop leads to escalating noise. This phenomenon has resulted in information overload, making it difficult for researchers to stay updated and identify significant findings. To address this, knowledge synthesis techniques are recommended. These methods help synthesize and visualize large bodies of literature, aiding researchers in navigating the expanding information landscape. Furthermore, artificial intelligence (AI) and natural language processing tools, such as text summarization, offer innovative solutions for managing information overload. However, the overuse of AI in producing scientific literature raises concerns about the quality and integrity of research. This manuscript highlights the need for balanced use of AI tools and collaborative efforts to maintain high‐quality scientific output while leveraging the benefits of extensive research. [ABSTRACT FROM AUTHOR]

This document is a summary of a special issue of the journal Future Internet, focusing on the state of Future Internet Technology in Italy. The issue covers a range of topics, including anonymous service delivery, virtual environments for furnishings sales, Italian-language sequence-to-sequence models for text summarization, knowledge scaffolding in Wikipedia, the integration of Operation Technology and Information Technology networking in Industry 4.0, a digital platform for healthcare operators, self-admitted technical debt in blockchain initiatives, unified analytics for smart cities, the integration of natural language processing and artificial intelligence in e-learning, and a system for attack detection in network architectures. The authors provide detailed explanations and evaluations of each topic, highlighting their potential benefits and applications. [Extracted from the article]

Large language models (LLMs) have substantially pushed artificial intelligence (AI) research and applications in the last few years. They are currently able to achieve high effectiveness in different natural language processing (NLP) tasks, such as machine translation, named entity recognition, text classification, question answering, or text summarization. Recently, significant attention has been drawn to OpenAI's GPT models' capabilities and extremely accessible interface. LLMs are nowadays routinely used and studied for downstream tasks and specific applications with great success, pushing forward the state of the art in almost all of them. However, they also exhibit impressive inference capabilities when used off the shelf without further training. In this paper, we aim to study the behavior of pre‐trained language models (PLMs) in some inference tasks they were not initially trained for. Therefore, we focus our attention on very recent research works related to the inference capabilities of PLMs in some selected tasks such as factual probing and common‐sense reasoning. We highlight relevant achievements made by these models, as well as some of their current limitations that open opportunities for further research. This article is categorized under:Fundamental Concepts of Data and Knowledge > Key Design Issues in Data MiningTechnologies > Artificial Intelligence [ABSTRACT FROM AUTHOR]

As the demand for edge services intensifies, text, being the most common type of data, has seen a significant expansion in data volume and an escalation in processing complexity. Furthermore, mobile edge computing (MEC) service systems often faces challenges such as limited computational capabilities and difficulties in data integration, requiring the development and implementation of more efficient and lightweight methodologies for text data processing. To swiftly extract and analysis vital information from MEC text data, an automatic generation scheme of multi-document text summarization based on knowledge graph is proposed in this paper, named KGCPN. For the text data from MEC devices and applications, the natural language processing technology is used to execute the data pre-processing steps, which transforms the MEC text data into a computationally tractable and semantically comprehensible format. Then, the knowledge graph of multi-document text is constructed by integrating the relationship paths and entity descriptions. The nodes and edges of the knowledge graph serve to symbolize the semantic relationships within the text, and the Graph Convolution Neural network (GCN) is used to understand the text and learn the semantic representation. Finally, a pointer-generator network model accepts the encoding information from GCN and automatically generate a general text summarization. The experimental results indicate that our scheme can effectively facilitate the smart pre-processing and integration of MEC data. [ABSTRACT FROM AUTHOR]

Although pretrained language models achieve high performance on various natural language processing tasks, they still require further improvements in the sentence embedding task. Many studies have improved performance in this task using pre-trained language models and contrastive learning, but these approaches are limited because they are based on naive average pooling and CLS tokens. Therefore, we propose an advanced sentence-embedding method based on weighted pooling that considers token importance. Specifically, the token importance is calculated by combining an explainable artificial-intelligence module with a text summarization model, and the final sentence embedding is derived through weighted pooling token embedding and token importance. Thus, we derive a sentence embedding that considers both the local information of the token embedding and the global information of the entire sentence. Experimental results reveal that our proposed sentence embedding outperforms other models on both text similarity tasks and text classification. Moreover, the proposed method's robustness is verified through the results of an ablation study. [ABSTRACT FROM AUTHOR]