Search

Nowadays, the Internet is a structure that people can access easily and at the same time produce content easily and without control. In parallel with this situation, the ability of extract information from the raw data that makes up big data has become more complex. The fact that the headlines of the contents contain uncontrolled and misleading elements makes it difficult to reach the right information. The headlines of the contents are important for people to reach the information they want in their limited time. In this study, it is aimed to produce headlines suitable for the content instead of headlines that may be misleading for news. For this purpose, an application that produces headlines for Turkish news with deep learning method has been developed. SuDer news corpus is used as dataset. For the training of the model, it is aimed to obtain more humanoid results in the production of news headlines by using the Transformer architecture, which is frequently preferred in natural language studies today and the abstract summarization method. In this study, in order to compare the performance of the Transformer model, models are prepared and trained with Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) architectures. At the end of 25 epochs of training with LSTM, GRU and Transformer architectures on the corpus, the values of loss are 1.03, 0.55 and 2.49 respectively. In the experiments performed on the validation data, measurements are made with ROUGE-1, ROUGE-2 and ROUGE-L metrics. As a result of the measurements, it is observed that the Transformer architecture is partially good, based on the metric values produced. In addition, when the headlines produced with these architectures are examined, it is observed that the headline obtained with the Transformer architecture produce headlines that are partially more suitable for the news content compared to other architectures. [ABSTRACT FROM AUTHOR]

Natural Language Generation (NLG) has improved exponentially in recent years thanks to the development of sequence-to-sequence deep learning technologies such as Transformer-based language models. This advancement has led to more fluent and coherent NLG, leading to improved development in downstream tasks such as abstractive summarization, dialogue generation, and data-to-text generation. However, it is also apparent that deep learning based generation is prone to hallucinate unintended text, which degrades the system performance and fails to meet user expectations in many real-world scenarios. To address this issue, many studies have been presented in measuring and mitigating hallucinated texts, but these have never been reviewed in a comprehensive manner before. In this survey, we thus provide a broad overview of the research progress and challenges in the hallucination problem in NLG. The survey is organized into two parts: (1) a general overview of metrics, mitigation methods, and future directions, and (2) an overview of task-specific research progress on hallucinations in the following downstream tasks, namely abstractive summarization, dialogue generation, generative question answering, data-to-text generation, and machine translation. This survey serves to facilitate collaborative efforts among researchers in tackling the challenge of hallucinated texts in NLG. [ABSTRACT FROM AUTHOR]

Abstractive text summarization is crucial to produce summaries of natural language with basic concepts fromlarge text documents. Despite the achievement of English language-related abstractive text summarization models, the models that support Arabic language text summarization are fewer in number. Recent abstractive Arabic summarization models encounter different issues that need to be resolved. Syntax inconsistency is a crucial issue resulting in the low-accuracy summary. A new technique has achieved remarkable outcomes by adding topic awareness in the text summarization process that guides the module by imitating human awareness. The current research article presents Abstractive Arabic Text Summarization using Hyperparameter Tuned Denoising Deep Neural Network (AATS-HTDDNN) technique. The presented AATS-HTDDNN technique aims to generate summaries of Arabic text. In the presented AATS-HTDDNN technique, the DDNN model is utilized to generate the summary. This study exploits the Chameleon Swarm Optimization (CSO) algorithm to fine-tune the hyperparameters relevant to the DDNN model since it considerably affects the summarization efficiency. This phase shows the novelty of the current study. To validate the enhanced summarization performance of the proposed AATS-HTDDNN model, a comprehensive experimental analysis was conducted. The comparison study outcomes confirmed the better performance of the AATS-HTDDNN model over other approaches. [ABSTRACT FROM AUTHOR]

In this paper, we present a rigorous mathematical examination of generative pre-trained transformer (GPT) models and their autoregressive self-supervised learning mechanisms. We begin by defining natural language space and knowledge space, which are two key concepts for understanding the dimensionality reduction process in GPT-based large language models (LLMs). By exploring projection functions and their inverses, we establish a framework for analyzing the language generation capabilities of these models. We then investigate the GPT representation space, examining its implications for the models' approximation properties. Finally, we discuss the limitations and challenges of GPT models and their learning mechanisms, considering trade-offs between complexity and generalization, as well as the implications of incomplete inverse projection functions. Our findings demonstrate that GPT models possess the capability to encode knowledge into low-dimensional vectors through their autoregressive self-supervised learning mechanism. This comprehensive analysis provides a solid mathematical foundation for future advancements in GPT-based LLMs, promising advancements in natural language processing tasks such as language translation, text summarization, and question answering due to improved understanding and optimization of model training and performance. [ABSTRACT FROM AUTHOR]

Large text documents are sometimes challenging to understand and time-consuming to extract vital information from. These issues are addressed by automatic text summarizing techniques, which condense lengthy texts while preserving their key information. Thus, the development of automatic summarization systems capable of fulfilling the ever-increasing demands of textual data becomes of utmost importance. It is even more vital with complex natural languages. This study explores five State-Of-The-Art (SOTA) Arabic deep Transformer-based Language Models (TLMs) in the task of text summarization by adapting various text summarization datasets dedicated to Arabic. A comparison against deep learning and machine learning-based baseline models has also been conducted. Experimental results reveal the superiority of TLMs, specifically the PEAGASUS family, against the baseline approaches, with an average F1-score of 90% on several benchmark datasets. [ABSTRACT FROM AUTHOR]