Search

The challenges faced by the massive increase in scientific publications draw parallels to the Larsen effect, where an amplified sound loop leads to escalating noise. This phenomenon has resulted in information overload, making it difficult for researchers to stay updated and identify significant findings. To address this, knowledge synthesis techniques are recommended. These methods help synthesize and visualize large bodies of literature, aiding researchers in navigating the expanding information landscape. Furthermore, artificial intelligence (AI) and natural language processing tools, such as text summarization, offer innovative solutions for managing information overload. However, the overuse of AI in producing scientific literature raises concerns about the quality and integrity of research. This manuscript highlights the need for balanced use of AI tools and collaborative efforts to maintain high‐quality scientific output while leveraging the benefits of extensive research. [ABSTRACT FROM AUTHOR]

Large language models (LLMs) have substantially pushed artificial intelligence (AI) research and applications in the last few years. They are currently able to achieve high effectiveness in different natural language processing (NLP) tasks, such as machine translation, named entity recognition, text classification, question answering, or text summarization. Recently, significant attention has been drawn to OpenAI's GPT models' capabilities and extremely accessible interface. LLMs are nowadays routinely used and studied for downstream tasks and specific applications with great success, pushing forward the state of the art in almost all of them. However, they also exhibit impressive inference capabilities when used off the shelf without further training. In this paper, we aim to study the behavior of pre‐trained language models (PLMs) in some inference tasks they were not initially trained for. Therefore, we focus our attention on very recent research works related to the inference capabilities of PLMs in some selected tasks such as factual probing and common‐sense reasoning. We highlight relevant achievements made by these models, as well as some of their current limitations that open opportunities for further research. This article is categorized under:Fundamental Concepts of Data and Knowledge > Key Design Issues in Data MiningTechnologies > Artificial Intelligence [ABSTRACT FROM AUTHOR]

Although pretrained language models achieve high performance on various natural language processing tasks, they still require further improvements in the sentence embedding task. Many studies have improved performance in this task using pre-trained language models and contrastive learning, but these approaches are limited because they are based on naive average pooling and CLS tokens. Therefore, we propose an advanced sentence-embedding method based on weighted pooling that considers token importance. Specifically, the token importance is calculated by combining an explainable artificial-intelligence module with a text summarization model, and the final sentence embedding is derived through weighted pooling token embedding and token importance. Thus, we derive a sentence embedding that considers both the local information of the token embedding and the global information of the entire sentence. Experimental results reveal that our proposed sentence embedding outperforms other models on both text similarity tasks and text classification. Moreover, the proposed method's robustness is verified through the results of an ablation study. [ABSTRACT FROM AUTHOR]