Search

Large language models (LLMs) have substantially pushed artificial intelligence (AI) research and applications in the last few years. They are currently able to achieve high effectiveness in different natural language processing (NLP) tasks, such as machine translation, named entity recognition, text classification, question answering, or text summarization. Recently, significant attention has been drawn to OpenAI's GPT models' capabilities and extremely accessible interface. LLMs are nowadays routinely used and studied for downstream tasks and specific applications with great success, pushing forward the state of the art in almost all of them. However, they also exhibit impressive inference capabilities when used off the shelf without further training. In this paper, we aim to study the behavior of pre‐trained language models (PLMs) in some inference tasks they were not initially trained for. Therefore, we focus our attention on very recent research works related to the inference capabilities of PLMs in some selected tasks such as factual probing and common‐sense reasoning. We highlight relevant achievements made by these models, as well as some of their current limitations that open opportunities for further research. This article is categorized under:Fundamental Concepts of Data and Knowledge > Key Design Issues in Data MiningTechnologies > Artificial Intelligence [ABSTRACT FROM AUTHOR]

Natural Language Processing (NLP) is the computerized approach to analyzing text using both structured and unstructured data. NLP is a simple, empirically powerful, and reliable approach. It achieves state-of-the-art performance in language processing tasks like Semantic Search (SS), Machine Translation (MT), Text Summarization (TS), Sentiment Analyzer (SA), Named Entity Recognition (NER) and Emotion Detection (ED). NLP is expected to be the technology of the future, based on current technology deployment and adoption. The primary question is: What does NLP have to offer in terms of reality, and what are the prospects? There are several problems to be addressed with this developing method, as it must be compatible with future technology. In this paper, the benefits, challenges and limitations of this innovative paradigm along with the areas open to do research are shown. [ABSTRACT FROM AUTHOR]

Eliciting informative user opinions from online reviews is a key success factor for innovative product design and development. The unstructured, noisy, and verbose nature of user reviews, however, often complicate large-scale need finding in a format useful for designers without losing important information. Recent advances in abstractive text summarization have created the opportunity to systematically generate opinion summaries from online reviews to inform the early stages of product design and development. However, two knowledge gaps hinder the applicability of opinion summarization methods in practice. First, there is a lack of formal mechanisms to guide the generative process with respect to different categories of product attributes and user sentiments. Second, the annotated training datasets needed for supervised training of abstractive summarization models are often difficult and costly to create. This article addresses these gaps by (1) devising an efficient computational framework for abstractive opinion summarization guided by specific product attributes and sentiment polarities, and (2) automatically generating a synthetic training dataset that captures various degrees of granularity and polarity. A hierarchical multi-instance attribute-sentiment inference model is developed for assembling a high-quality synthetic dataset, which is utilized to fine-tune a pretrained language model for abstractive summary generation. Numerical experiments conducted on a large dataset scraped from three major e-Commerce retail stores for apparel and footwear products indicate the performance, feasibility, and potentials of the developed framework. Several directions are provided for future exploration in the area of automated opinion summarization for user-centered design. [ABSTRACT FROM AUTHOR]

Generative ability is a crucial need for artificial intelligence applications, such as chatbots, virtual assistants, machine translation systems etc. In recent years, the transformer-based neural architectures gave a huge boost to generate human-like English texts. In our research we did experiments to create pre-trained generative transformer models for Hungarian language and fine-tune them for multiple types of natural language processing tasks. In our focus, multilingual models were trained. We have pre-trained a multilingual BART, then fine-tuned it to various NLP tasks, such as text classification, abstractive summarization. In our experiments, we focused on transfer learning techniques to increase the performance. Furthermore, a M2M100 multilingual model was fine-tuned for a 12-lingual Hungarian- Centric machine translation. Last but not least, a Marian NMT based machine translation system was also built from scratch for the 12-lingual Hungarian-Centric machine translation task. In our results, using the cross-lingual transfer method we could achieve higher performance in all of our tasks. In our machine translation experiment, using our fine-tuned M2M100 model we could outperform the Google Translate, Microsoft Translator and eTranslation. [ABSTRACT FROM AUTHOR]

Although pretrained language models achieve high performance on various natural language processing tasks, they still require further improvements in the sentence embedding task. Many studies have improved performance in this task using pre-trained language models and contrastive learning, but these approaches are limited because they are based on naive average pooling and CLS tokens. Therefore, we propose an advanced sentence-embedding method based on weighted pooling that considers token importance. Specifically, the token importance is calculated by combining an explainable artificial-intelligence module with a text summarization model, and the final sentence embedding is derived through weighted pooling token embedding and token importance. Thus, we derive a sentence embedding that considers both the local information of the token embedding and the global information of the entire sentence. Experimental results reveal that our proposed sentence embedding outperforms other models on both text similarity tasks and text classification. Moreover, the proposed method's robustness is verified through the results of an ablation study. [ABSTRACT FROM AUTHOR]

Precision education is a new challenge in leveraging artificial intelligence, machine learning, and learning analytics to enhance teaching quality and learning performance. To facilitate precision education, text marking skills can be used to determine students' learning process. Text marking is an essential learning skill in reading. In this study, we proposed a model that leverages the state-of-the-art text summarization technique, Bidirectional Encoder Representations from Transformers (BERT), to calculate the marking score for 130 graduate students enrolled in an accounting course. Then, we applied learning analytics to analyze the correlation between their marking scores and learning performance. We measured students' self-regulated learning (SRL) and clustered them into four groups based on their marking scores and marking frequencies to examine whether differences in reading skills and text marking influence students' learning performance and awareness of self-regulation. Consistent with past research, our results did not indicate a strong relationship between marking scores and learning performance. However, high-skill readers who use more marking strategies perform better in learning performance, task strategies, and time management than high-skill readers who use fewer marking strategies. Furthermore, high-skill readers who actively employ marking strategies also achieve superior scores of environment structure, and task strategies in SRL than low-skill readers who are inactive in marking. The findings of this research provide evidence supporting the importance of monitoring and training students' text marking skill and facilitating precision education. [ABSTRACT FROM AUTHOR]

Natural language processing is the trending topic in the latest research areas, which allows the developers to create the human-computer interactions to come into existence. The natural language processing is an integration of artificial intelligence, computer science and computer linguistics. The research towards natural Language Processing is focused on creating innovations towards creating the devices or machines which operates basing on the single command of a human. It allows various Bot creations to innovate the instructions from the mobile devices to control the physical devices by allowing the speech-tagging. In our paper, we design a search engine which not only displays the data according to user query but also performs the detailed display of the content or topic user is interested for using the summarization concept. We find the designed search engine is having optimal response time for the user queries by analyzing with number of transactions as inputs. Also, the result findings in the performance analysis show that the text summarization method has been an efficient way for improving the response time in the search engine optimizations. [ABSTRACT FROM AUTHOR]

Semantic applications typically exploit structures such as dependency parse trees, phrase-chunking, semantic role labeling or open information extraction. In this paper, we introduce a novel application of Open IE as an intermediate layer for text summarization. Text summarization is an important method for providing relevant information in large digital libraries. Open IE is referred to the process of extracting machine-understandable structural propositions from text. We use these propositions as a building block to shorten the sentence and generate a summary of the text. The proposed system offers a new form of summarization that is able to break the structure of the sentence and extract the most significant sub-sentential elements. Other advantages include the ability to identify and eliminate less important sections of the sentence (such as adverbs, adjectives, appositions or dependent clauses), or duplicate pieces of sentences which in turn opens up the space for entering more sentences in the summary to enhance the coverage and coherency of it. The proposed system is localized for Persian language; however, it can be adopted to other languages. Experiments performed on a standard data set “Pasokh” with a standard comparison tool showed promising results for the proposed approach. We used summaries produced by the system in a real-world application in the virtual library of Shahid Beheshti University and received good feedbacks from users. [ABSTRACT FROM AUTHOR]

Text summarization namely, automatically generating a short summary of a given document, is a difficult task in natural language processing. Nowadays, deep learning as a new technique has gradually been deployed for text summarization, but there is still a lack of large-scale high quality datasets for this technique. In this paper, we proposed a novel deep learning method to identify high quality document--summary pairs for building a large-scale pairs dataset. Concretely, a long short-term memory (LSTM)-based model was designed to measure the quality of document--summary pairs. In order to leverage information across all parts of each document, we further proposed an improved LSTM-based model by removing the forget gate in the LSTM unit. Experiments conducted on the training set and the test set built upon Sina Weibo (a Chinese microblog website similar to Twitter) showed that the LSTM-based models significantly outperformed baseline models with regard to the area under receiver operating characteristic curve (AUC) value. [ABSTRACT FROM AUTHOR]

Text summarization is either extractive or abstractive. Extractive summarization is to select the most salient pieces of information (words, phrases, and/or sentences) from a source document without adding any external information. Abstractive summarization allows an internal representation of the source document so as to produce a faithful summary of the source. In this case, external text can be inserted into the generated summary. Because of the complexity of the abstractive approach, the vast majority of work in text summarization has adopted an extractive approach. In this work, we focus on concepts fusion and generalization, i.e. where different concepts appearing in a sentence can be replaced by one concept which covers the meanings of all of them. This is one operation that can be used as part of an abstractive text summarization system. The main goal of this contribution is to enrich the research efforts on abstractive text summarization with a novel approach that allows the generalization of sentences using semantic resources. This work should be useful in intelligent systems more generally since it introduces a means to shorten sentences by producing more general (hence abstractions of the) sentences. It could be used, for instance, to display shorter texts in applications for mobile devices. It should also improve the quality of the generated text summaries by mentioning key (general) concepts. One can think of using the approach in reasoning systems where different concepts appearing in the same context are related to one another with the aim of finding a more general representation of the concepts. This could be in the context of Goal Formulation, expert systems, scenario recognition, and cognitive reasoning more generally. We present our methodology for the generalization and fusion of concepts that appear in sentences. This is achieved through (1) the detection and extraction of what we define as generalizable sentences and (2) the generation and reduction of the space of generalization versions. We introduce two approaches we have designed to select the best sentences from the space of generalization versions. Using four NLTK 1 1 The Natural Language Tool Kit ( http://nltk.org/ ) corpora, the first approach estimates the “acceptability” of a given generalization version. The second approach is Machine Learning-based and uses contextual and specific features. The recall, precision and F1-score measures resulting from the evaluation of the concept generalization and fusion approach are presented. [ABSTRACT FROM AUTHOR]

Abstract: The rises of the life index quality together with the medical technology improvement lead to a longer life expectancy. Thus a better health care program, especially for elderly, is needed. The common health problems facing those senior citizens are changed from acute diseases to chronic diseases, such as diabetes, hypertension, etc. Along with these changes, medical tourism is becoming the trend of the future. In this paper, we propose a decision support systems, the Ubiquitous Context-aware Healthcare Service System (UCHS), which uses micro sensors integrate RFID to sense user’s life vital signal, such as electrocardiogram (ECG/EKG), heart rate (HR), respiratory rate (RR), blood pressure (BP), blood sugar (BS), and temperature and light. The UCHS is composted of Situation-Aware Medical Tourism Service Search Subsystem (SAMTS3), Healthy-life Map Guiding Subsystem (HMGS), Intelligent Curative Food Decision Support Subsystem (ICFDSS), and 4D Emergency Indication and Ambulance Dispatch Subsystem (4DEIADS) to provide relevant nature medicine recommendations to its user. The UCHS built upon an integrated service platform in which medical experts’ knowledge and all position and negative influence of the proposed therapy are inferred by using semantic network. [ABSTRACT FROM AUTHOR]