Search

Automatic detection of prerequisite relations between concepts in education has always been a challenging AI task for researchers. Identification of prerequisite relations enables students to study new subjects more effectively and systematically, while allowing instructors to better tailor their learning materials to students' needs. However, to accurately detect these relations, the AI system must understand the context and meaning behind each concept and how it relates to other concepts in the domain. This requires a deep understanding of the educational curriculum and the ability to analyze large amounts of text and data. Large language models (LLMs) are a recent innovation in AI. LLMs have the capability to understand and generate human-like text since they are trained on a vast amount of text from the internet, books, articles, and more. LLMs can also be fine-tuned to specialize in tasks such as document summarization, question answering, or detecting user sentiment in reviews. Fine-tuning is done with a smaller, task-specific dataset. In this work, we introduce strategies for fine-tuning LLMs to improve their capability to detect prerequisite relations between educational concepts. To the best of our knowledge, this is the first work that utilizes fine-tuned LLMs for prerequisite detection. Our evaluation results demonstrate that fine-tuned LLMs are effective models for prerequisite detection. Our fine-tuning process also facilitates the generation of explanations to shed light on the reasoning behind prerequisite relations. Datasets we generated and used in fine-tuning are made public for the research community. We hope our contributions can aid in organizing and presenting knowledge in education and serve as a foundation for future research in the field. [ABSTRACT FROM AUTHOR]

Medical imaging description and disease diagnosis are vitally important yet time-consuming. Automated diagnosis report generation (DRG) from medical imaging description can reduce clinicians' workload and improve their routine efficiency. To address this natural language generation task, fine-tuning a pre-trained large language model (LLM) is cost-effective and indispensable, and its success has been witnessed in many downstream applications. However, semantic inconsistency of sentence embeddings has been massively observed from undesirable repetitions or unnaturalness in text generation. To address the underlying issue of anisotropic distribution of token representation, in this study, a contrastive learning penalized cross-entropy (CLpCE) objective function is implemented to enhance the semantic consistency and accuracy of token representation by guiding the fine-tuning procedure towards a specific task. Furthermore, to improve the diversity of token generation in text summarization and to prevent sampling from unreliable tail of token distributions, a diversity contrastive search (DCS) decoding method is designed for restricting the report generation derived from a probable candidate set with maintained semantic coherence. Furthermore, a novel metric named the maximum of token repetition ratio (maxTRR) is proposed to estimate the token diversity and to help determine the candidate output. Based on the LLM of a generative pre-trained Transformer 2 (GPT-2) of Chinese version, the proposed CLpCE with DCS (CLpCEwDCS) decoding framework is validated on 30,000 desensitized text samples from the "Medical Imaging Diagnosis Report Generation" track of 2023 Global Artificial Intelligence Technology Innovation Competition. Using four kinds of metrics evaluated from n-gram word matching, semantic relevance, and content similarity as well as the maxTRR metric extensive experiments reveal that the proposed framework effectively maintains semantic coherence and accuracy (BLEU-1, 0.4937; BLEU-2, 0.4107; BLEU-3, 0.3461; BLEU-4, 0.2933; METEOR, 0.2612; ROUGE, 0.5182; CIDER, 1.4339) and improves text generation diversity and naturalness (maxTRR, 0.12). The phenomenon of dull or repetitive text generation is common when fine-tuning pre-trained LLMs for natural language processing applications. This study might shed some light on relieving this issue by developing comprehensive strategies to enhance semantic coherence, accuracy and diversity of sentence embeddings. [ABSTRACT FROM AUTHOR]

GPT-3 is a neuronal language model that performs tasks such as classification, question-answering and text summarization. Although chatbots like BlenderBot-3 work well in a conversational sense, and GPT-3 can assist experts in evaluating questions, they are quantifiably worse than real teachers in several pedagogical dimensions. We present the first systematic literature review that analyzes the main contributions and uses of GPT-3 in the field of education. The protocols suggested in the PRISMA 2020 statement were followed for the drafting of the review. According to the results, 34 significant productions were identified through a systematic search in ISI Web of Science, SCOPUS and Google Scholar. GPT-3 has been considered in the academic, ethical and medical fields, in humanities and in computer science, in the formulation of questions and answers, and through cooperative educational dialogs. GPT-3 has been proven to have valuable applications in education, such as the automation of routine tasks, in making quick diagnoses of the students' weaknesses and in the automatic generation of questions, but it still faces challenges and limitations that require additional investigation. We discuss the educational possibilities and the limitations to the use of GPT-3. [ABSTRACT FROM AUTHOR]

This document is a summary of a special issue of the journal Future Internet, focusing on the state of Future Internet Technology in Italy. The issue covers a range of topics, including anonymous service delivery, virtual environments for furnishings sales, Italian-language sequence-to-sequence models for text summarization, knowledge scaffolding in Wikipedia, the integration of Operation Technology and Information Technology networking in Industry 4.0, a digital platform for healthcare operators, self-admitted technical debt in blockchain initiatives, unified analytics for smart cities, the integration of natural language processing and artificial intelligence in e-learning, and a system for attack detection in network architectures. The authors provide detailed explanations and evaluations of each topic, highlighting their potential benefits and applications. [Extracted from the article]

Large language models (LLMs) have substantially pushed artificial intelligence (AI) research and applications in the last few years. They are currently able to achieve high effectiveness in different natural language processing (NLP) tasks, such as machine translation, named entity recognition, text classification, question answering, or text summarization. Recently, significant attention has been drawn to OpenAI's GPT models' capabilities and extremely accessible interface. LLMs are nowadays routinely used and studied for downstream tasks and specific applications with great success, pushing forward the state of the art in almost all of them. However, they also exhibit impressive inference capabilities when used off the shelf without further training. In this paper, we aim to study the behavior of pre‐trained language models (PLMs) in some inference tasks they were not initially trained for. Therefore, we focus our attention on very recent research works related to the inference capabilities of PLMs in some selected tasks such as factual probing and common‐sense reasoning. We highlight relevant achievements made by these models, as well as some of their current limitations that open opportunities for further research. This article is categorized under:Fundamental Concepts of Data and Knowledge > Key Design Issues in Data MiningTechnologies > Artificial Intelligence [ABSTRACT FROM AUTHOR]

This article discusses the role of Artificial Intelligence (AI) in supporting the production and distribution of high-quality news in the face of challenges such as misinformation and declining trust in institutions. The article explores how AI, including Machine Learning and generative AI, can be used in various stages of news production and dissemination. It also highlights the importance of considering factors beyond technology, such as organizational and economic contexts, in order to successfully harness the potential of AI in the media sector. The article presents several papers that examine different aspects of AI's impact on news, including its influence on audience behavior, financial reporting, local journalism, news recommendation systems, and misinformation correction. The authors emphasize the need for ethical considerations and collaboration among stakeholders to ensure the benefits of AI in supporting quality media production. [Extracted from the article]

Eliciting informative user opinions from online reviews is a key success factor for innovative product design and development. The unstructured, noisy, and verbose nature of user reviews, however, often complicate large-scale need finding in a format useful for designers without losing important information. Recent advances in abstractive text summarization have created the opportunity to systematically generate opinion summaries from online reviews to inform the early stages of product design and development. However, two knowledge gaps hinder the applicability of opinion summarization methods in practice. First, there is a lack of formal mechanisms to guide the generative process with respect to different categories of product attributes and user sentiments. Second, the annotated training datasets needed for supervised training of abstractive summarization models are often difficult and costly to create. This article addresses these gaps by (1) devising an efficient computational framework for abstractive opinion summarization guided by specific product attributes and sentiment polarities, and (2) automatically generating a synthetic training dataset that captures various degrees of granularity and polarity. A hierarchical multi-instance attribute-sentiment inference model is developed for assembling a high-quality synthetic dataset, which is utilized to fine-tune a pretrained language model for abstractive summary generation. Numerical experiments conducted on a large dataset scraped from three major e-Commerce retail stores for apparel and footwear products indicate the performance, feasibility, and potentials of the developed framework. Several directions are provided for future exploration in the area of automated opinion summarization for user-centered design. [ABSTRACT FROM AUTHOR]

Although pretrained language models achieve high performance on various natural language processing tasks, they still require further improvements in the sentence embedding task. Many studies have improved performance in this task using pre-trained language models and contrastive learning, but these approaches are limited because they are based on naive average pooling and CLS tokens. Therefore, we propose an advanced sentence-embedding method based on weighted pooling that considers token importance. Specifically, the token importance is calculated by combining an explainable artificial-intelligence module with a text summarization model, and the final sentence embedding is derived through weighted pooling token embedding and token importance. Thus, we derive a sentence embedding that considers both the local information of the token embedding and the global information of the entire sentence. Experimental results reveal that our proposed sentence embedding outperforms other models on both text similarity tasks and text classification. Moreover, the proposed method's robustness is verified through the results of an ablation study. [ABSTRACT FROM AUTHOR]