Search

Temporal logic (TL) tasks consist of complex and temporally extended subgoals and they are common for many real-world applications, such as service and navigation robots. However, it is often inefficient or even infeasible to train reinforcement learning (RL) agents to solve multiple TL tasks, since rewards are sparse and non-Markovian in these tasks. A promising solution to this problem is to learn task-conditioned policies which can zero-shot generalize to new TL tasks without further training. However, influenced by some practical issues, such as issues of lossy symbolic observation and long time-horizon of completing TL task, previous works suffer from sample inefficiency in training and sub-optimality (or even infeasibility) in task execution. In order to tackle these issues, this paper proposes an option-based framework to generalize TL tasks, consisting of option training and task execution parts. We have innovations in both parts. In option training, we propose to learn options dependent on the future subgoals via a novel approach. Additionally, we propose to train a multi-step value function which can propagate the rewards of satisfying future subgoals more efficiently in long-horizon tasks. In task execution, in order to ensure the optimality and safety, we propose a model-free MPC planner for option selection, circumventing the learning of a transition model which is required by previous MPC planners. In experiments on three different domains, we evaluate the generalization capability of the agent trained by the proposed method, showing its significant advantage over previous methods. [ABSTRACT FROM AUTHOR]

The move toward artificial intelligence (AI)-native sixth-generation (6G) networks has put more emphasis on the importance of explainability and trustworthiness in network management operations, especially for mission-critical use-cases. Such desired trust transcends traditional post-hoc explainable AI (XAI) methods to using contextual explanations for guiding the learning process in an in-hoc way. This paper proposes a novel graph reinforcement learning (GRL) framework named TANGO which relies on a symbolic subsystem. It consists of a Bayesian-graph neural network (GNN) Explainer, whose outputs, in terms of edge/node importance and uncertainty, are periodically translated to a logical GRL reward function. This adjustment is accomplished through defined symbolic reasoning rules within a Reasoner. Considering a real-world testbed proof-of-concept (PoC), a gNodeB (gNB) radio resource allocation problem is formulated, which aims to minimize under- and over-provisioning of physical resource blocks (PRBs) while penalizing decisions emanating from the uncertain and less important edge-nodes relations. Our findings reveal that the proposed in-hoc explainability solution significantly expedites convergence compared to standard GRL baseline and other benchmarks in the deep reinforcement learning (DRL) domain. The experiment evaluates performance in AI, complexity, energy consumption, robustness, network, scalability, and explainability metrics. Specifically, the results show that TANGO achieves a noteworthy accuracy of 96.39% in terms of optimal PRB allocation in inference phase, outperforming the baseline by $1.22\times $ .

Autonomous robots can be employed in exploring unknown environments and performing many tasks, such as, e.g. detecting areas of interest, collecting target objects, etc. Deep reinforcement learning (RL) is often used to train this kind of robot. However, concerning the artificial environments aimed at testing the robot, there is a lack of available data sets and a long time is needed to create them from scratch. A good data set is in fact usually produced with high effort in terms of cost and human work to satisfy the constraints imposed by the expected results. In the first part of this paper, we focus on the specification of the properties of the solutions needed to build a data set, making the case of environment exploration. In the proposed approach, rather than using imperative programming, we explore the possibility of generating data sets using constraint programming in Prolog. In this phase, geometric predicates describe a virtual environment according to inter-space requirements. The second part of the paper is focused on testing the generated data set in an AI gym via space search techniques. We developed a Neuro-Symbolic agent built from the following: (i) A deep Q-learning component implemented in Python, able to address via RL a search problem in the virtual space; the agent has the goal to explore a generated virtual environment to seek for a target, improving its performance through a RL process. (ii) A symbolic component able to re-address the search when the Q-learning component gets stuck in a part of the virtual environment; these components stimulate the agent to move to and explore other parts of the environment. Wide experimentation has been performed, with promising results, and is reported, to demonstrate the effectiveness of the approach. [ABSTRACT FROM AUTHOR]

As the applications of robots expand across a wide variety of areas, high-level task planning considering human–robot interactions is emerging as a critical issue. Various elements that facilitate flexible responses to humans in an ever-changing environment, such as scene understanding, natural language processing, and task planning, are thus being researched extensively. In this study, a visual question answering (VQA) task was examined in detail from among an array of technologies. By further developing conventional neuro-symbolic approaches, environmental information is stored and utilized in a symmetric graph format, which enables more flexible and complex high-level task planning. We construct a symmetric graph composed of information such as color, size, and position for the objects constituting the environmental scene. VQA, using graphs, largely consists of a part expressing a scene as a graph, a part converting a question into SPARQL, and a part reasoning the answer. The proposed method was verified using a public dataset, CLEVR, with which it successfully performed VQA. We were able to directly confirm the process of inferring answers using SPARQL queries converted from the original queries and environmental symmetric graph information, which is distinct from existing methods that make it difficult to trace the path to finding answers. [ABSTRACT FROM AUTHOR]

The construction of high-quality word embeddings is essential in natural language processing. In existing approaches using a large text corpus, the word embeddings learn only sequential patterns in the context; thus, accurate learning of the syntax and semantic relationships between words is limited. Several methods have been proposed for constructing word embeddings using syntactic information. However, these methods are not trained for the semantic relationships between words in sentences or external knowledge. In this paper, we present a method for improved word embeddings using symbolic graphs for external knowledge and the relationships of the syntax and semantic role between words in sentences. The proposed model sequentially learns two symbolic graphs with different properties through a graph convolutional network (GCN) model. A new symbolic graph representation is generated to understand sentences grammatically and semantically. This graph representation includes comprehensive information that combines dependency parsing and semantic role labeling. Subsequently, word embeddings are constructed through the GCN model. The same GCN model initializes the word representations that are created in the first step and trains the relationships of ConceptNet using the relationships between words. The proposed word embeddings outperform the baselines in benchmarks and extrinsic tasks. [ABSTRACT FROM AUTHOR]

Security and risk assessment aims to prioritize detected vulnerabilities for remediation in a computer networking system. The widely used expert-based risk prioritization approach, e.g., Common Vulnerability Scoring System (CVSS), cannot realistically associate vulnerabilities to the likelihood of exploitation. The CVSS metrics are calculated from static formulas, and cannot easily integrate attackers’ motivations and capabilities w.r.t. the network environmental factors. To address this issue, this paper proposes LICALITY, a vulnerability risk prioritization system. LICALITY captures the attacker’s preference on exploiting vulnerabilities through a threat modeling method, and learns threat attributes that contribute to the exploitation of vulnerability. LICALITY creatively uses a neuro-symbolic model, with neural network (NN) and probabilistic logic programming (PLP) techniques, to learn such threat attributes. The risk of vulnerability is assessed from the criticality of exploitation and the likelihood of exploitation. LICALITY consolidates these two measurements by using a logic reasoning engine. In the evaluation, the historical threat and future threat are from real attack scenarios. The results reveal that LICALITY reduces the vulnerability remediation work of the future threat required by the CVSS by a factor of 2.89 in the first case study and by a factor of 1.85 in the second case study. Such future threats are identified as the top routinely exploited vulnerabilities and the APT attack chained vulnerabilities reported in the Cybersecurity and Infrastructure Security Agency (CISA) alerts. [ABSTRACT FROM AUTHOR]

In this paper we introduce Detect, Understand, Act (DUA), a neuro-symbolic reinforcement learning framework. The Detect component is composed of a traditional computer vision object detector and tracker. The Act component houses a set of options, high-level actions enacted by pre-trained deep reinforcement learning (DRL) policies. The Understand component provides a novel answer set programming (ASP) paradigm for symbolically implementing a meta-policy over options and effectively learning it using inductive logic programming (ILP). We evaluate our framework on the Animal-AI (AAI) competition testbed, a set of physical cognitive reasoning problems. Given a set of pre-trained DRL policies, DUA requires only a few examples to learn a meta-policy that allows it to improve the state-of-the-art on multiple of the most challenging categories from the testbed. DUA constitutes the first holistic hybrid integration of computer vision, ILP and DRL applied to an AAI-like environment and sets the foundations for further use of ILP in complex DRL challenges. [ABSTRACT FROM AUTHOR]

The construction of high-quality word embeddings is essential in natural language processing. In existing approaches using a large text corpus, the word embeddings learn only sequential patterns in the context; thus, accurate learning of the syntax and semantic relationships between words is limited. Several methods have been proposed for constructing word embeddings using syntactic information. However, these methods are not trained for the semantic relationships between words in sentences or external knowledge. In this paper, we present a method for improved word embeddings using symbolic graphs for external knowledge and the relationships of the syntax and semantic role between words in sentences. The proposed model sequentially learns two symbolic graphs with different properties through a graph convolutional network (GCN) model. A new symbolic graph representation is generated to understand sentences grammatically and semantically. This graph representation includes comprehensive information that combines dependency parsing and semantic role labeling. Subsequently, word embeddings are constructed through the GCN model. The same GCN model initializes the word representations that are created in the first step and trains the relationships of ConceptNet using the relationships between words. The proposed word embeddings outperform the baselines in benchmarks and extrinsic tasks.

As the roles of robots continue to expand in general, there is an increasing demand for research on automated task planning for a multi-agent system that can independently execute tasks in a wide and dynamic environment. This study introduces a plugin framework in which multiple robots can be involved in task planning in a broad range of areas by combining symbolic and connectionist approaches. The symbolic approach for understanding and learning human knowledge is useful for task planning in a wide and static environment. The network-based connectionist approach has the advantage of being able to respond to an ever-changing dynamic environment. A planning domain definition language-based planning algorithm, which is a symbolic approach, and the cooperative–competitive reinforcement learning algorithm, which is a connectionist approach, were utilized in this study. The proposed architecture is verified through a simulation. It is also verified through an experiment using 10 unmanned surface vehicles that the given tasks were successfully executed in a wide and dynamic environment.

Recommendation systems assist users in finding items that are relevant to them. However, recommendation is not only an inductive statistics problem using data; it is also a cognitive task of reasoning decisions based on knowledge extracted from information. Hence, a logic system can naturally be incorporated for reasoning in a recommendation task. Although logic-based hard-rule approaches can provide a powerful reasoning ability, they struggle to cope with inconsistent and incomplete knowledge in real-world tasks, especially for complex tasks such as recommendation. To address these issues, we propose a neuro-symbolic recommendation model, which transforms user history interactions into a logic expression and then transforms the recommendation prediction into a query task based on this logic expression. The logic expressions are then computed based on the modular logic operations of the neural network. We also construct an implicit logic encoder to reasonably reduce the complexity of the logic computation. Finally, a user's interest items can be queried in the vector space based on the computation results. Experiments on three well-known datasets verified that the proposed method outperforms state of the art shallow, deep, session, and reasoning models. • We transforms recommendation problem into a first-order logic-based query problem. • We improve model's characterization ability while reduce its computational complexity. • We use separate neural networks for logic predicate operations. • We demonstrate our method on several real recommendation datasets. [ABSTRACT FROM AUTHOR]

The detection and segmentation of cerebral aneurysms is a crucial step in the development of a clinical decision support system for estimating aneurysm rupture risk. However, accurately identifying and segmenting regions of interest in two-dimensional (2D) medical images is often challenging, particularly when using deep learning (DL) methods on small datasets with limited annotated data. The accuracy of DL approaches is often affected by the availability of large, annotated training datasets that are required for effective deep learning. Additionally, when using DL to differentiate aneurysms from arterial loops in 2D DSA images, DL can fail to detect aneurysms in areas where dye concentration is low. To address these issues and enhance the reliability and accuracy of aneurysm detection and segmentation methods, incorporating medical expert-advised, hand-crafted features can provide a clinical perspective to DL methods. This approach can help to improve the performance of DL methods by providing additional information that is not captured in the data. To this end, a novel Neuro-symbolic AI-based DeepInfusion model is proposed which allows for the infusion of human intellect through hand-crafted features into deep neural networks (DNNs), thus combining the strengths of DL with the knowledge and expertise of medical professionals. The proposed approach includes a novel technique for dynamic layer selection and feature weight adjustment during the model infusion process. The performance of the DeepInfusion model is evaluated on an in-house prepared dataset of 409 DSA images, and experimental results demonstrate the effectiveness of the proposed method for the segmentation of cerebral aneurysms. The model achieves an IOU score of 96.76% and an F1-score of 94.15% on unseen DSA images. The model is also tested on two publicly available datasets of Kvasir-SEG polyp and DRIVE for vessel segmentation of retinal images. The results show a significant improvement compared to existing methods, which indicates the generalizability of the approach in medical segmentation. The complete code for DeepInfusion is available on our GitHub repository at https://github.com/smileslab/deep-infusion/blob/main/deepinfusion.ipynb. [ABSTRACT FROM AUTHOR]

In the general machine learning domain, solutions based on the integration of deep learning models with knowledge-based approaches are emerging. Indeed, such hybrid systems have the advantage of improving the recognition rate and the model's interpretability. At the same time, they require a significantly reduced amount of labeled data to reliably train the model. However, these techniques have been poorly explored in the sensor-based Human Activity Recognition (HAR) domain. The common-sense knowledge about activity execution can potentially improve purely data-driven approaches. While a few knowledge infusion approaches have been proposed for HAR, they rely on rigid logic formalisms that do not take into account uncertainty. In this paper, we propose P-NIMBUS, a novel knowledge infusion approach for sensor-based HAR that relies on probabilistic reasoning. A probabilistic ontology is in charge of computing symbolic features that are combined with the features automatically extracted by a CNN model from raw sensor data and high-level context data. In particular, the symbolic features encode probabilistic common-sense knowledge about the activities consistent with the user's surrounding context. These features are infused within the model before the classification layer. We experimentally evaluated P-NIMBUS on a HAR dataset of mobile devices sensor data that includes 14 different activities performed by 25 users. Our results show that P-NIMBUS outperforms state-of-the-art neuro-symbolic approaches, with the advantage of requiring a limited amount of training data to reach satisfying recognition rates (i.e., more than 80% of F1-score with only 20% of labeled data). [ABSTRACT FROM AUTHOR]

As the roles of robots continue to expand in general, there is an increasing demand for research on automated task planning for a multi-agent system that can independently execute tasks in a wide and dynamic environment. This study introduces a plugin framework in which multiple robots can be involved in task planning in a broad range of areas by combining symbolic and connectionist approaches. The symbolic approach for understanding and learning human knowledge is useful for task planning in a wide and static environment. The network-based connectionist approach has the advantage of being able to respond to an ever-changing dynamic environment. A planning domain definition language-based planning algorithm, which is a symbolic approach, and the cooperative–competitive reinforcement learning algorithm, which is a connectionist approach, were utilized in this study. The proposed architecture is verified through a simulation. It is also verified through an experiment using 10 unmanned surface vehicles that the given tasks were successfully executed in a wide and dynamic environment. [ABSTRACT FROM AUTHOR]