Search

Artifical neural networks (ANNs) have made their way into marine robotics in the last years, where they are used in control and perception systems, to name a few examples. At the same time, the black-box nature of ANNs is responsible for key challenges related to interpretability and trustworthiness, which need to be addressed if ANNs are to be deployed safely in real-life operations. In this paper, we implement three XAI methods to provide explanations to the decisions made by a deep reinforcement learning agent: Kernel SHAP, LIME and Linear Model Trees (LMTs). The agent was trained via Proximal Policy Optimization (PPO) to perform automatic docking on a fully-actuated vessel. We discuss the properties and suitability of the three methods, and juxtapose them with important attributes of the docking agent to provide context to the explanations.

Deep neural networks (DNNs) can be useful within the marine robotics field, but their utility value is restricted by their black-box nature. Explainable artificial intelligence methods attempt to understand how such black-boxes make their decisions. In this work, linear model trees (LMTs) are used to approximate the DNN controlling an autonomous surface vessel (ASV) in a simulated environment and then run in parallel with the DNN to give explanations in the form of feature attributions in real-time. How well a model can be understood depends not only on the explanation itself, but also on how well it is presented and adapted to the receiver of said explanation. Different end-users may need both different types of explanations, as well as different representations of these. The main contributions of this work are (1) significantly improving both the accuracy and the build time of a greedy approach for building LMTs by introducing ordering of features in the splitting of the tree, (2) giving an overview of the characteristics of the seafarer/operator and the developer as two different end-users of the agent and receiver of the explanations, and (3) suggesting a visualization of the docking agent, the environment, and the feature attributions given by the LMT for when the developer is the end-user of the system, and another visualization for when the seafarer or operator is the end-user, based on their different characteristics.

Autonomous cyber-physical systems, CPS, in dynamic environments must work impeccably. The cyber-physical systems must handle tasks consistently and trustworthily, i.e., with a robust behavior. Robust systems, in general, require making valid and solid decisions using one or a combination of robust reasoning strategies, algorithms, and robustness analysis. However, in dynamic environments, data can be incomplete, skewed, contradictory, and redundant impacting the reasoning. Basing decisions on these data can lead to inconsistent, irrational, and unreasonable cyber-physical systems’ movements, adversely impacting the system’s reliability and integrity. This paper presents the assessment of robust reasoning for autonomous cyber-physical systems in dynamic environments. In this work, robust reasoning is considered as 1) the capability of drawing conclusions with available data by applying classical and non-classical reasoning strategies and algorithms and 2) act and react robustly and safely in dynamic environments by employing robustness analysis to provide options on possible actions and evaluate alternative decisions. The result of the research shows that different common existing strategies, algorithms and analyses can be provided together with a comparison of their applicabilities, benefits, and drawbacks in the context of cyber-physical systems operating in dynamically changing environments. The conclusion is that robust reasoning in cyber-physical systems can handle dynamic environments. Moreover, combining these strategies and algorithms with robustness analysis can support achieving robust behavior in autonomous cyber-physical systems while operating in dynamically changing environments.