Search

Identifying bio-signals based-sleep stages requires time-consuming and tedious labor of skilled clinicians. Deep learning approaches have been introduced in order to challenge the automatic sleep stage classification conundrum. However, the difficulties can be posed in replacing the clinicians with the automatic system due to the differences in many aspects found in individual bio-signals, causing the inconsistency in the performance of the model on every incoming individual. Thus, we aim to explore the feasibility of using a novel approach, capable of assisting the clinicians and lessening the workload. We propose the transfer learning framework, entitled MetaSleepLearner, based on Model Agnostic Meta-Learning (MAML), in order to transfer the acquired sleep staging knowledge from a large dataset to new individual subjects. The framework was demonstrated to require the labelling of only a few sleep epochs by the clinicians and allow the remainder to be handled by the system. Layer-wise Relevance Propagation (LRP) was also applied to understand the learning course of our approach. In all acquired datasets, in comparison to the conventional approach, MetaSleepLearner achieved a range of 5.4\% to 17.7\% improvement with statistical difference in the mean of both approaches. The illustration of the model interpretation after the adaptation to each subject also confirmed that the performance was directed towards reasonable learning. MetaSleepLearner outperformed the conventional approaches as a result from the fine-tuning using the recordings of both healthy subjects and patients. This is the first work that investigated a non-conventional pre-training method, MAML, resulting in a possibility for human-machine collaboration in sleep stage classification and easing the burden of the clinicians in labelling the sleep stages through only several epochs rather than an entire recording., Comment: IEEE Journal of Biomedical and Health Informatics (Accepted) (source code is available at https://github.com/IoBT-VISTEC/MetaSleepLearner)

Adaptive systems enable legged robots to cope with a wide range of environmental settings and unforeseen events. Existing reactive methods adapt either the walking frequency or the amplitude to only simple perturbations. This letter proposes an adaptive mechanism for central pattern generator (CPG)-based locomotion control that online-reacts to both internal and external soft constraints by adapting both the frequency and amplitude of driving signals. Our approach, namely GRAdient-Based shape adaptive control (GRAB), utilises real-time sensory signals for adapting the dynamics of the CPG. GRAB reacts to locomotion soft constraints given in a loss function. It can quickly adapt CPG’s dynamics variables to reduce such a loss, with a gradient-descent-like update step. The update perturbs the shape of the driving signal, which implicitly changes both frequency and amplitude of the robot locomotion pattern. We test the GRAB mechanism on a hexapod robot and its simulation, where we demonstrate its several benefits over a state-of-the-art adaptive control baseline. First, we show that it can be used for reducing the tracking error by simultaneously changing the walking amplitude and frequency. Also, GRAB can be used for limiting the maximum torque/current, preventing motor damage from unexpected perturbations. Finally, we demonstrate how GRAB can be utilised to naturally adjust the robot’s walking speed while taking into account multiple constraints, including target walking speed, external weight perturbations, and the robot’s physical limit.

One of the main concerns of deep reinforcement learning (DRL) is the data inefficiency problem, which stems both from an inability to fully utilize data acquired and from naive exploration strategies. In order to alleviate these problems, we propose a DRL algorithm that aims to improve data efficiency via both the utilization of unrewarded experiences and the exploration strategy by combining ideas from unsupervised auxiliary tasks, intrinsic motivation, and hierarchical reinforcement learning (HRL). Our method is based on a simple HRL architecture with a metacontroller and a subcontroller. The subcontroller is intrinsically motivated by the metacontroller to learn to control aspects of the environment, with the intention of giving the agent: 1) a neural representation that is generically useful for tasks that involve manipulation of the environment and 2) the ability to explore the environment in a temporally extended manner through the control of the metacontroller. In this way, we reinterpret the notion of pixel- and feature-control auxiliary tasks as reusable skills that can be learned via an intrinsic reward. We evaluate our method on a number of Atari 2600 games. We found that it outperforms the baseline in several environments and significantly improves performance in one of the hardest games—Montezuma’s revenge—for which the ability to utilize sparse data is key. We found that the inclusion of intrinsic reward is crucial for the improvement in the performance and that most of the benefit seems to be derived from the representations learned during training.

In most contemporary work in deep reinforcement learning (DRL), agents are trained in simulated environments. Not only are simulated environments fast and inexpensive, they are also ‘safe’. By contrast, training in a real world environment (using robots, for example) is not only slow and costly, but actions can also result in irreversible damage, either to the environment or to the agent (robot) itself. In this paper, we consider taking advantage of the inherent safety in computer simulation by extending the Deep Q-Network (DQN) algorithm with an ability to measure and take risk. In essence, we propose a novel DRL algorithm that encourages risk-seeking behaviour to enhance information acquisition during training. We demonstrate the merit of the exploration heuristic by (i) arguing that our risk estimator implicitly contains both parametric uncertainty and inherent uncertainty of the environment which are propagated back through Temporal Difference error across many time steps and (ii) evaluating our method on three games in the Atari domain and showing that the technique works well on Montezuma’s Revenge, a game that epitomises the challenge of sparse reward.