Search

A conventional adopted method for operating a waiter robot is based on the static position control, where predefined goal positions are marked on a map. However, this solution is not optimal in a dynamic setting, such as in a coffee shop or an outdoor catering event, because the customers often change their positions. This article explores an alternative human-robot interface design where a human operator communicates the identity of the customer to the robot instead. Inspired by how human communicates, we propose a framework for communicating a visual goal to the robot, through interactive two-way communications. The framework exploits concepts from two machine learning domains: human-in-the-loop machine learning, where active learning is used to acquire informative data, and deep metric learning, where a suitable embedding can improve the learning ability of a classifier. We also propose novel class imbalance handling techniques, which aim to actively alleviate the class imbalance problem found to be important in this mode of communication. The framework is evaluated using publicly available pedestrian datasets. We demonstrate that the proposed framework can help reduce the number of required two-way interactions and increases the robustness of the predictive model. We successfully implement the framework on a mobile robot for a delivery service in a cafe-like environment. Through the online visual goal human-robot communication, the robot can detect, recognize, and autonomously navigate to the target customer.

Identifying bio-signals based-sleep stages requires time-consuming and tedious labor of skilled clinicians. Deep learning approaches have been introduced in order to challenge the automatic sleep stage classification conundrum. However, the difficulties can be posed in replacing the clinicians with the automatic system due to the differences in many aspects found in individual bio-signals, causing the inconsistency in the performance of the model on every incoming individual. Thus, we aim to explore the feasibility of using a novel approach, capable of assisting the clinicians and lessening the workload. We propose the transfer learning framework, entitled MetaSleepLearner, based on Model Agnostic Meta-Learning (MAML), in order to transfer the acquired sleep staging knowledge from a large dataset to new individual subjects. The framework was demonstrated to require the labelling of only a few sleep epochs by the clinicians and allow the remainder to be handled by the system. Layer-wise Relevance Propagation (LRP) was also applied to understand the learning course of our approach. In all acquired datasets, in comparison to the conventional approach, MetaSleepLearner achieved a range of 5.4\% to 17.7\% improvement with statistical difference in the mean of both approaches. The illustration of the model interpretation after the adaptation to each subject also confirmed that the performance was directed towards reasonable learning. MetaSleepLearner outperformed the conventional approaches as a result from the fine-tuning using the recordings of both healthy subjects and patients. This is the first work that investigated a non-conventional pre-training method, MAML, resulting in a possibility for human-machine collaboration in sleep stage classification and easing the burden of the clinicians in labelling the sleep stages through only several epochs rather than an entire recording., Comment: IEEE Journal of Biomedical and Health Informatics (Accepted) (source code is available at https://github.com/IoBT-VISTEC/MetaSleepLearner)

Objective: Advances in the motor imagery (MI)-based brain-computer interfaces (BCIs) allow control of several applications by decoding neurophysiological phenomena, which are usually recorded by electroencephalography (EEG) using a non-invasive technique. Despite significant advances in MI-based BCI, EEG rhythms are specific to a subject and various changes over time. These issues point to significant challenges to enhance the classification performance, especially in a subjectindependent manner. Methods: To overcome these challenges, we propose MIN2Net, a novel end-to-end multi-task learning to tackle this task. We integrate deep metric learning into a multi-task autoencoder to learn a compact and discriminative latent representation from EEG and perform classification simultaneously. Results: This approach reduces the complexity in pre-processing, results in significant performance improvement on EEG classification. Experimental results in a subject-independent manner show that MIN2Net outperforms the state-of-the-art techniques, achieving an F1-score improvement of 6.72% and 2.23% on the SMR-BCI and OpenBMI datasets, respectively. Conclusion: We demonstrate that MIN2Net improves discriminative information in the latent representation. Significance: This study indicates the possibility and practicality of using this model to develop MI-based BCI applications for new users without calibration. This work was supported in part by PTT Public Company Limited, in part by The SCB Public Company Limited, in part by Thailand Science Research, and Innovation under Grant SRI62W1501, in part by The Office of the Permanent Secretary of the Ministry of Higher Education, Science, Research, and Innovation, Thailand under Grant RGNS63-252, and in part by the National Research Council of Thailand under Grant N41A640131.

One of the main concerns of deep reinforcement learning (DRL) is the data inefficiency problem, which stems both from an inability to fully utilize data acquired and from naive exploration strategies. In order to alleviate these problems, we propose a DRL algorithm that aims to improve data efficiency via both the utilization of unrewarded experiences and the exploration strategy by combining ideas from unsupervised auxiliary tasks, intrinsic motivation, and hierarchical reinforcement learning (HRL). Our method is based on a simple HRL architecture with a metacontroller and a subcontroller. The subcontroller is intrinsically motivated by the metacontroller to learn to control aspects of the environment, with the intention of giving the agent: 1) a neural representation that is generically useful for tasks that involve manipulation of the environment and 2) the ability to explore the environment in a temporally extended manner through the control of the metacontroller. In this way, we reinterpret the notion of pixel- and feature-control auxiliary tasks as reusable skills that can be learned via an intrinsic reward. We evaluate our method on a number of Atari 2600 games. We found that it outperforms the baseline in several environments and significantly improves performance in one of the hardest games—Montezuma’s revenge—for which the ability to utilize sparse data is key. We found that the inclusion of intrinsic reward is crucial for the improvement in the performance and that most of the benefit seems to be derived from the representations learned during training.

This paper presents an integrated robotic platform for advanced collaborative robots and demonstrates an application of multiple robots collaboratively transporting an object to different positions in a factory environment. The proposed platform integrates a drone, a mobile manipulator robot, and a dual-arm robot to work autonomously, while also collaborating with a human worker.

Many CPG-based locomotion models have a problem known as the tracking error problem, where the mismatch between the CPG driving signal and the state of the robot can cause undesirable behaviours for legged robots. Towards alleviating this problem, we introduce a mechanism that modulates the CPG signal using the robot’s interoceptive information. The key concept is to generate a driving signal that is easier for the robot to follow, yet can drive the locomotion of the robot. This can be done by nudging the CPG signal in the direction of lower tracking error, which can be analytically calculated. Unlike other reactive CPG, the proposed method does not rely on any parametric learning ability to adjust the shape of the signal, making it a unique option for a biological adaptive motor control. Our experiment results show that the proposed method successfully reduces the tracking error. We also show that the CPG signal, regulated by the proposed method, is robust to perturbation and can smoothly return back to the default pattern.

Overfitting to learning partners is a known problem, in multi-agent reinforcement learning (MARL), due to the co-evolution of learning agents. Previous works explicitly add diversity to learning partners for mitigating this problem. However, since there are many approaches for introducing diversity, it is not clear which one should be used under what circumstances. In this work, we clarify the situation and reveal that widely used methods such as partner sampling and population-based training are unreliable at introducing diversity under fully cooperative multi-agent Markov decision process. We find that generating pre-trained partners is a simple yet effective procedure to achieve diversity. Finally, we highlight the impact of diversified learning partners on the generalization of learning agents using cross-play and ad-hoc team performance as evaluation metrics.

In most contemporary work in deep reinforcement learning (DRL), agents are trained in simulated environments. Not only are simulated environments fast and inexpensive, they are also ‘safe’. By contrast, training in a real world environment (using robots, for example) is not only slow and costly, but actions can also result in irreversible damage, either to the environment or to the agent (robot) itself. In this paper, we consider taking advantage of the inherent safety in computer simulation by extending the Deep Q-Network (DQN) algorithm with an ability to measure and take risk. In essence, we propose a novel DRL algorithm that encourages risk-seeking behaviour to enhance information acquisition during training. We demonstrate the merit of the exploration heuristic by (i) arguing that our risk estimator implicitly contains both parametric uncertainty and inherent uncertainty of the environment which are propagated back through Temporal Difference error across many time steps and (ii) evaluating our method on three games in the Atari domain and showing that the technique works well on Montezuma’s Revenge, a game that epitomises the challenge of sparse reward.