Search

This article proposes a novel decision-making framework that bridges a gap between model-based (MB) and model-free (MF) control processes through only adjusting the planning horizon. Specifically, the output policy is obtained by solving a model predictive control problem with a locally optimal state value as terminal constraints. When the planning horizon decreases to zero, the MB control will transform into the MF control smoothly. Meanwhile, inspired by the neural mechanism of emotion modulation on decision-making, we build a biologically plausible computational model of emotion processing. This model can generate an uncertainty-related emotional response on the basis of the state prediction error and reward prediction error, and then dynamically modulates the planning horizon in the tasks. The simulation results demonstrate that the proposed decision-making framework can produce better policies than traditional methods. Emotion modulation can shift the MB and MF control well to improve the learning efficiency and the speed of decision-making. [ABSTRACT FROM AUTHOR]

Owing to its potential superiorities in terms of flexibility, compliance, and robustness, the musculoskeletal robotic system has become a promising direction for next-generation robots. However, motion learning and generalization of musculoskeletal systems are still challenging problems. In this article, a muscle-synergies-based neuromuscular control is proposed. First, a new computational model of time-varying muscle synergies is constructed, which utilizes both phasic and tonic muscle synergies to characterize the basic features of muscle excitations more sufficiently. Second, a novel neuromuscular control method is proposed for realizing the motion learning and generalization of musculoskeletal systems. Therein, a radial basis function (RBF) neural network is designed to modulate muscle synergies according to different movement targets. Muscle excitations are computed with the combination of modulated muscle synergies. Covariance matrix adaptation evolutionary strategy is applied to realize the synchronous optimization of muscle synergies and the RBF neural network. In the experiment, a sophisticated musculoskeletal system learns to perform center-out reaching tasks through trial-and-error learning on a few targets. With the muscle synergies and neural modulation learned from a few targets, the musculoskeletal system can also reach many unexperienced targets. The proposed method not only improves the speed and accuracy of motion learning but also enhances motion generalization. This article also promotes the development of the musculoskeletal robotic system and the fusion of neuroscience and robotics. [ABSTRACT FROM AUTHOR]

Policy search in reinforcement learning (RL) is a practical approach to interact directly with environments in parameter spaces, that often deal with dilemmas of local optima and real-time sample collection. A promising algorithm, known as guided policy search (GPS), is capable of handling the challenge of training samples using trajectory-centric methods. It can also provide asymptotic local convergence guarantees. However, in its current form, the GPS algorithm cannot operate in sequential multitask learning scenarios. This is due to its batch-style training requirement, where all training samples are collectively provided at the start of the learning process. The algorithm’s adaptation is thus hindered for real-time applications, where training samples or tasks can arrive randomly. In this paper, the GPS approach is reformulated, by adapting a recently proposed, lifelong-learning method, and elastic weight consolidation. Specifically, Fisher information is incorporated to impart knowledge from previously learned tasks. The proposed algorithm, termed sequential multitask learning-GPS, is able to operate in sequential multitask learning settings and ensuring continuous policy learning, without catastrophic forgetting. Pendulum and robotic manipulation experiments demonstrate the new algorithms efficacy to learn control policies for handling sequentially arriving training samples, delivering comparable performance to the traditional, and batch-based GPS algorithm. In conclusion, the proposed algorithm is posited as a new benchmark for the real-time RL and robotics research community. [ABSTRACT FROM AUTHOR]