Search

Small-scale legged robots have found widespread utilization in various industrial and biomedical applications due to their compact size and superior locomotion capabilities. Reducing the number of actuators is often desirable to decrease the robot's size and weight, which comes at the expense of the robot's workspace. Our study proposes a method to enhance the mobility of small-scale legged robots with limited degrees of actuators (DoAs) by co-optimizing both morphology parameters and control policy. The co-optimization is formulated as a bi-level optimization problem, where the control policy is designed using deep reinforcement learning algorithms and central pattern generators (CPGs) at the lower level. The inclusion of CPGs significantly speeds up training and enables the application of simulation results in real-world scenarios. At the upper level, morphology optimization is achieved through Bayesian optimization based on dual-networks. This approach eliminates the need to train a policy for each morphology candidate from scratch, leveraging previous experience to enhance efficiency. Through simulation and physical experiments, the effectiveness of our proposed approach is demonstrated, showcasing its ability to discover optimal morphology and gait for small-scale legged robots with limited DoAs. These findings have potential long-term impacts on small-scale legged robot design and locomotion control.

In this article, an optimal output synchronization solution to the $H_{\infty }$ optimization of linear discrete-time (DT) multiagent systems is investigated. Compared with current approaches, the issue of designated convergence rate is handled with system optimality, while less computation cost is required. Specifically, the internal model principle is employed to derive a cooperative regulation problem of DT systems, wherein no explicit solution to output regulation equations is needed for learning. Then, we introduce a convergence rate parameter to construct a group of auxiliary cooperative systems, based on which the zero-sum game in $H_{\infty }$ optimization is formulated. The data-efficient off-policy reinforcement learning and output-feedback technique are applied to solve the enhanced Bellman equations with a designated convergence rate. This results in an online optimal synchronization solution learning from only the input-output data along the system trajectories. It is shown that the proposed optimal synchronization protocol achieves asymptotic synchronization for the original systems with the consensus error converging to zero at a designated rate. The effectiveness of the proposed approach is verified by the simulation results.

Deep reinforcement learning (DRL) has been applied to satellite navigation and positioning applications. Its performance relies heavily on the function-approximation capability of deep neural networks. However, existing DRL models suffer from catastrophic interference, resulting in inaccurate function approximation. The sparse-coding-based DRL is an effective method to mitigating this interference, but existing methods involve the following two challenging issues: first, the value function estimation network suffers from instability problems with gradient backpropagation, including gradient explosion and gradient vanishing, second, existing methods are limited to using hand-crafted sparse regularizers that produce only static sparsity, which may be difficult to apply in various dynamic reinforcement learning (RL) environments. In this article, we propose a novel dictionary learning (DL)-structured RL model with adaptive-sparsity regularizer (ASR) that alleviates the catastrophic interference and enables accurate value function approximation, thereby improving the RL performance. To alleviate the interference and avoid the instability problems in RL, a feedforward DL-structured RL model is constructed to predict the value function without the need for gradient backpropagation. To learn data-driven sparse representations with adaptive sparsity, we propose to use the learnable sparse regularizer ASR in the model, where the key hyperparameters of ASR can be trained to be adaptive to variable RL environments. To optimize the model efficiently, the model parameters are first pretrained in the pretraining stage, with only the value weights used for value function approximation needing to be fine-tuned for actual RL applications in the control training stage. Our comparative experiments in benchmark environments demonstrate that the proposed method can outperform existing state-of-the-art sparse-coding-based RL algorithms. In terms of accumulated rewards (used to measure the quality of the learned policy), the improvement was over 63% in Cart Pole environment and nearly 10% for Puddle World. Furthermore, the proposed algorithm can maintain its relatively high performance in the presence of noise up to 20 dB.

As one efficient technique in reinforcement learning, policy iteration (PI) requires an initial admissible (or stabilizing for linear systems) control policy that renders the existing PI-based results to be model dependent. To attain a completely data-driven adaptive optimal control, this article suggests integrating a homotopic design with PI for unknown continuous-time nonlinear systems. Technically, we leverage a homotopic constant to construct an artificially stable system that allows zero control to initialize PI. Utilizing a homotopic strategy, we recursively update the artificial system and then enforce it to gradually recover the original system. This ultimately allows us to obtain an admissible control policy in a finite number of iterations without carrying out a model-based initialization. Once the admissible control is obtained, the proposed homotopic PI inherits fast convergence from the traditional PI technique and ensures learning the optimal control solution from the data measured from unknown nonlinear systems.

This article focuses on the problem of adaptive optimal tracking control of interconnected switched systems on dwell time. The objective is to establish interconnection between switched systems, stabilize each switched system, and achieve optimal performance over the dwell time. An actor-critic learning technique is utilized to design the optimal controller by evaluating the performance via the critic and improving the policy via the actor. During the design of the cost function and optimization process, we employed nonzero-sum game theory to incorporate the interconnection factors among systems. The proposed control scheme designs a novel relaxed stability condition for switched subsystems and validates the effectiveness by using a Lyapunov function with a value function. Finally, the designed control scheme can be verified by simulation.

This work proposes a hybrid two-stage analog-to-digital converter (ADC) consisting of a coarse 3.3-V successive-approximation register (SAR) ADC and a fine 1.8-V comparator-based (CB) incremental $\Delta \Sigma $ modulator (ISDM) ADC. The coarse ADC boosts the maximum input amplitude to improve signal-to-noise and distortion ratio (SNDR), and the fine ISDM ADC adopts an asynchronous operation and a time-domain noise-slicing (TDNS) technique to improve power efficiency and conversion speed. Using the unequal data weighting of ISDM and the noise–slope relationship of the CB amplifier, the TDNS technique reconstructs the noise and power weighting of each cycle to optimize the figure of merit (FoM). Furthermore, the noise of the CB amplifier is suppressed by the noise-shaping (NS) technique. The ADC operates at a 370-kHz frequency and achieves 94.5/89.3-dB dynamic range (DR)/SNDR in a 185-kHz bandwidth (BW). It achieves a Schreier FoM of 174.88 dB and consumes 511 $\mu \text{W}$ .