Search

We tackle the recently introduced benchmark for whole-body humanoid control HumanoidBench using MuJoCo MPC. We find that sparse reward functions of HumanoidBench yield undesirable and unrealistic behaviors when optimized; therefore, we propose a set of regularization terms that stabilize the robot behavior across tasks. Current evaluations on a subset of tasks demonstrate that our proposed reward function allows achieving the highest HumanoidBench scores while maintaining realistic posture and smooth control signals. Our code is publicly available and will become a part of MuJoCo MPC, enabling rapid prototyping of robot behaviors., Comment: 3 pages, 3 figures, submitted to IEEE Conference on Robotics and Automation (ICRA@40)

Low-fidelity analytical models of turbine wakes have traditionally been used for wind farm planning, performance evaluation, and demonstrating the utility of advanced control algorithms in increasing the annual energy production. In practice, however, it remains challenging to correctly estimate the flow and achieve significant performance gains using conventional low-fidelity models. This is due to the over-simplified static nature of wake predictions from models that are agnostic to the effects of atmospheric boundary layer turbulence and the complex aerodynamic interactions among wind turbines. To improve the predictive capability of low-fidelity models while remaining amenable to control design, we offer a stochastic dynamical modeling framework for capturing the effect of atmospheric turbulence on the thrust force and power generation as determined by the actuator disk concept. In this approach, we use stochastically forced linear models of the turbulent velocity field to augment the analytically computed wake velocity and achieve consistency with higher-fidelity models in capturing power and thrust force measurements. The power-spectral densities of our stochastic models are identified via convex optimization to ensure consistency with partially available velocity statistics or power and thrust force measurements while preserving model parsimony. We demonstrate the utility of our approach in capturing turbulence intensity variations behind wind turbines and estimating thrust force and power signals generated by large-eddy simulations of the flow over a cascade of turbines. Our results provide insight into the significance of sparse field measurements in recovering the statistical signature of wind farm turbulence using stochastic linear models., Comment: 21 pages, 11 figures

We present in-hand manipulation skills on a dexterous, compliant, anthropomorphic hand. Even though these skills were derived in a simplistic manner, they exhibit surprising robustness to variations in shape, size, weight, and placement of the manipulated object. They are also very insensitive to variation of execution speeds, ranging from highly dynamic to quasi-static. The robustness of the skills leads to compositional properties that enable extended and robust manipulation programs. To explain the surprising robustness of the in-hand manipulation skills, we performed a detailed, empirical analysis of the skills' performance. From this analysis, we identify three principles for skill design: 1) Exploiting the hardware's innate ability to drive hard-to-model contact dynamics. 2) Taking actions to constrain these interactions, funneling the system into a narrow set of possibilities. 3) Composing such action sequences into complex manipulation programs. We believe that these principles constitute an important foundation for robust robotic in-hand manipulation, and possibly for manipulation in general., Comment: Published in Robotics: Science and Systems 2021. Proceedings at http://www.roboticsproceedings.org/rss17/p089.html Spotlight talk at https://youtu.be/2vwdP4WjGoQ Complete video playlist at https://www.youtube.com/playlist?list=PLb-CNILz7vmt6Ae_yD9i15TrCw0S8bKCn

Sample efficiency is a crucial problem in deep reinforcement learning. Recent algorithms, such as REDQ and DroQ, found a way to improve the sample efficiency by increasing the update-to-data (UTD) ratio to 20 gradient update steps on the critic per environment sample. However, this comes at the expense of a greatly increased computational cost. To reduce this computational burden, we introduce CrossQ: A lightweight algorithm for continuous control tasks that makes careful use of Batch Normalization and removes target networks to surpass the current state-of-the-art in sample efficiency while maintaining a low UTD ratio of 1. Notably, CrossQ does not rely on advanced bias-reduction schemes used in current methods. CrossQ's contributions are threefold: (1) it matches or surpasses current state-of-the-art methods in terms of sample efficiency, (2) it substantially reduces the computational cost compared to REDQ and DroQ, (3) it is easy to implement, requiring just a few lines of code on top of SAC., Comment: Published at ICLR 2024. Project page at http://aditya.bhatts.org/CrossQ and code release at https://github.com/adityab/CrossQ

In the NeurIPS 2018 Artificial Intelligence for Prosthetics challenge, participants were tasked with building a controller for a musculoskeletal model with a goal of matching a given time-varying velocity vector. Top participants were invited to describe their algorithms. In this work, we describe the challenge and present thirteen solutions that used deep reinforcement learning approaches. Many solutions use similar relaxations and heuristics, such as reward shaping, frame skipping, discretization of the action space, symmetry, and policy blending. However, each team implemented different modifications of the known algorithms by, for example, dividing the task into subtasks, learning low-level control, or by incorporating expert knowledge and using imitation learning.