Search

People spend a substantial part of their lives at rest in bed. 3D human pose and shape estimation for this activity would have numerous beneficial applications, yet line-of-sight perception is complicated by occlusion from bedding. Pressure sensing mats are a promising alternative, but training data is challenging to collect at scale. We describe a physics-based method that simulates human bodies at rest in a bed with a pressure sensing mat, and present PressurePose, a synthetic dataset with 206K pressure images with 3D human poses and shapes. We also present PressureNet, a deep learning model that estimates human pose and shape given a pressure image and gender. PressureNet incorporates a pressure map reconstruction (PMR) network that models pressure image generation to promote consistency between estimated 3D body models and pressure image input. In our evaluations, PressureNet performed well with real data from participants in diverse poses, even though it had only been trained with synthetic data. When we ablated the PMR network, performance dropped substantially., Comment: 18 pages, 18 figures, 5 tables. Accepted for oral presentation at CVPR 2020

Fall prevention is one of the most important components in senior care. We present a technique to augment an assistive walking device with the ability to prevent falls. Given an existing walking device, our method develops a fall predictor and a recovery policy by utilizing the onboard sensors and actuators. The key component of our method is a robust human walking policy that models realistic human gait under a moderate level of perturbations. We use this human walking policy to provide training data for the fall predictor, as well as to teach the recovery policy how to best modify the person’s gait when a fall is imminent. Our evaluation shows that the human walking policy generates walking sequences similar to those reported in biomechanics literature. Our experiments in simulation show that the augmented assistive device can indeed help recover balance from a variety of external perturbations. We also provide a quantitative method to evaluate the design choices for an assistive device.

We present a new approach for transfer of dynamic robot control policies such as biped locomotion from simulation to real hardware. Key to our approach is to perform system identification of the model parameters {\mu} of the hardware (e.g. friction, center-of-mass) in two distinct stages, before policy learning (pre-sysID) and after policy learning (post-sysID). Pre-sysID begins by collecting trajectories from the physical hardware based on a set of generic motion sequences. Because the trajectories may not be related to the task of interest, presysID does not attempt to accurately identify the true value of {\mu}, but only to approximate the range of {\mu} to guide the policy learning. Next, a Projected Universal Policy (PUP) is created by simultaneously training a network that projects {\mu} to a low-dimensional latent variable {\eta} and a family of policies that are conditioned on {\eta}. The second round of system identification (post-sysID) is then carried out by deploying the PUP on the robot hardware using task-relevant trajectories. We use Bayesian Optimization to determine the values for {\eta} that optimizes the performance of PUP on the real hardware. We have used this approach to create three successful biped locomotion controllers (walk forward, walk backwards, walk sideways) on the Darwin OP2 robot., Comment: International Conference on Intelligent Robots and Systems (IROS), 2019

Modeling realistic human joint limits is important for applications involving physical human-robot interaction. However, setting appropriate human joint limits is challenging because it is pose-dependent: the range of joint motion varies depending on the positions of other bones. The paper introduces a new technique to accurately simulate human joint limits in physics simulation. We propose to learn an implicit equation to represent the boundary of valid human joint configurations from real human data. The function in the implicit equation is represented by a fully connected neural network whose gradients can be efficiently computed via back-propagation. Using gradients, we can efficiently enforce realistic human joint limits through constraint forces in a physics engine or as constraints in an optimization problem.

Differential dynamic programming (DDP) is a widely used trajectory optimization technique that addresses nonlinear optimal control problems, and can readily handle nonlinear cost functions. However, it does not handle either state or control constraints. This paper presents a novel formulation of DDP that is able to accommodate arbitrary nonlinear inequality constraints on both state and control. The main insight in standard DDP is that a quadratic approximation of the value function can be derived using a recursive backward pass, however the recursive formulae are only valid for unconstrained problems. The main technical contribution of the presented method is a derivation of the recursive quadratic approximation formula in the presence of nonlinear constraints, after a set of active constraints has been identified at each point in time. This formula is used in a new Constrained-DDP (CDDP) algorithm that iteratively determines these active set and is guaranteed to converge toward a local minimum. CDDP is demonstrated on several underactuated optimal control problems up to 12D with obstacle avoidance and control constraints and is shown to outperform other methods for accommodating constraints.

During robot-assisted dressing, a robot manipulates a garment in contact with a person's body. Inferring the forces applied to the person's body by the garment might enable a robot to provide more effective assistance and give the robot insight into what the person feels. However, complex mechanics govern the relationship between the robot's end effector and these forces. Using a physics-based simulation and data-driven methods, we demonstrate the feasibility of inferring forces across a person's body using only end effector measurements. Specifically, we present a long short-term memory (LSTM) network that at each time step takes a 9-dimensional input vector of force, torque, and velocity measurements from the robot's end effector and outputs a force map consisting of hundreds of inferred force magnitudes across the person's body. We trained and evaluated LSTMs on two tasks: pulling a hospital gown onto an arm and pulhng shorts onto a leg. For both tasks, the LSTMs produced force maps that were similar to ground truth when visualized as heat maps across the limbs. We also evaluated their performance in terms of root-mean-square error. Their performance degraded when the end effector velocity was increased outside the training range, but generalized well to limb rotations. Overall, our results suggest that robots could learn to infer the forces people feel during robot-assisted dressing, although the extent to which this will generalize to the real world remains an open question.

Traversing environments with arbitrary obstacles poses significant challenges for bipedal robots. In some cases, whole body motions may be necessary to maneuver around an obstacle, but most existing footstep planners can only select from a discrete set of predetermined footstep actions; they are unable to utilize the continuum of whole body motion that is truly available to the robot platform. Existing motion planners that can utilize whole body motion tend to struggle with the complexity of large-scale problems. We introduce a planning method, called the "Randomized Possibility Graph", which uses high-level approximations of constraint manifolds to rapidly explore the "possibility" of actions, thereby allowing lower-level motion planners to be utilized more efficiently. We demonstrate simulations of the method working in a variety of semi-unstructured environments. In this context, "semi-unstructured" means the walkable terrain is flat and even, but there are arbitrary 3D obstacles throughout the environment which may need to be stepped over or maneuvered around using whole body motions., Comment: Accepted by IEEE International Conference on Robotics and Automation 2017

Model-based trajectory optimization often fails to find a reference trajectory for under-actuated bipedal robots performing highly-dynamic, contact-rich tasks in the real world due to inaccurate physical models. In this paper, we propose a complete system that automatically designs a reference trajectory that succeeds on tasks in the real world with a very small number of real world experiments. We adopt existing system identification techniques and show that, with appropriate model parameterization and control optimization, an iterative system identification framework can be effective for designing reference trajectories. We focus on a set of tasks that leverage the momentum transfer strategy to rapidly change the whole-body from an initial configuration to a target configuration by generating large accelerations at the center of mass and switching contacts.

Dressing is an important activity of daily living (ADL) with which many people require assistance due to impairments. Robots have the potential to provide dressing assistance, but physical interactions between clothing and the human body can be complex and difficult to visually observe. We provide evidence that data-driven haptic perception can be used to infer relationships between clothing and the human body during robot-assisted dressing. We conducted a carefully controlled experiment with 12 human participants during which a robot pulled a hospital gown along the length of each person's forearm 30 times. This representative task resulted in one of the following three outcomes: the hand missed the opening to the sleeve; the hand or forearm became caught on the sleeve; or the full forearm successfully entered the sleeve. We found that hidden Markov models (HMMs) using only forces measured at the robot's end effector classified these outcomes with high accuracy. The HMMs' performance generalized well to participants (98.61% accuracy) and velocities (98.61% accuracy) outside of the training data. They also performed well when we limited the force applied by the robot (95.8% accuracy with a 2N threshold), and could predict the outcome early in the process. Despite the lightweight hospital gown, HMMs that used forces in the direction of gravity substantially outperformed those that did not. The best performing HMMs used forces in the direction of motion and the direction of gravity.

Learning a parameterized skill is essential for autonomous robots operating in an unpredictable environment. Previous techniques learned a policy for each example task individually and constructed a regression model to map between task and policy parameter spaces. However, these techniques have less success when applied to whole-body dynamic skills, such as jumping or walking, which involve the challenges of handling discrete contacts and balancing an under-actuated system under gravity. This paper introduces an evolutionary optimization algorithm for learning parameterized skills to achieve whole-body dynamic tasks. Our algorithm simultaneously learns policies for a range of tasks instead of learning each policy individually. The problem can be formulated as a nonconvex optimization whose solution is a closed segment of curve instead of a point in the policy parameter space. We develop a new optimization algorithm which maintains a parameterized probability distribution for the entire range of tasks and iteratively updates the distribution using selected elite samples. Our algorithm is able to better exploit each sample, greatly reducing the number of samples required to optimize a parameterized skill for all the tasks in the range of interest.

This paper introduces a new planning algorithm to minimize the damage of humanoid falls by utilizing multiple contact points. Given an unstable initial state of the robot, our approach plans for the optimal sequence of contact points such that the initial momentum is dissipated with minimal impacts on the robot. Instead of switching among a collection of individual control strategies, we propose a general algorithm which plans for appropriate responses to a wide variety of falls, from a single step to recover a gentle nudge, to a rolling motion to break a high-speed fall. Our algorithm transforms the falling problem into a sequence of inverted pendulum problems and use dynamic programming to solve the optimization efficiently. The planning algorithm is validated in physics simulation and experimentally tested on a BioloidGP humanoid.

With no initial angular momentum an ordinary house cat is capable of flipping over onto its feet in mid-air and landing safely after a fall. As the field of robotics advances and robots become more dynamic, control algorithms for landing safely from a long, intended fall will become more necessary. Here we present an algorithm that leverages nonholonomic trajectory planning inspired by the falling cat to orient an articulated robot through configuration changes to achieve a pose that reduces the impact at landing. The calculated impact pose results in minimal loss of energy through rolling, while maximizing the rolling time. In addition to orienting and rolling, our controller guides the system to behave like a damped spring- mass system to reduce the magnitude of contact forces. Our framework is general and is applicable to systems that can be modeled as a connected tree of rigid bodies. We illustrate the feasibility of the algorithm through simulation and physical experiments with a planar three-link robot. I. INTRODUCTION When falling, an ordinary house cat has the remarkable ability to twist its body in mid-air to land on its feet. This behavior is made more extraordinary by the fact that the cat is able to do this without any initial angular velocity and using no external forces to orient itself. Despite the consid- erable advances in robotics, this feat of agility is beyond the capability of current hardware platforms. However, as articulated robots advance and become more dynamic, it will be necessary to have at least a modest ability to properly survive the impact of a fall. Here we present an algorithm that leverages nonholonomic trajectory planning inspired by the falling cat to orient an articulated robot by configuration changes to achieve a final pose that rolls and reduces the impact during landing. We define a pose as the combination of configuration (joint state) and orientation (attitude of the root body with respect to the inertial frame). Beyond orienting we also address the impact of landing, because, as the saying goes, "It's not the fall that kills you. It's the sudden stop at the end." The key word is sudden in this aphorism. The damage incurred by an impact with the ground is largely due to the magnitude of the impulse required to change the momentum of the falling body. Therefore, we suggest two strategies to reduce the detrimental effects of "the sudden stop at the end." First, perform a rolling action to reduce the change in momentum caused by the impact. Second, lengthen the duration of the impulse by acting like a damped spring-mass system, which decreases the magnitude of the impulse force at peak. To achieve these two goals on an articulated robot, we propose algorithms that compute the optimal impact pose and a set of joint stiffnesses to reduce the change in momentum and peak contact force during landing. In this work we provide a theoretical framework for find- ing an impact pose and joint stiffness that reduce the impact impulse, as well as the sequence of configurations to orient an articulated robot to the impact pose during falling. The theoretical framework is general and should be applicable to any system that can be modeled as a connected tree of rigid bodies. Next, we present an application of this framework to a planar three-link robot with implementation details. Finally, we present simulation and experimental results to show the feasibility of our control algorithm.

Social robots can benefit from motion variance because non-repetitive gestures will be more natural and intuitive for human partners. We introduce a new approach for synthesizing variance, both with and without constraints, using a stochastic process. Based on optimal control theory and operational space control, our method can generate an infinite number of variations in real-time that resemble the kinematic and dynamic characteristics from the single input motion sequence. We also introduce a stochastic method to generate smooth but nondeterministic transitions between arbitrary motion variants. Furthermore, we quantitatively evaluate task-aware variance against random white torque noise, operational space control, style-based inverse kinematics, and retargeted human motion to prove that task-aware variance generates human-like motion. Finally, we demonstrate the ability of task-aware variance to maintain velocity and time-dependent features that exist in the input motion.

Secondary action, a concept borrowed from character animation, improves the animation realism by augmenting natural, passive motion to primary action. We use dynamic simulation to induce three techniques of secondary motion for robot hardware, which exploit actuation passivity to overcome hardware constraints and change the dynamic perception of the robot and its motion characteristics. Results of secondary motion due to internal and external forces are presented including discussion on how to choose the appropriate technique for a particular application.

Stylized motion is prevalent in the field of Human-Robot Interaction (HRI). Robot designers typically hand craft or work with professional animators to design behaviors for a robot that will be communicative or life-like when interacting with a human partner. A challenge is to apply this stylized trajectory in varied contexts (e.g. performing a stylized gesture with different end-effector constraints). The goal of this research is to create useful, task-based motion with variance that spans the reachable space of the robot and satisfies constraints, while preserving the “style” of the original motion. We claim the appropriate representation for adapting and generalizing a trajectory is not in Cartesian or joint angle space, but rather in joint velocity space, which allows for unspecified initial conditions to be supplied by interaction with the dynamic environment. The benefit of this representation is that a single trajectory can be extended to accomplish similar tasks in the world given constraints in the environment. We present quantitative data using a continuity metric to prove that, given a stylized initial trajectory, we can create smoother generalized motion than with traditional techniques such as cyclic-coordinate descent.