Search

Estimating human pose from images is the key to enabling machines to understand human actions. Existing works on human pose estimation mainly focus on designing more resultful deep neural networks to regress the locations of human joints. Although the human pose is obedient to anatomy and shows rich anatomical features, reasoning human body structure by machine in a complex environment is still an open problem. This paper proposes AnatPose which can effectively capture the structural dependency among human body parts by both deep neural network architecture and learning objectives: (1) For the deep neural network architecture, a bidirectional learning paradigm is proposed to learn body-part proportions and dependencies by organizing human body parts as sequential data. This innovation enables the messages to pass in a bidirectional way and makes the human body exchange information about each part deeper during training. (2) For the learning objective, the proposed AnatPose learns a probabilistic representation of multi-scale anatomical features, including keypoint heatmaps, bone heatmaps, and symmetry heatmaps. This innovation enables the multi-scale anatomical features to successfully capture the structural dependency at both low-level joints and high-level associations from the anatomical priors of the human body. Extensive experimental results demonstrate that the proposed AnatPose shows state-of-the-art performance on three challenging datasets. It achieves a PCK@0.2 detection rate of 95.2% on the LSP dataset, a PCKh@0.5 detection rate of 92.9% on the MPII dataset, and an mAP of 76.6% on the Microsoft COCO dataset. Benefiting from its state-of-the-art accuracy, the proposed approach is expected to be widely used in various human pose estimation-driven applications. • AnatPose organizes human body parts as anatomically sequential data. • AnatPose learns the anatomical properties by a bidirectional recurrent neural network. • An anatomically significant objective is designed for optimizing the neural network. • Extensive results show superior performance on human pose estimation. [ABSTRACT FROM AUTHOR]

3D human pose estimation plays an important role in various human-machine interactive applications, but how to effectively extract and represent the kinematical features of human body structure in video has always been a challenge. This paper presents some inspiring observations on the human body properties that hold heuristic patterns of human poses: 1) There is distinct temporal coherence in any kind of human pose; 2) there exist evident spatial and temporal correlations among local joints even though the human is doing complex actions. According to the observed patterns, a locally structured feature encoder and a spatial–temporal feature transform are proposed for kinematics-aware feature extraction and enhancement. Unlike existing works directly projecting every bone joint to pose features without distinction, the proposed locally-structured feature encoder maps the local connection property of human body structure to kinematical features which are neural embeddings extracted from both local and global groups of human bone joints. Since the local and global bone-joint groups are pre-defined according to human body kinematics, the kinematical features are able to represent body kinematics. The kinematical features are then transformed by the proposed spatial–temporal feature transform to enhance the spatial and temporal correlations among human bone joints. The overall framework well promotes the representation of human body kinematics for 3D pose estimation. Extensive experimental results on commonly used datasets show that the mean per joint position error (MPJPE) is significantly reduced when compared with state-of-the-art methods under the same experimental condition. The improvement is expected to promote machines to better understand human poses for building superior human-centered automation systems. • Spatial–temporal kinematic-awareness is studied for 3D human pose estimation. • Hybrid-kinematical feature encoder extracts kinematical features of 2D pose. • Spatial–temporal feature transform enhances the spatial and temporal correlations. • The fusion of spatial and temporal features promote the final 3D pose estimation. [ABSTRACT FROM AUTHOR]

3D human pose estimation has many important applications in human-computer interaction and human action recognition. Simultaneously achieving real-time speed, varying human number, and high accuracy from a single RGB image is a challenging problem. To this end, this paper proposes a multi-task and multi-level neural network structure with physical constraint. The unique network structure estimates 3D human poses from single RGB image in an end-to-end way and achieves both high accuracy and high speed. Experimental results shows that the proposed system achieves 21 fps on RTX 2080 GPU with only 33 mm accuracy loss compared with conventional works. The mechanism of the network is also analyzed through network visualization. This work shows the possibility of estimating 3D human pose from a single RGB monocular camera with real-time speed. [ABSTRACT FROM AUTHOR]

Video-based 3D human pose estimation is an important yet challenging task for many human-involved pattern recognition systems. Existing deep learning-based 3D human pose estimation methods are faced with the problems of lacking large-scale training data and lacking effective solutions to represent the complicated human body structure. To this end, this paper proposes a jo intl y learning framework entitled JoyPose that simultaneously leverages both human pose data augmentation and human pose estimation. In particular, JoyPose consists of an evolutionary data augmentation module and an anatomy-aware global–local pose feature representation module for 3D human pose estimation. The evolution for data augmentation is guided by a reinforcement learning strategy in a probabilistic way according to pose estimation loss. The anatomy-aware global–local pose feature representation module separately captures global features and local features according to anatomical and kinematic patterns observed from pose estimation errors across different human joints. The performance of the final human pose estimation is leveraged by both data augmentation and anatomy-aware global–local feature representation. Extensive experiments on three real-world datasets demonstrate the superiority and robustness against state-of-the-art methods. • JoyPose simultaneously leverages 3D human pose augmentation and pose estimation. • The distributions of crossover and mutation are learned for human pose augmentation. • Pose estimation quality is utilized to guide the distribution updating. • Global and local features are extracted based on the kinematical characters. [ABSTRACT FROM AUTHOR]