Search

3D human pose estimation plays an important role in various human-machine interactive applications, but how to effectively extract and represent the kinematical features of human body structure in video has always been a challenge. This paper presents some inspiring observations on the human body properties that hold heuristic patterns of human poses: 1) There is distinct temporal coherence in any kind of human pose; 2) there exist evident spatial and temporal correlations among local joints even though the human is doing complex actions. According to the observed patterns, a locally structured feature encoder and a spatial–temporal feature transform are proposed for kinematics-aware feature extraction and enhancement. Unlike existing works directly projecting every bone joint to pose features without distinction, the proposed locally-structured feature encoder maps the local connection property of human body structure to kinematical features which are neural embeddings extracted from both local and global groups of human bone joints. Since the local and global bone-joint groups are pre-defined according to human body kinematics, the kinematical features are able to represent body kinematics. The kinematical features are then transformed by the proposed spatial–temporal feature transform to enhance the spatial and temporal correlations among human bone joints. The overall framework well promotes the representation of human body kinematics for 3D pose estimation. Extensive experimental results on commonly used datasets show that the mean per joint position error (MPJPE) is significantly reduced when compared with state-of-the-art methods under the same experimental condition. The improvement is expected to promote machines to better understand human poses for building superior human-centered automation systems. • Spatial–temporal kinematic-awareness is studied for 3D human pose estimation. • Hybrid-kinematical feature encoder extracts kinematical features of 2D pose. • Spatial–temporal feature transform enhances the spatial and temporal correlations. • The fusion of spatial and temporal features promote the final 3D pose estimation. [ABSTRACT FROM AUTHOR]

Video-based 3D human pose estimation is an important yet challenging task for many human-involved pattern recognition systems. Existing deep learning-based 3D human pose estimation methods are faced with the problems of lacking large-scale training data and lacking effective solutions to represent the complicated human body structure. To this end, this paper proposes a jo intl y learning framework entitled JoyPose that simultaneously leverages both human pose data augmentation and human pose estimation. In particular, JoyPose consists of an evolutionary data augmentation module and an anatomy-aware global–local pose feature representation module for 3D human pose estimation. The evolution for data augmentation is guided by a reinforcement learning strategy in a probabilistic way according to pose estimation loss. The anatomy-aware global–local pose feature representation module separately captures global features and local features according to anatomical and kinematic patterns observed from pose estimation errors across different human joints. The performance of the final human pose estimation is leveraged by both data augmentation and anatomy-aware global–local feature representation. Extensive experiments on three real-world datasets demonstrate the superiority and robustness against state-of-the-art methods. • JoyPose simultaneously leverages 3D human pose augmentation and pose estimation. • The distributions of crossover and mutation are learned for human pose augmentation. • Pose estimation quality is utilized to guide the distribution updating. • Global and local features are extracted based on the kinematical characters. [ABSTRACT FROM AUTHOR]