1. Enhancing human behavior recognition with spatiotemporal graph convolutional neural networks and skeleton sequences.
- Author
-
Xu, Jianmin, Liu, Fenglin, Wang, Qinghui, Zou, Ruirui, Wang, Ying, Zheng, Junling, Du, Shaoyi, and Zeng, Wei
- Subjects
GRAPH neural networks ,CONVOLUTIONAL neural networks ,HUMAN activity recognition ,HUMAN behavior ,CARTESIAN coordinates ,HUMAN skeleton - Abstract
Objectives: This study aims to enhance supervised human activity recognition based on spatiotemporal graph convolutional neural networks by addressing two key challenges: (1) extracting local spatial feature information from implicit joint connections that is unobtainable through standard graph convolutions on natural joint connections alone. (2) Capturing long-range temporal dependencies that extend beyond the limited temporal receptive fields of conventional temporal convolutions. Methods: To achieve these objectives, we propose three novel modules integrated into the spatiotemporal graph convolutional framework: (1) a connectivity feature extraction module that employs attention to model implicit joint connections and extract their local spatial features. (2) A long-range frame difference feature extraction module that captures extensive temporal context by considering larger frame intervals. (3) A coordinate transformation module that enhances spatial representation by fusing Cartesian and spherical coordinate systems. Findings: Evaluation across multiple datasets demonstrates that the proposed method achieves significant improvements over baseline networks, with the highest accuracy gains of 2.76 % on the NTU-RGB+D 60 dataset (Cross-subject), 4.1 % on NTU-RGB+D 120 (Cross-subject), and 4.3 % on Kinetics (Top-1), outperforming current state-of-the-art algorithms. This paper delves into the realm of behavior recognition technology, a cornerstone of autonomous systems, and presents a novel approach that enhances the accuracy and precision of human activity recognition. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF