1. A lightweight hybrid vision transformer network for radar-based human activity recognition
- Author
-
Sha Huan, Zhaoyue Wang, Xiaoqiang Wang, Limei Wu, Xiaoxuan Yang, Hongming Huang, and Gan E. Dai
- Subjects
Medicine ,Science - Abstract
Abstract Radar-based human activity recognition (HAR) offers a non-contact technique with privacy protection and lighting robustness for many advanced applications. Complex deep neural networks demonstrate significant performance advantages when classifying the radar micro-Doppler signals that have unique correspondences with human behavior. However, in embedded applications, the demand for lightweight and low latency poses challenges to the radar-based HAR network construction. In this paper, an efficient network based on a lightweight hybrid Vision Transformer (LH-ViT) is proposed to address the HAR accuracy and network lightweight simultaneously. This network combines the efficient convolution operations with the strength of the self-attention mechanism in ViT. Feature Pyramid architecture is applied for the multi-scale feature extraction for the micro-Doppler map. Feature enhancement is executed by the stacked Radar-ViT subsequently, in which the fold and unfold operations are added to lower the computational load of the attention mechanism. The convolution operator in the LH-ViT is replaced by the RES-SE block, an efficient structure that combines the residual learning framework with the Squeeze-and-Excitation network. Experiments based on two human activity datasets indicate our method’s advantages in terms of expressiveness and computing efficiency over traditional methods.
- Published
- 2023
- Full Text
- View/download PDF