Search

Video compression sensing can use a few measurements to obtain the original video by reconstruction algorithms. There is a natural correlation between video frames, and how to exploit this feature becomes the key to improving the reconstruction quality. More and more deep learning-based video compression sensing (VCS) methods are proposed. Some methods overlook interframe information, so they fail to achieve satisfactory reconstruction quality. Some use complex network structures to exploit the interframe information, but it increases the parameters and makes the training process more complicated. To overcome the limitations of existing VCS methods, we propose an efficient end-to-end VCS network, which integrates the measurement and reconstruction into one whole framework. In the measurement part, we train a measurement matrix rather than a pre-prepared random matrix, which fits the video reconstruction task better. An unfolded LSTM network is utilized in the reconstruction part, deeply fusing the intra- and interframe spatial–temporal information. The proposed method has higher reconstruction accuracy than existing video compression sensing networks and even performs well at measurement ratios as low as 0.01. [ABSTRACT FROM AUTHOR]

Most dynamic sign word misclassifications are caused by redundant spatial–temporal (SPT) feature pruning that lacks language semantic and temporal dependencies. SPT feature recognition is one of the important aspects for the evaluation of the misclassification of dynamic sign words. The redundant pruning of SPT feature space influences the language model of sign confusion, model complexity, and SPT feature similarity. The purpose of this article is to develop a new multi-scale SPT feature-based dynamic sign word recognition approach via a low-cost feature selection method (FS) and End-to-end Fourier convolution neural network (EFCNN). Instead of a sensor fusion technique for obtaining frame position alignment, in the EFCNN, new 3D frame position and coordinates are determined using a pixel weighting and alignment function of the first and succeeding 25 spatial intensities of the 3D video changes across hand motion. The new spatial weight and the original spatial coordinates are fused and truncated in the Fourier domain. We generate the temporal dependence of the fused features. A feature selection known as the FS-EFCNN is introduced to select compact features with a preserved language meaning. Five state-of-the-art feature selection methods, namely Infinite FS (InFS), Relief FS, Fisher, MIM, ILFS, and ensemble FS-EFCNN were deployed to guide and optimize the learning performance of EFCNN. The experimental result analysis highlighted the improved results of the FS-EFCNN method with the best accuracy of 99.86%, 99.89%, and 90.69% on 3D American Sign Language, British Sign Language, and Greek Sign Language data sets, respectively. [Display omitted] • FS-EFCNN is an automatic sign-based FS model for sign language recognition. • The FS model combines 3D spatial–temporal features and Fourier features. • The FS-EFCNN is verified using 3D real-life data sets without complex preprocessing. • The FS-EFCNN model achieved state-of-the-art performance with compact features. [ABSTRACT FROM AUTHOR]

Video compression sensing can use a few measurements to obtain the original video by reconstruction algorithms. There is a natural correlation between video frames, and how to exploit this feature becomes the key to improving the reconstruction quality. More and more deep learning-based video compression sensing (VCS) methods are proposed. Some methods overlook interframe information, so they fail to achieve satisfactory reconstruction quality. Some use complex network structures to exploit the interframe information, but it increases the parameters and makes the training process more complicated. To overcome the limitations of existing VCS methods, we propose an efficient end-to-end VCS network, which integrates the measurement and reconstruction into one whole framework. In the measurement part, we train a measurement matrix rather than a pre-prepared random matrix, which fits the video reconstruction task better. An unfolded LSTM network is utilized in the reconstruction part, deeply fusing the intra- and interframe spatial–temporal information. The proposed method has higher reconstruction accuracy than existing video compression sensing networks and even performs well at measurement ratios as low as 0.01.