Search

In the deep learning-based video action recognitio, the function of the neural network is to acquire spatial information, motion information, and the associated information of the above two kinds of information over an uneven time span. This paper puts forward a network extracting video sequence semantic information based on deep integration of local Spatial-Temporal information. The network uses 2D Convolutional Neural Network (2DCNN) and Multi Spatial-Temporal scale 3D Convolutional Neural Network (MST_3DCNN) respectively to extract spatial information and motion information. Spatial information and motion information of the same time quantum receive 3D convolutional integration to generate the temporary Spatial-Temporal information of a certain moment. Then, the Spatial-Temporal information of multiple single moments enters Temporal Pyramid Net (TPN) to generate the local Spatial-Temporal information of multiple time scales. Finally, bidirectional recurrent neutral network is used to act on the Spatial-Temporal information of all parts so as to acquire the context information spanning the length of the entire video, which endows the network with video context information extraction capability. Through the experiments on the three video action recognitio common experimental data sets UCF101, UCF11, UCFSports, the Spatial-Temporal information deep fusion network proposed in this paper has a high correct recognition rate in the task of video action recognitio.

The training method that uses predefined degradation methods to provide training data for deep convolutional neural networks suffers from overfitting and image distortion when performing face super-resolution reconstruction. In this paper, we propose a multi-task facial super-resolution reconstruction framework embedded in a degraded augmented GAN network. Compared with the commonly used method of generating low-quality face images through pixel interpolation, this paper inversely embeds a generative adversarial network composed of residual coding blocks in the network. Through the feature learning ability of the network, the feature fitting of the face images collected in the natural scene is performed to generate training data that can more fully represent the real noise in the natural scene, thereby improving the noise reduction and image reconstruction capabilities of the network. In order to take full advantage of the task-specific features, the semantic features of the dataset are extracted through a pre-trained facial organ semantic detection framework and the data is used to add face semantic information in the super-resolution reconstruction network. The experimental results show that the model proposed in this study can achieve super-resolution reconstruction of ultra-low-resolution face images simply and efficiently, and achieve super-resolution face images with lower visual quality than other advanced methods with lower model complexity. It solves the problem of single style and distortion of face reconstruction images caused by the training method of the data set obtained by the usual predefined degradation.

In video action recognition based on deep learning, the design of the neural network is focused on how to acquire effective spatial information and motion information quickly. This paper proposes a kind of deep network that can obtain both spatial information and motion information in video classification. It is called MDFs (the multidimensional motion features of deep feature map net). This method can be used to obtain spatial information and motion information in videos only by importing image frame data into a neural network. MDFs originate from the definition of 3D convolution. Multiple 3D convolution kernels with different information focuses are used to act on depth feature maps so as to obtain effective motion information at both spatial and temporal. On the other hand, we split the 3D convolution at space dimension and time dimension, and the spatial network feature map has reduced the dimensions of the original frame image data, which realizes the mitigation of computing resources of the multichannel grouped 3D convolutional network. In order to realize the region weight differentiation of spatial features, a spatial feature weighted pooling layer based on the spatial-temporal motion information guide is introduced to realize the attention to high recognition information. By means of multilevel LSTM, we realize the fusion between global semantic information acquisition and depth features at different levels so that the fully connected layers with rich classification information can provide frame attention mechanism for the spatial information layer. MDFs need only to act on RGB images. Through experiments on three universal experimental datasets of action recognition, UCF10, UCF11, and HMDB51, it is concluded that the MDF network can achieve an accuracy comparable to two streams (RGB and optical flow) that requires the import of both frame data and optical flow data in video classification tasks.

3D human motion analysis from a single viewpoint is an extremely challenge computer vision task due to the lack of depth information and complex human movements. To resolve these problems, based on the quantum computing and immune clonal operator, a novel evolution algorithm, called a quantum-behaved clonal algorithm (QBCA), is proposed for 3D human motion analysis. Firstly, a 2D part-based human detector (PBD) is used to compute the 2D landmarks of key body joints. Then, human motion analysis is performed by optimizing a distance similarity function between the detected 2D landmarks and 2D projection of predicted 3D joint points using QBCA. Moreover, our method not only has a good balance between exploitation and exploration, but also searches both local optimum solution and global optimum solution, simultaneously. Extensive results on PARSE and HumanEva dataset demonstrate the robustness and effectiveness of our proposed method.

Although the classic TLD (tracking-learning-detection) target tracking algorithm can track a single target robustly in a long period, its real-time performance is poor. CT (Compressing Tracking) real-time tracking algorithm is real-time and efficient, but it cannot accurately track the scale-changing targets. Aiming at handling both methods' shortcomings, this paper proposes an improved TLD tracking method based on the tracking by variable-scale compressing. This method uses variable-scale compression detection instead of TLD detector and uses P-N(positive-negative) learning method to update CT detector's classifier. Experimental results show that the proposed algorithm presents better performance in robustness and real-time tracking.

In the deep learning-based video action recognition, the function of the neural network is to acquire spatial information, motion information, and the associated information of the above two kinds of information over an uneven time span. We propose a network for extracting semantic information of video sequences based on the deep fusion feature of local spatial–temporal information. Convolutional neural networks (CNNs) are used in the network to extract local spatial information and local motion information, respectively. The spatial information is in three-dimensional convolution with the motion information of the corresponding time to obtain local spatial–temporal information at a certain moment. The local spatial–temporal information is then input into the long- and short-time memory (LSTM) to obtain the context relationship of the local spatial–temporal information in the long-time dimension. We add the ability of the regional attention mechanism of video frames in the neural network mechanism for obtaining context. That is, the last layer of convolutional layer spatial information and the first layer of the fully connected layer are, respectively, input into different LSTM networks, and the outputs of the two LSTMs at each time are merged again. This enables a fully connected layer that is rich in categorical information to provide a frame attention mechanism for the spatial information layer. Through the experiments on the three action recognition common experimental datasets UCF101, UCF11, and UCFSports, the spatial–temporal information deep fusion network proposed has a high correct recognition rate in the task of action recognition.