Search

International audience; Crowd counting is a conspicuous task in computer vision owing to scale variations, perspective distortions, and complex backgrounds. Existing research usually adopts the dilated convolution network to enlarge the receptive fields to solve the problem of scale variations. However, these methods easily bring background information into the large receptive fields to generate poor quality density maps. To address this problem, we propose a novel backbone called Context-guided Dense Attentional Dilated Network (CDADNet). CDADNet contains three components: an attentional module, a context-guided module and a dense attentional dilated module. The attentional module is used to provide attention maps which can remove background information, while the context-guided module is proposed to extract multi-scale contextual information. Moreover, the dense attentional dilated module aims to generate high-granularity density maps and the cascaded strategy is used to preserve information from changing scales. To verify the feasibility of our method, we compare it to the existing approaches on five crowd counting datasets (ShanghaiTech (Part_A and Part_B), WorldEXPO’10, UCSD, UCF_CC_50). The comparison results demonstrate that CDADNet is effective and robust for various scenes.

International audience; Human action recognition in videos is still an important while challenging task. Existing methods based on RGB image or optical flow are easily affected by clutters and ambiguous backgrounds. In this paper, we propose a novel Pose-Guided Inflated 3D ConvNet framework (PI3D) to address this issue. First, we design a spatial–temporal pose module, which provides essential clues for the Inflated 3D ConvNet (I3D). The pose module consists of pose estimation and pose-based action recognition. Second, for multi-person estimation task, the introduced pose estimation network can determine the action most relevant to the action category. Third, we propose a hierarchical pose-based network to learn the spatial–temporal features of human pose. Moreover, the pose-based network and I3D network are fused at the last convolutional layer without loss of performance. Finally, the experimental results on four data sets (HMDB-51, SYSU 3D, JHMDB and Sub-JHMDB) demonstrate that the proposed PI3D framework outperforms the existing methods on human action recognition. This work also shows that posture cues significantly improve the performance of I3D.

International audience; Crowd counting is a valuable technology for extremely dense scenes in the transportation. Existing methods generally have higher-order inconsistencies between ground truth density maps and generated density maps. To address this issue, we incorporate an attentional discriminator to take charge of checking the density map between the generator and the ground truth. Thus, a Cascaded Attentional Generative Adversarial Network (CACrowdGAN) is proposed that enables the attentional-driven discriminator to distinguish implausible density maps and simultaneously to guide the generator to deliver fine-grained high quality density maps. The proposed CACrowdGAN consists of two components: an attentional generator and a cascaded attentional discriminator. The attentional generator has an attention module and a density module. The attention module is developed for the generator to focus on the crowd regions of the input images, while the density module is used to provide the attentional input of the discriminator. In addition, a cascaded attentional discriminator is proposed to synthesize attentional-driven fine-grained details at different crowd regions of the input image and compute a per-pixel fine-grained loss for training generator. The proposed CACrowdGAN achieves the state-of-the-art performance on five popular crowd counting datasets (ShanghaiTech, WorldEXPO'10, UCSD, UCF_CC_50 and UCF_QNRF), which demonstrates the effectiveness and robustness of the proposed approach in the complex scenes.

International audience; With the fast development of effective and low-cost human skeleton capture systems, skeleton-based action recognition has attracted much attention recently. Most existing methods using Convolutional Neural Networks (CNN) and Long Short Term Memory (LSTM) have achieved promising performance for skeleton-based action recognition. However, these approaches are limited in the ability to explore the rich spatial–temporal relational information. In this paper, we propose a new spatial–temporal model with an end-to-end bidirectional LSTM-CNN (BiLSTM-CNN). First, a hierarchical spatial–temporal dependent relational model is used to explore rich spatial–temporal information in the skeleton data. Then a new framework is proposed to fuse CNN and LSTM. In this framework, the skeleton data are built by the dependent relational model and serve as the input of the proposed network. Then LSTM is used to extract the temporal features, and followed by a standard CNN to explore the spatial information from the output of LSTM. Finally, the experimental results demonstrate the effectiveness of the proposed model on the NTU RGB+D, SBU Interaction and UTD-MHAD dataset.

In the skeleton-based human action recognition domain, the spatial-temporal graph convolution networks (ST-GCNs) have made great progress recently. However, they use only one fixed temporal convolution kernel, which is not enough to extract the temporal cues comprehensively. Moreover, simply connecting the spatial graph convolution layer (GCL) and the temporal GCL in series is not the optimal solution. To this end, we propose a novel enhanced spatial and extended temporal graph convolutional network (EE-GCN) in this paper. Three convolution kernels with different sizes are chosen to extract the discriminative temporal features from shorter to longer terms. The corresponding GCLs are then concatenated by a powerful yet efficient one-shot aggregation (OSA) + effective squeeze-excitation (eSE) structure. The OSA module aggregates the features from each layer once to the output, and the eSE module explores the interdependency between the channels of the output. Besides, we propose a new connection paradigm to enhance the spatial features, which expand the serial connection to a combination of serial and parallel connections by adding a spatial GCL in parallel with the temporal GCLs. The proposed method is evaluated on three large scale datasets, and the experimental results show that the performance of our method exceeds previous state-of-the-art methods.

The analysis of human actions, such as recognition and person identification, is important but challenging. Previous studies usually separate person identification from action recognition. In this paper, we propose a skeleton-based end-to-end model that can simultaneously implement both person identification and action recognition and strengthen the learning of hard samples. Inspired by the recent application of recurrent neural networks and convolution neural networks to action analysis, the models of the temporal and spatial domains are created by modeling the global, local, and detailed feature-related information of the human skeleton. Moreover, during training, we add a mining and learning mechanism for hard samples that mines and focuses on them to avoid overfitting. This mining and learning mechanism enables the model to avoid a bottleneck and improve performance. The proposed model is experimentally verified on three datasets, where the results show that it can yield the state-of-the-art performance.

International audience; Generating action proposals in untrimmed videos is a challenging task, since video sequences usually contain lots of irrelevant contents and the duration of an action instance is arbitrary. The quality of action proposals is key to action detection performance. The previous methods mainly rely on sliding windows or anchor boxes to cover all ground-truth actions, but this is infeasible and computationally inefficient. To this end, this article proposes a RecapNet--a novel framework for generating action proposal, by mimicking the human cognitive process of understanding video content. Specifically, this RecapNet includes a residual causal convolution module to build a short memory of the past events, based on which the joint probability actionness density ranking mechanism is designed to retrieve the action proposals. The RecapNet can handle videos with arbitrary length and more important, a video sequence will need to be processed only in one single pass in order to generate all action proposals. The experiments show that the proposed RecapNet outperforms the state of the art under all metrics on the benchmark THUMOS14 and ActivityNet-1.3 datasets. The code is available publicly at https://github.com/tianwangbuaa/RecapNet.

International audience; This paper provides a mixed attention network for remote sensing image classification whose idea comes from VQA methods. In this way, it could deal with more complex requests beyond just identifying what is in the picture. The HAN (Hierarchial Attention Network) consists of an attention model to detect details on one hand and a self-attention model to detect global information on the other hand. Through attention heat maps we could see division of work is really effective and the HAN has a great performance on NWPU-RESISC45 data set. Furthermore, we may add some other subnetworks to reinforce this ability in the future.

International audience; At present, human action recognition is a challenging and complex task in the field of computer vision. The combination of CNN and RNN is a common and effective network structure for this task. Especially, we use 3DCNN in CNN part and ConvLSTM in RNN part. We divide the video into multiple temporal segments by average and compress each segment into one feature map by pooling layer. Adding the pooling layer, dropout layer, and batch normalization layer into ConvLSTM is our groundbreaking work. We test our model on KTH, UCF‐11, and HMDB51 datasets and achieve a high accuracy of action recognition.

Pose-based action recognition has always been an important research field in computer vision. However, most existing pose-based methods are built upon human skeleton data, which cannot be used to exploit the feature of the motion-related object, i.e., a crucial clue of discriminating human actions. To address this issue, we propose a novel pose-flow relational model, which can benefit from both pose dynamics and optical flow. First, we introduce a pose estimation module to extract the skeleton data of the key person from the raw video. Second, a hierarchical pose-based network is proposed to effectively explore the rich spatial–temporal features of human skeleton positions. Third, we embed an inflated 3D network to capture the subtle cues of the motion-related object from optical flow. Additionally, we evaluate our model on four popular action recognition benchmarks (HMDB-51, JHMDB, sub-JHMDB, and SYSU 3D). Experimental results demonstrate that the proposed model outperforms the existing pose-based methods in human action recognition.

International audience; Image classification is still a hot and challenging task in the field of computer vision. With the combination of traditional CNN and Gabor CNN, we designed color-texture convolutional neural networks. We believe that color and texture is the most important feather to describe an object. So the original image, which main consist of color information, and the image processed by Gabor, which contain the texture information, input two stream networks separately is a natural idea. And merging the output of two stream at the end of network to complete the task of classification. We design a simple but effective network structure and test it on Cifar-10, STL-10 and NWPU-RESISC45 dataset. The result shows that the network achieve a well accuracy and proves that our idea is feasible

International audience; Abnormal event detection is one of the most important objectives in security surveillance for public scenes. In this paper, a new high-performance algorithm based on spatio-temporal motion information is proposed to detect global abnormal events from the video stream as well as the local abnormal event. We firstly propose a feature descriptor to represent the movement by adopting the covariance matrix coding optical flow and the corresponding partial derivatives of multiple connective frames or the patches of the frames. The covariance matrix of multi-RoI (region of interest) which consists of frames or patches can represent the movement in high accuracy. For public surveillance video, the normal samples are abundant while there are few abnormal samples. Thus the one-class classification method is suitable for handling this problem inherently. The nonlinear one-class support vector machine based on a proposed kernel for Lie group element is applied to detect abnormal events by merely training the normal samples. The computational complexity and time performance of the proposed method is analyzed. The PETS, UMN and UCSD benchmark datasets are employed to verify the advantages of the proposed method for both global abnormal and local abnormal event detection. This method can be used for event detection for a surveillance video and outperforms the state-of-the-art algorithms. Thus it can be adopted to detect the abnormal event in the monitoring video.

International audience; Feature descriptor is one of the important subjects in video analysis problem. In this paper, we propose one video feature descriptor combining motion and appearance cues. The length-invariant characteristics of this proposed feature descriptor are clarified. Further, this feature descriptor is adopted to represent the video sequence for abnormal event detection problem, which is one challenging research field in the video surveillance. We proposed one abnormal event detection algorithm which consists of the feature descriptor and the nonlinear one-class classification method. Experiments on the benchmark dataset and comparisons with the state-of-the-art methods validate the advantages of our proposed feature descriptor.

In solving the partial Fourier Multiple Measurement Vectors (FMMV) problem, existing greedy pursuit algorithms such as Simultaneous Orthogonal Matching Pursuit (SOMP), Simultaneous Subspace Pursuit (SSP), Hybrid Matching Pursuit (HMP), and Forward-Backward Pursuit (FBP) suffer from low recovery ability or need sparsity as a prior information. This paper combines SOMP and FBP to propose a Hybrid Orthogonal Forward-Backward Pursuit (HOFBP) algorithm. As an iterative algorithm, each iteration of HOFBP consists of two stages. In the first stage, α indices selected by SOMP are added to the support set. In the second stage, the support set is shrank by removing β indices. The choice of α and β is critical to the performance of this algorithm. The simulation results showed that, by using proper parameters, HOFBP has better performance than other greedy pursuit algorithms at the expense of more computing time in some cases. HOFBP does not need sparsity as a prior knowledge.

International audience; For remote sensing image understanding, target detection is one of the most important tasks. In this paper, we propose one object detection method based on region proposal detection via active contour model and detection based on one-class classification method. The large scale remote sensing image is split into several connected components. And then, the proposed algorithm detects the object from the background. The proposed algorithm has been tested on several scenes of real unmanned aerial vehicle image datasets, and achieves promising results.

Human pose estimation has broad application prospects in the fields of human behavior recognition and human-computer interaction. Although the current human pose estimation methods have made tremendous progress, the partial occlusion of human bodies still remains a challenging problem. In this paper, we address the challenging joints in human bodies by the hard joints mining technique. The proposed hard joints mining method is based on the generative adversarial network, which consists of two stacked hourglasses with a similar architecture: the generator and the discriminator. During the training period, the discriminator distinguishes the generated heatmaps from the ground-truth heatmaps and introduces the adversarial loss to the generator through back-propagation to induce generator generates a more reasonable prediction. Moreover, the hard joints mining technique is used to focus the training attention on the difficult joint points in the generator. Finally, the experimental results demonstrate the effectiveness of the proposed approach for human pose estimation on Leeds Sports Pose (LSP) Dataset, LSP-extended datasets and MPII Human Pose Datasets.

International audience; In order to recognize the abnormal event, such as emergency or panic, happened in public scenes timely, an algorithm based on features extraction and BP Adaboost to detect abnormal frame event from surveillance video of complex situation is proposed. The proposed method detects an abnormal event where people are running, and this panic situation is simulated by the frame in a video. Experiments show that the method can distinguish and detect the abnormal event effectively and efficiently, which has potentiality to be used in the real public monitoring.

Safety is considered as one of the most crucial aspects in the modern transportation domain. In this paper, we benefit from the videos captured by multiple external video sensors from infrastructure, and propose an algorithm to perceive the environment via these data from different aspects. The algorithm consists of two parts: the descriptor for representing the event and the classification method for analyzing the scenes. The covariance matrix feature descriptor is proposed to fuse the optical flow and the intensity of the image, and the nonlinear one-class SVM with a multi-kernel strategy is used to detect the unusual events in the scene. The method is applied to analyze events in the video surveillance dataset with promising results obtained.

State-of-the-art methods for articulated human pose estimation are based on pictorial structures model (PS). Most of these methods predict the pose directly in part-based models and only consider rigid parts guided by human anatomy. In this paper, we propose a new framework for human pose estimation which is composed of two stages: pre-estimation and estimation. The first stage includes three steps: upper body detection, upper body categorization, and model selection. In the second stage, a new upper body category based multiple mixture parts (MMP) model is proposed. We present quantitative results demonstrating that our model significantly improves the accuracy of the pose estimation.

The problem of human pose estimation in still images is considered. Most previous works predicted the pose directly with either local deformable models or a global mixture representation in the pose space. We argue that this process of pose estimation can be divided into different stages. We propose a new two-stage framework for human pose estimation. In the pre-estimation stage, there are three steps: upper body detection, model category estimation for the upper body, and full model selection for pose estimation. A new method based on pairwise scores of the upper body is proposed for upper body detection. In the estimation stage, we address the problem of a variety of human poses and activities. The upper body-based multiple mixture parts (MMP) model is proposed. This model not only joins different mixture models together, but can also analyze activities with complex kinematic structures. The model is compared with the state-of-the-art. The experimental results demonstrate the effectiveness of the MMP model.