Search

Although deep features have achieved the state-of-the-art performance in action recognition recently, the hand-crafted shallow features still play a critical role in characterizing human actions for taking advantage of visual contents in an intuitive way such as edge features. Therefore, the shallow features can serve as auxiliary visual cues supplementary to deep representations. In this paper, we propose a discriminative subspace learning model (DSLM) to explore the complementary properties between the hand-crafted shallow feature representations and the deep features. As for the RGB action recognition, this is the first work attempting to mine multi-level feature complementaries by the multi-view subspace learning scheme. To sufficiently capture the complementary information among heterogeneous features, we construct the DSLM by integrating the multi-view reconstruction error and classification error into an unified objective function. To be specific, we first use Fisher Vector to encode improved dense trajectories (iDT+FV) for shallow representations and two-stream convolutional neural network models (T-CNN) for generating deep features. Moreover, the presented DSLM algorithm projects multi-level features onto a shared discriminative subspace with the complementary information and discriminating capacity simultaneously incorporated. Finally, the action types of test samples are identified by the margins from the learned compact representations to the decision boundary. The experimental results on three datasets demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

In accurate action recognition, discriminative human-region representation as auxiliary information is critical for fusing multiple visual clues in a video and further improving the recognition performance. To this end, in this paper we propose integrating a novel representation named multilayer deep features (MDF) of both the human region and whole image area into an extended region-aware multiple kernel learning (ER-MKL) framework. To be specific, we first exploit the human cues with the help of the off-the-shelf semantic segmentation models. Then more powerful representations MDF are constructed by concatenating activations at the last convolutional layer and fully connected layer. Finally, the proposed framework termed ER-MKL is presented to learn a robust classifier for fusing human-region MDF and whole-region MDF. In addition to combining multiple kernels derived from features of heterogeneous image regions, ER-MKL also considers the sets of pre-learned classifiers and incorporates prior knowledge of different regions. Extensive evaluations on the JHMDB and UCF Sports datasets validate the effectiveness and the superiority of our proposed approach. [ABSTRACT FROM AUTHOR]

With the popularization of Wi-Fi signals and urgent demands for passive human action recognition, wireless sensing based activity recognition has been a hot topic in recent years. Most existing researches rely on traditional hand-crafted features, and limited work focuses on how to effectively extract deep features with spatial-temporal information. In this paper, we develop an accurate device-free action recognition system utilizing a Commodity Off-The-Shelf (COTS) router and propose a novel deep learning framework (termed two-stream network) mining spatial-temporal cues in channel state information (CSI). Specifically, an entire action sample is segmented into a series of coherent sub-activity clips. Then we try to capture the complementary features on appearance from the original CSI clips and motion between CSI frames. The spatial and temporal information are processed with separate networks which are then integrated for the final recognition task. The extensive experiments are implemented on the data collected from two indoor environments, respectively reaching 97.6% and 96.9% recognition accuracies. [ABSTRACT FROM AUTHOR]

Crowd counting is an important task in computer vision, which has many applications in video surveillance. Although the regression-based framework has achieved great improvements for crowd counting, how to improve the discriminative power of image representation is still an open problem. Conventional holistic features used in crowd counting often fail to capture semantic attributes and spatial cues of the image. In this paper, we propose integrating semantic information into learning locality-aware feature (LAF) sets for accurate crowd counting. First, with the help of a convolutional neural network, the original pixel space is mapped onto a dense attribute feature map, where each dimension of the pixelwise feature indicates the probabilistic strength of a certain semantic class. Then, LAF built on the idea of spatial pyramids on neighboring patches is proposed to explore more spatial context and local information. Finally, the traditional vector of locally aggregated descriptor (VLAD) encoding method is extended to a more generalized form weighted-VLAD (W-VLAD) in which diverse coefficient weights are taken into consideration. Experimental results validate the effectiveness of our presented method. [ABSTRACT FROM AUTHOR]

The semantic segmentation task is highly related to detection and apparently can provide complementary information for detection. In this paper, we propose integrating deep semantic segmentation feature maps into the original pedestrian detection framework which combines feature channels with AdaBoost classifiers. Firstly, we develop shallow-deep channels by concatenating shallow hand-crafted and deep segmentation channels to capture appearance clues as well as semantic attributes. Then a set of manually designed filters are utilized on the new channels to generate more response feature maps. Finally a cascade AdaBoost classifier is learned for hard negatives selection and pedestrian detection. With abundant feature information, our proposed detector achieves superior results on Caltech USA 10x and ETH dataset. [ABSTRACT FROM AUTHOR]

In this paper, we propose to use direction-dependent feature pairs (DDFP) to represent actions and a novel non-negative low rank sparse model (NLRM) is developed to encode the features. We summarize our main contributions into three aspects. First, for a video we apply eight different directions to describe the spatio-temporal relations between features, and construct directional feature pairs according to their relative positions. Second, we present a non-negative low rank sparse model which incorporates the low rank term and the non-negative constraint. Our model can not only ensure the consistency of similar DDFP by the low rank term, but also enforce the sparsity of coding coefficients by the modified l 2 , 1 -norm regularization. Third, we utilize a direction-specific dictionary for each direction and encode DDFP of a specific direction by the corresponding dictionary. A video is finally represented by the concatenation of each direction׳s pooling result. Experimental results on the KTH, Weizmann and UCF sports dataset show the effectiveness of our proposed framework for human action recognition. [ABSTRACT FROM AUTHOR]

With the widespread of commercial communication equipment, WiFi signals are ubiquitous in human life. Therefore, utilizing WiFi signals to implement intelligent sensing applications is an inevitable trend. In WiFi sensing applications, through-the-wall crowd counting is a challenging problem. In the through-the-wall scenario, the wireless signal transmitted through the wall will carry a lot of noises and is severely attenuated. Therefore, the influence of human activities on the wireless signal is difficult to extract. To solve this problem, we propose TWCC, a through-the-wall crowd counting system using ambient WiFi signals. TWCC utilizes commercial WiFi equipments to extract the phase difference data of the channel state information (CSI) and transform it to sense the environment. First, TWCC preprocesses the data to remove uncorrelated noise, and then combines the sub-carrier correlation to achieve through-the-wall human detection. When people exist, TWCC extracts features from four domains as feature groups, namely time domain, subcarrier domain, frequency domain, and time-frequency domain. Then TWCC uses different backpropagation (BP) neural networks for the features of the four domains and combines with weighting and threshold judgment to realize the through-the-wall crowd counting detection. Extensive real-world experiments show that TWCC achieves an average recognition accuracy of about 90% and maintains strong robustness to different speeds and environments. [ABSTRACT FROM AUTHOR]

With the increasing need for indoor localization based application and widely use of Wi-Fi device, Wi-Fi signal is employed to the human positioning of indoor environments due to the absence of global position system. Differing from prior works of position-based scheme and track-based scheme, the path reconstruction for indoor localization proposed in this paper utilizes path segmentation and combination to reconstruct trajectory of human motion. To enhance the universality in different environments and roam easily to new scenarios, our proposed method applies unsupervised learning identification models based on artificial neural network to recognize different benchmark paths. Then, the segments of test motion path will be matched with the most similar benchmark paths, respectively, to reconstruct whole motion path in chronological order. Experimental results of two experiment scenarios indicate that the proposed system could achieve satisfactory accuracy and robustness. [ABSTRACT FROM AUTHOR]

Based on Low-Rank Representation (LRR), this paper presents a novel dictionary learning method to learn a discriminative dictionary which is more suitable for face recognition. Specifically, in order to make the dictionary more discriminating, we introduce an ideal regularization term with label information of training data to obtain low-rank coefficients. In the dictionary learning process, by optimizing the within-class reconstruction error and minimizing of between-class sub dictionaries, the learned dictionary has good representation ability for the training samples. In addition, we also suggest each sub dictionary is low-rank, which can violate with noise contained in training samples and make the dictionary more pure and compact. The learned dictionary and structured discriminative low-rank representation then will be used for classification. The proposed Discriminative Low-Rank Dictionary Learning (DLR_DL) method is evaluated on public face databases in comparison with previous dictionary learning under the same learning conditions. The experimental results demonstrate the effectiveness and robustness of our approach. [ABSTRACT FROM AUTHOR]