Search

The recent years have witnessed significant advances in deep video action recognition. However, the performances of deep learning-based video action recognition methods are limited in the case of relatively small samples or no labeled samples. Therefore, trying to use unlabeled video data to generate clustering label is essential for small sample learning and zero sample learning. In this paper, we propose a novel deep video action clustering network, which aims to learn the similarity relationship among the unlabeled video samples, and generate the clustering label for each video sample. Specifically, the proposed method simultaneously learns the spatio-temporal features and subspace representations under a jointly optimized framework. It consists of a 3D U-Net self-representation generator, a video-clip reconstruction discriminator, and a confidence-based feedback mechanism. The 3D U-Net self-representation generator learns the spatio-temporal features of the video clips and produces subspace representation matrix. Then, the similarity graph is constructed based on this subspace representation matrix, and the clustering result is obtained. In the learning procedure, a confidence-based feedback mechanism is designed to feed the high-confidence labels of partial samples back to further guide the subspace structure learning, so that the optimal result can be obtained. During training, the video-clip reconstruction discriminator is introduced to evaluate the reconstructed video clips, which is beneficial for capturing the discriminative spatio-temporal features. Experimental results on a video benchmark dataset demonstrate the effectiveness of the proposed method.

Sketch-based image retrieval (SBIR) is a challenging task due to the cross-domain gap between sketch queries and natural images. In this paper, we propose a novel edge-guided cross-domain learning network to reduce the domain gap. In particular, edge maps extracted from natural images are introduced as the bridge between two domains, and the inputs from different domains are embedded into a common feature space. An edge guidance module is proposed to fuse natural images and the corresponding edge maps, which guides the network to generate more discriminative features in the domain alignment process. Meanwhile, a shape regression module is proposed to capture the inherent shape similarity between sketches and natural images. By training the proposed network in an end-to-end process, the sketch and natural image domains can be effectively associated, which potentially overcomes the challenge of the common feature learning for two heterogeneous domains. The experimental results on the SBIR dataset demonstrate that the proposed method achieves superior performance compared with the state-of-the-art methods.

Recently, video action recognition receives lots of attention, and deep learning based methods have achieved promising performance. Most existing methods focus on spatiotemporal information encoding to learn video representation, which ignore the relevance among channels. In this paper, we propose a novel Channel-wise Temporal Attention Network (CTAN) to explore the fine-grained key information for action recognition. First, the channel-wise attention generation module is proposed to emphasize the fine-grained informative features in each frame. Then, the temporal information aggregation module is introduced before attention generation to exploit the interaction of different frames. Finally, a discriminative video-level representation for action recognition is generated by end-to-end training. Experimental results on two benchmarks, UCF101 and HMDB51, demonstrate the effectiveness of the proposed CTAN.