Search

The image-to-image translation aims to learn the corresponding information between the source and target domains. Several state-of-the-art works have made significant progress based on generative adversarial networks (GANs). However, most existing one-to-one translation methods ignore the correlations among different domain pairs. We argue that there is common information among different domain pairs and it is vital to multiple domain pairs translation. In this paper, we propose a unified circular framework for multiple domain pairs translation, leveraging a shared knowledge module across numerous domains. One selected translation pair can benefit from the complementary information from other pairs, and the sharing knowledge is conducive to mutual learning between domains. Moreover, absolute consistency loss is proposed and applied in the corresponding feature maps to ensure intra-domain consistency. Furthermore, our model can be trained in an end-to-end manner. Extensive experiments demonstrate the effectiveness of our approach on several complex translation scenarios, such as Thermal IR switching, weather changing, and semantic transfer tasks. [ABSTRACT FROM AUTHOR]

Domain adaptation aims to alleviate the distribution discrepancy between source and target domains. Most conventional methods focus on one target domain setting adapted from one or multiple source domains while neglecting the multi-target domain setting. We argue that different target domains also have complementary information, which is very important for performance improvement. In this paper, we propose an Attention-guided Multiple source-and-target Domain Adaptation (AMDA) method to capture the context dependency information on transferable regions among multiple source and target domains. The innovation points of this paper are as follows: (1) We use numerous adversarial strategies to harvest sufficient information from multiple source and target domains, which extends the generalization and robustness of the feature pools. (2) We propose an intra-domain and inter-domain attention module to explore transferable context information. The proposed attention module can learn domain-invariant representations and reduce the negative transfer by focusing on transferable knowledge. Extensive experiments validate the effectiveness of our method with achieving state-of-the-art performance on several unsupervised domain adaptation datasets. [ABSTRACT FROM AUTHOR]

Inpainting arbitrary missing regions is challenging because learning valid features for various masked regions is nontrivial. Though U-shaped encoder-decoder frameworks have been witnessed to be successful, most of them share a common drawback of mask unawareness in feature extraction because all convolution windows (or regions), including those with various shapes of missing pixels, are treated equally and filtered with fixed learned kernels. To this end, we propose our novel mask-aware inpainting solution. Firstly, a Mask-Aware Dynamic Filtering (MADF) module is designed to effectively learn multi-scale features for missing regions in the encoding phase. Specifically, filters for each convolution window are generated from features of the corresponding region of the mask. The second fold of mask awareness is achieved by adopting Point-wise Normalization (PN) in our decoding phase, considering that statistical natures of features at masked points differentiate from those of unmasked points. The proposed PN can tackle this issue by dynamically assigning point-wise scaling factor and bias. Lastly, our model is designed to be an end-to-end cascaded refinement one. Supervision information such as reconstruction loss, perceptual loss and total variation loss is incrementally leveraged to boost the inpainting results from coarse to fine. Effectiveness of the proposed framework is validated both quantitatively and qualitatively via extensive experiments on three public datasets including Places2, CelebA and Paris StreetView. [ABSTRACT FROM AUTHOR]

Remote person identification by gait is one of the most important topics in the field of computer vision and pattern recognition. However, gait recognition suffers severely from the appearance variance caused by the view change. It is very common that gait recognition has a high performance when the view is fixed but the performance will have a sharp decrease when the view variance becomes significant. Existing approaches have tried all kinds of strategies like tensor analysis or view transform models to slow down the trend of performance decrease but still have potential for further improvement. In this paper, a discriminative projection with list-wise constraints (DPLC) is proposed to deal with view variance in cross-view gait recognition, which has been further refined by introducing a rectification term to automatically capture the principal discriminative information. The DPLC with rectification (DPLCR) embeds list-wise relative similarity measurement among intraclass and inner-class individuals, which can learn a more discriminative and robust projection. Based on the original DPLCR, we have introduced the kernel trick to exploit nonlinear cross-view correlations and extended DPLCR to deal with the problem of multiview gait recognition. Moreover, a simple yet efficient gait representation, namely gait individuality image (GII), based on gait energy image is proposed, which could better capture the discriminative information for cross view gait recognition. Experiments have been conducted in the CASIA-B database and the experimental results demonstrate the outstanding performance of both the DPLCR framework and the new GII representation. It is shown that the DPLCR-based cross-view gait recognition has outperformed the-state-of-the-art approaches in almost all cases under large view variance. The combination of the GII representation and the DPLCR has further enhanced the performance to be a new benchmark for cross-view gait recognition. [ABSTRACT FROM AUTHOR]

Projective analysis is an important solution in three-dimensional (3D) shape retrieval, since human visual perceptions of 3D shapes rely on various 2D observations from different viewpoints. Although multiple informative and discriminative views are utilized, most projection-based retrieval systems suffer from heavy computational cost, and thus cannot satisfy the basic requirement of scalability for search engines. In the past three years, shape retrieval contest (SHREC) pays much attention to the scalability of 3D shape retrieval algorithms, and organizes several large scale tracks accordingly [1]– [3]. However, the experimental results indicate that conventional algorithms cannot be directly applied to large datasets. In this paper, we present a real-time 3D shape search engine based on the projective images of 3D shapes. The real-time property of our search engine results from the following aspects: 1) efficient projection and view feature extraction using GPU acceleration; 2) the first inverted file, called F-IF, is utilized to speed up the procedure of multiview matching; and 3) the second inverted file, which captures a local distribution of 3D shapes in the feature manifold, is adopted for efficient context-based reranking. As a result, for each query the retrieval task can be finished within one second despite the necessary cost of IO overhead. We name the proposed 3D shape search engine, which combines GPU acceleration and inverted file (t wice), as GIFT. Besides its high efficiency, GIFT also outperforms state-of-the-art methods significantly in retrieval accuracy on various shape benchmarks (ModelNet40 dataset, ModelNet10 dataset, PSB dataset, McGill dataset) and competitions (SHREC14LSGTB, ShapeNet Core55, WM-SHREC07). [ABSTRACT FROM PUBLISHER]

Pedestrian density is one of the important factors in designing visual surveillance and intelligent transportation systems, but it is challenging to obtain accurate and robust estimates because of both inconsistent crowd patterns in the scenes and target drift caused by imbalanced data distribution. Most of existing global regression frameworks focus on the former challenge to improve the robustness of regression learning, but very few work concerns on mitigating the suffering from the latter one. This paper proposes a novel counting-by-regression framework to utilize the importance of training samples to improve the robustness against inconsistent feature-target relationship based on a recently-proposed learning paradigm—learning with privileged information. To this end, the concept of back-propagation is for the first time considered to select more informative samples contributed to robust fitting performance. Moreover, the direction of target drift along the continuously-changing target dimension is discovered by learning local classifiers under different situation of pedestrian density, which can thus be exploited in our algorithm to further boost the performance. Experimental evaluation on the public UCSD and shopping Mall benchmarks verifies that our approach significantly beats the state-of-the-art counting-by-regression frameworks. [ABSTRACT FROM AUTHOR]

The performance of most gait recognition methods would drop down if the viewpoint of test data is different from the viewpoint of training data. In this paper, we present an idea of estimating the view angle of a test sample in advance so as to compare it with the corresponding training samples with the same or approximate viewpoint. In order to obtain reliable estimation results, the view-sensitive features should be extracted. We propose a novel and effective feature extraction method to characterize the silhouettes from different views. The discrimination power of this representation is also verified through experiments. Afterwards, the robust regression method is employed to estimate the viewpoint of gait. The view angles of test samples from BUAA-IRIP Gait Database are estimated with the regression models learned from CASIA Gait Database. Compared with the ground truth angles, such estimation is satisfactory with a small error level. Therefore, it can provide necessary help for gait application systems when the view angles of test data are uncertain. This point is verified experimentally through integrating the view angle estimation into a gait based gender classification system. [ABSTRACT FROM AUTHOR]

In this paper, a competitive method for 3-D face recognition (FR) using spherical harmonic features (SHF) is proposed. With this solution, 3-D face models are characterized by the energies contained in spherical harmonics with different frequencies, thereby enabling the capture of both gross shape and fine surface details of a 3-D facial surface. This is in clear contrast to most 3-D FR techniques which are either holistic or feature based, using local features extracted from distinctive points. First, 3-D face models are represented in a canonical representation, namely, spherical depth map, by which SHF can be calculated. Then, considering the predictive contribution of each SHF feature, especially in the presence of facial expression and occlusion, feature selection methods are used to improve the predictive performance and provide faster and more cost-effective predictors. Experiments have been carried out on three public 3-D face datasets, SHREC2007, FRGC v2.0, and Bosphorus, with increasing difficulties in terms of facial expression, pose, and occlusion, and which demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

This paper proposes a supervised modeling approach for gait-based gender classification. Different from traditional temporal modeling methods, male and female gait traits are competitively learned by the addition of gender labels. Shape appearance and temporal dynamics of both genders are integrated into a sequential model called mixed conditional random field (CRF) (MCRF), which provides an open framework applicable to various spatiotemporal features. In this paper, for the spatial part, pyramids of fitting coefficients are used to generate the gait shape descriptors; for the temporal part, neighborhood-preserving embeddings are clustered to allocate the stance indexes over gait cycles. During these processes, we employ evaluation functions like the partition index and Xie and Beni's index to improve the feature sparseness. By fusion of shape descriptors and stance indexes, the MCRF is constructed in coordination with intra- and intergender temporary Markov properties. Analogous to the maximum likelihood decision used in hidden Markov models (HMMs), several classification strategies on the MCRF are discussed. We use CASIA (Data set B) and IRIP Gait Databases for the experiments. The results show the superior performance of the MCRF over HMMs and separately trained CRFs. [ABSTRACT FROM AUTHOR]

Ancient Chinese character recognition is a challenging problem in the field of pattern recognition. It is difficult to collect all character classes during the training stage due to the numerous classes of ancient Chinese characters and the likelihood of discovering new characters over time. A solution to address this problem is continual learning. However, most continual learning methods are not well-suited for large-scale applications, making them insufficient for solving the problem of ancient Chinese character recognition. Although saving raw data for old classes is a good approach for continual learning to address large-scale problems, it is often infeasible due to the lack of data accessibility in reality. To solve these problems, we propose a large-scale continual learning framework based on the convolutional prototype network (CPN), which does not save raw data for old classes. In this paper, several basic strategies have been proposed for the initial training stage to enhance the feature extraction ability and robustness of the network, which can improve the performance of the model in continual learning. In addition, we propose two practical methods in varying feature space (parameters of feature extractor are changeable) and fixed feature space (parameters of feature extractor are fixed), which enable the model to carry out large-scale continual learning. The proposed method does not save the raw data of old classes and enables simultaneous classification of all existing classes without knowing the incremental batch number. Experiments on the CASIA-AHCDB dataset with 5000 character classes demonstrate the effectiveness and superiority of the proposed method. • We propose a prototype-based framework for large-scale continual learning. • The proposed strategies in the initial stage are good for continual learning. • We propose two continual learning methods in varying and fixed feature space. • The proposed methods achieve SOTA results on the CASIA-AHCDB dataset with 5000 classes. [ABSTRACT FROM AUTHOR]

A number of computer vision problems such as facial age estimation, crowd counting and pose estimation can be solved by learning regression mapping on low-level imagery features. We show that visual regression can be substantially improved by two-stage regression where imagery features are first mapped to an attribute space which explicitly models latent correlations across continuously-changing output. We propose an approach to automatically discover “spectral attributes” which avoids manual work required for defining hand-crafted attribute representations. Visual attribute regression outperforms direct visual regression and our spectral attribute visual regression achieves state-of-the-art accuracy in multiple applications. [ABSTRACT FROM AUTHOR]