Search

Abstract: The deformable part model is one of the most effective methods for object detection. However, it simultaneously computes the scores for a holistic filter and several part filters in a relatively high-dimensional feature space, which causes the problem of low computational efficiency. This paper proposes an approach to select compact and effective features by learning a sparse deformable part model using L1-norm latent SVM. A stochastic truncated sub-gradient descent method is presented to solve the L1-norm latent SVM problem. Convergence of the algorithm is proved. Extensive experiments are conducted on the INRIA and PASCAL VOC 2007 datasets. A highly compact feature in our method can reach the state-of-the-art performance. The feature dimensionality is reduced to 12% of the original one in the INRIA dataset and less than 30% in most categories of PASCAL VOC 2007 dataset. Compared with the features used in L2-norm latent SVM, the average precisions (AP) have almost no drop using the reduced feature. With our method, the speed of the detection score computation is faster than that of the L2-norm latent SVM method by 3 times. When the cascade strategy is applied, it can be further speeded up by about an order of magnitude. [Copyright &y& Elsevier]

Deep neural networks provide end-to-end tools to learn effective representations from data directly. The deep structure makes it possible to model a complicated pattern, even if it has a variety of changes. This leads to a problem that noises and outliers are usually treated as a specific pattern, which is also learned in the network. It is one reason for overfitting incapable of being addressed sufficiently in deep networks. This paper proposes a new method called subdomain contraction (SDC) to tackle the problem. The idea is that our approach inclines to learn more about the shared features between the subsets of the samples but less about the specific features found in only one or two subsets. To this end, the SDC loss penalizes the distribution distance between sub-domains in the feature space to constrain the sharing level of features. By applying the SDC loss term, the data drive the learning process to an optimal tradeoff between modeling noises and the varieties of the pattern. In this manner, the SDC models the pattern as much as possible and ignores most noises, thus improving the generalization ability. The SDC loss can be efficiently computed in minibatches and can also work collaboratively with other regularization methods such as dropout to further improve the performance. Extensive experiments demonstrate that SDC can improve the effectiveness and robustness of representation learning in deep networks against noises, and the superiority is most remarkable with noisy data. [ABSTRACT FROM AUTHOR]

Traditional non-negative matrix factorization (NMF) is an unsupervised method that represents non-negative data by a part-based dictionary and non-negative codes. Recently, the unsupervised NMF has been extended to discriminative ones for classification problems. However, these discriminative methods may become inefficient when outliers are presented in the data, e.g. mislabeled samples, because the outliers usually deviate from the normal samples in one class and would perturb the discriminative dictionary. In this paper, we propose a novel method, called robust discriminative non-negative matrix factorization (RDNMF), to reduce the effect of outliers and improve the discriminative strength. The RDNMF learns a non-negative dictionary for each class, and each dictionary contains two parts: a discriminative part and an outlier part. The discriminative parts are obtained by minimizing the cosine similarity between classes. The codes on the outlier part are required to be sparse so that most outliers can be modeled by this part, without large influence over the discriminative part.The final dictionary is obtained by concatenating the discriminative parts of all classes, and the non-negative codes for each sample, as well as test sample, are obtained by coding with this dictionary. Experimental comparisons with existing dictionary learning methods on MNIST, PIE, Yale B and ORL demonstrate the effectiveness and robustness of our approach. [ABSTRACT FROM AUTHOR]

Abstract: This paper proposes a novel algorithm to efficiently compute the histograms in densely overlapped polygonal regions. An incremental scheme is used to reduce the computational complexity. By this scheme, only a few entries in an existing histogram need to be updated to obtain a new histogram. The updating procedure makes use of a few histograms attached to the polygon's edges, which can be efficiently pre-computed in a similar incremental manner. Thus, the overall process can achieve higher computational efficiency. Further, we extend our method to efficiently evaluate objective functions on the histograms in polygonal regions. The experiments on natural images demonstrate the high efficiency of our method. [Copyright &y& Elsevier]

Activity-dependent plasticity plays an important role in biological neural network learning. Unlike the backpropagation-based learning in artificial neural networks that depends on supervised signals, biological neurons adjust themselves using their own historical behaviors as a clue to benefit information processing. Inspired by this biological neural mechanism, this paper proposes a novel neuron model, named Activity-Dependent Neuron (ADN). The basic idea of ADN is to add a gate in the neuron model to control its signal conduction capability, that is, the gate will facilitate transmission of important signals while suppress trivial signals. This idea can be achieved by self-tuning of activation behaviors. We also develop an iterative training algorithm, so that the ADNs can be smoothly incorporated into deep neural networks to jointly learn with the network weights. Experimental results found that the ADNs can efficiently improve the noise-resistant capability. Compared with the state-of-the-art, it is more robust to unforeseen noises. It does not need noise data for training. [ABSTRACT FROM AUTHOR]

To compute the optimal strategy in competitive games, algorithms have been developed to achieve the Nash equilibrium. Current deep learning algorithms have succeeded in many games; however, optimizing the algorithms to approach the Nash equilibrium in imperfect-information games like StarCraft and Poker remains challenging. Neural Fictitious Self-Play (NFSP) is an effective end-to-end algorithm to learn an approximate Nash equilibrium in imperfect-information games. However, because a player in NFSP trains its best response according to its opponents' past strategies, a discrepancy exists between the optimal strategy and the learned best response after the player updates its strategies. We call this discrepancy the optimality gap. During training, the optimality gap does not decay monotonically, which causes suboptimal results or unstable convergence of NFSP. We improve the performance of NFSP by allowing the optimality gap to decay monotonically. In this study, we propose Regret Minimization Fictitious Self-Play (RM-FSP), which applies a regret minimization method to compute NFSP's best response. The regret minimization method effectively converges the optimality gap monotonically and faster than in NFSP. We prove there will be a better learning bound than the original NFSP after applying regret minimization methods to NFSP. Experiments on three typical environments in OpenSpiel demonstrate that RM-FSP outperforms NFSP in both exploitability (discrepancy between the learned policy profile and the Nash equilibrium) and time efficiency. [ABSTRACT FROM AUTHOR]

Abstract Deep learning, especially Convolutional Neural Networks (CNNs), has been widely applied in many domains. The large number of parameters in a CNN allow it to learn complex features, however, they may tend to hinder generalization by over-fitting training data. Despite many previously proposed regularization methods, over-fitting is still a problem in training a robust CNN. Among many factors that lead to over-fitting, the numerous parameters of fully-connected layers (FCLs) of a typical CNN should be taken into account. This paper proposes the SparseConnect, a simple idea which alleviates over-fitting by sparsifying connections to FCLs. Experimental results on three benchmark datasets MNIST, CIFAR10 and ImageNet show that the SparseConnect outperforms several state-of-the-art regularization methods. [ABSTRACT FROM AUTHOR]