Search

Graph attention networks (GATs) have been shown effectively for representation learning. However, existing GATs only employ the first-order attention mechanism and thus fail to fully exploit and learn node's contextual feature representations. To address this issue, we propose a novel Graph Context-Attention Network (GCAN) via low and high order aggregation for representation learning. The proposed model fully exploits both low-order and high-order information of nodes for representation learning by employing high-order attention mechanism and adversary regularization constraints. The main benefit of the proposed method is that it can effectively capture the discriminative feature differences among node representations while enhancing the diversity and richness of node features. Thus it can alleviate the issue of over-smoothing in the network learning. We perform extensive experiments on nine datasets and the experimental results demonstrate the effectiveness and better performance of our approach. [ABSTRACT FROM AUTHOR]

Low rank tensor recovery (LRTR) problem has attracted wide attentions and researches, and has been applied to various fields. In this paper, we mainly consider the nonconvex relaxtion methods for LRTR. Firstly, we newly propose the tensor schatten capped p ( SC p ) norm as a nonconvex surrogate of the tensor tubal rank, which can provide a closer approximation of the tubal rank compared with some existing nonconvex approximations. Secondly, we introduce an implicit Plug-and-Play (PnP)-based regularization into the nonconvex model to further improve the recovered performance. The proposed model is abbreviated as SC p -PnP. Based on the alternating direction method of multipliers, we propose an efficient algorithm to solve SC p -PnP. The convergence of this algorithm is also provided. Finally, extensive experiments on different kinds of datasets show that our method significantly outperforms several state-of-the-art LRTR methods in terms of both quality metrics and visual effects. [ABSTRACT FROM AUTHOR]

Nuclei detection is a fundamental analytical step in digital histopathology image analysis. Since labeling the centroids for each nucleus in histopathology images is extremely time-consuming, researchers attempt to explore consistency-based approaches for efficient semi-supervised nuclei detection. However, existing methods can only be used for detection on small patches mostly containing one nucleus and contextual information among neighboring nuclei is not considered. On the contrary, using the whole image to achieve nuclei detection in a semi-supervised manner may suffer from a large amount of background noise and thus cannot yield optimal performance. To address these problems, we propose a novel semi-supervised learning method for nuclei detection on a full-size histopathology image via global consistency regularization and local consistency adversarial learning. Specifically, the proposed dual consistency semi-supervised learning can improve the efficiency in inference and learn the context-aware nuclei features by global consistency regularization. Meanwhile, local consistency adversarial learning is introduced to focus on nuclei regions and reinforce the local spatial contiguity of prediction maps. We have evaluated the proposed dual consistency semi-supervised method on public CRCHisto and collected SemiBCN datasets, and the results show that with the synergy of global consistency regularization and local consistency adversarial learning, our method delivers a significant improvement over the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Trust region policy optimization (TRPO) is one of the landmark policy optimization algorithms in deep reinforcement learning. Its purpose is to maximize a surrogate objective based on an advantage function, subject to the limited Kullback–Leibler (KL) divergence of two consecutive policies. Although there have been many successful applications of this algorithm in the literature, the approach has often been criticized for suppressing the exploration ability of some application environments due to its strict divergence constraint. As such, most researchers prefer to use entropy regularization, which is added to the expected discounted rewards or the surrogate objectives. That said, there is much debate about whether there might be an alternative strategy for regularizing TRPOs. In this paper, we present just that. Our strategy is to regularize the KL divergence-based constraint via Shannon entropy. This approach enlarges the difference between two consecutive policies and thus derives a new TRPO scheme with entropy regularization for use with KL divergence constraint. Next, the surrogate objective and Shannon entropy are approximated linearly, while the KL divergence is expanded quadratically. An efficient conjugate gradient optimization procedure then solves two sets of linear equations, providing a detailed code-level implementation that can be used for a fair experimental comparison. Extensive experiments within eight benchmark environments demonstrate that our proposed method is superior to both the original TRPO and the entropy regularized objective TRPO. Further, theoretical and experimental analysis shows that three TRPO-like methods have an equal time complexity and a close computational burden. [ABSTRACT FROM AUTHOR]

The level set is a classical image segmentation model. In order to achieve its stable evolution, the level set function should be a signed distance function (SDF). However, due to the common appearance of irregularities, it must periodically be initialized in order to remain a SDF near the zero level set. Distance regularization terms have been used to maintain the stable evolution of the level set function. We provide a survey of the various distance regularization potential functions. Firstly, we summarize many kinds of distance regularization potential functions studied in the literature. We then divide them into five classes according to the type of potential function. Secondly, we analyze the properties of every class of potential functions and their diffusion rate functions. Finally, to demonstrate the effectiveness of the distance regularization potential functions, we apply them with a region based level set energy functional for image segmentation. Experimental analyses are conducted to compare the segmentation performance of various distance regularization potential functions when combined with the classical Chan Vese model. [ABSTRACT FROM AUTHOR]

Partial labeled data is ubiquitous in the big data era. Selecting informative features, and avoiding redundant and noise features is an important task for constructing robust learning models. The inherent characteristics of samples and features need to be surveyed simultaneously. In this study, a feature selection method for partial labeled data based on dual-graph regularized is proposed. Self-representation can well mine the relation between features. Then, the self-representation is combined with the framework of sparse learning to well embody the character of data and an adaptive redundancy regularization term based on self-representation is designed to minimize the redundancy between features. In the self-representation based regularization term, the relation between features is updated dynamically during the iteration so as to capture the inherent relation between features more accurately. The manifold structure in both feature space and data space are considered jointly. Moreover, the L 2 , p - norm is imposed on the feature self-representation regularization term, self-representation coefficient matrix, and feature selection matrix, respectively. These constrains aim to enhance its robustness to outliers, and select the representative features with discriminative and low redundancy. The convex (p = 1) and non-convex (0 < p < 1) are involved in the proposed method. Then, a unified solution for the proposed method is investigated and its convergence is proved. Experimental results on public data sets show that the proposed method is effective for classification tasks when compared with some feature selection methods. [ABSTRACT FROM AUTHOR]

Estimating dynamic effective connectivity (dEC) networks is crucial to understand the time-varying directional interconnections among brain regions. It is now widely understood that brain networks have the property of being scale-free. However, this property has seldom been considered and is often inadequately preserved using conventional dEC estimation methods. As a result, important hubs and network graphical characteristics cannot be accurately obtained. In this work, we develop a new method to use a group-wise penalty together with spatial sparsity and temporal smoothness regularizations (namely Group-wise Spatial–Temporal Regularizations, GSTR) for the inference of scale-free dEC networks from functional magnetic resonance imaging (fMRI). The method employs a time-varying vector autoregressive (VAR) model, where the model coefficients can be formed as adjacency matrices of the dEC networks. Meanwhile, the proposed group-wise regularization is able to preserve the connectivities of potential hubs in scale-free networks by grouping them as an entire set. To deal with the complexity of optimization with multiple regularizations, we propose an effective algorithm based on the augmented Lagrangian multiplier. The accuracy of the GSTR method is validated using a variety of synthetic datasets with the scale-free property. Furthermore, we apply the GSTR method to an open fMRI dataset recorded from a block design visual task-related experiment containing 255 healthy participants to estimate visual-induced dEC networks and find GSTR can achieve reasonable and interpretable dEC estimates. Results from both synthetic and real-world datasets suggest that the proposed GSTR method could serve as a powerful analytical tool to accurately infer scale-free dEC patterns. [ABSTRACT FROM AUTHOR]

Existing supervised Multi-Target Feature Selection (MTFS) methods seldom consider the nearest-neighbor relationship and statistical correlation of samples underlying the output space, which leads the result of feature selection to be easily interfered by the output noise, thus making it difficult to achieve satisfactory performance. This paper proposes a novel MTFS method to preserve both the global and local target correlations. Specifically, the low-rank constraint is introduced to achieve multi-layer regression structure to better decouple the inter-input and inter-target relationships. Moreover, the local nearest-neighbor relationships and variable correlations of the sample points in the output space are also explored through adaptive graph and manifold learning, to better utilize the target correlations to improve the MTFS performance. Following the above principle, the resulting objective function and the corresponding optimization algorithm are proposed. Extensive experiments on several public datasets show that the proposed method is superior to other state-of-the-art methods. • A new Embedded MTFS method is proposed to learn the feature redundancy and target correlations in MTR problems. • The nearest-neighbor relationships and target correlations are introduced to achieve superior performance in MTFS tasks. • Extensive experiments on several public datasets show that the proposed method is superior to the compared methods. [ABSTRACT FROM AUTHOR]

For classification tasks, dictionary learning based methods have attracted lots of attention in recent years. One popular way to achieve this purpose is to introduce label information to generate a discriminative dictionary to represent samples. However, compared with traditional dictionary learning, this category of methods only achieves significant improvements in supervised learning, and has little positive influence on semi-supervised or unsupervised learning. To tackle this issue, we propose a Dynamic Label Dictionary Learning (DLDL) algorithm to generate the soft label matrix for unlabeled data. Specifically, we employ hypergraph manifold regularization to keep the relations among original data, transformed data, and soft labels consistent. We demonstrate the efficiency of the proposed DLDL approach on two kinds of tasks, including remote sensing image classification and human activity recognition. The significant improvements have demonstrated the efficiency of our method. [ABSTRACT FROM AUTHOR]

The traditional robust principal component analysis (RPCA) via decomposition into low-rank plus sparse matrices offers a powerful framework for a large variety of applications in computer vision. However, the reconstructed image experiences serious interference by Gaussian noise, resulting in the degradation of image quality during the denoising process. Thus, a novel manifold constrained joint sparse learning (MCJSL) via non-convex regularization approach is proposed in this paper. Morelly, the manifold constraint is adopted to preserve the local geometric structures and the non-convex joint sparsity is introduced to capture the global row-wise sparse structures. To solve MCJSL, an efficient optimization algorithm using the manifold alternating direction method of multipliers (MADMM) is designed with closed-form solutions and it achieves a fast and convergent procedure. Moreover, the convergence is analyzed mathematically and numerically. Comparisons among the proposed MCJSL and some state-of-the-art solvers, on several accessible datasets, are presented to demonstrate its superiority in image denoising and background subtraction. The results indicate the importance to incorporate the manifold learning and non-convex joint sparse regularization into a general RPCA framework. [ABSTRACT FROM AUTHOR]

Sparse Bayesian learning (SBL) provides state-of-the-art performance in accuracy, sparsity and probabilistic prediction for classification. In SBL, the regularization priors are automatically determined that avoids an exhaustive hyperparameter selection by cross-validation. However, scalability to large problems is a drawback of SBL because of the inversion of a potentially enormous covariance matrix for updating the regularization priors in every iteration. This paper develops an approximate SBL algorithm called ARP-SBL, where the regularization priors are approximated without inverting the covariance matrix. Therefore, our approach can easily scale up to problems with large data size or feature dimension. It alleviates the long training time and high memory complexity in SBL. Based on ARP-SBL, two scalable nonlinear SBL models: scalable relevance vector machine (ARP-RVM) and scalable sparse Bayesian extreme learning machine (ARP-SBELM) are developed for problems of large data size and large feature size respectively. Experiments on a variety of benchmarks have shown that the proposed models are with competitive accuracy compared to existing methods while i) converging faster; ii) requiring thousands of times less memory; iii) without exhaustive regularized hyperparameter selection; and iv) easily scaling up to large data size and high dimensional features. [ABSTRACT FROM AUTHOR]

• A general framework of neural grey system model is developed. • Levenberg-Marquardt algorithm and Bayesian regularization are used for the semiparametric estimation. • Bayesian regularization can tune the regularization parameters automatically. • The proposed model outperforms the other existing models in 6 real-world cases. Most machine learning models are essentially "Black-box" models, of which the performance heavily relies on large scale data sets. In this work the idea of "Grey-box" modelling is adopted in order to take most advantage of known information represented by deterministic structure, and then the neural grey system model is developed. Levenberg-Marquardt algorithm is used to train the proposed model, and the Bayesian regularization is used to tune the regularized parameter automatically. Six real world case studies are presented to show the performance of the proposed model, comparing to 6 existing machine learning models and 17 grey system models. The results show that the proposed model can significantly overperform the other models and has very good generality, illustrating its high potential in a wide variety of real world applications and the efficiency of the proposed modelling method. [ABSTRACT FROM AUTHOR]

To address the ill-posed nature of image compressive sensing (CS) recovery, it is of great importance to properly exploit image priors. In this paper, we propose a novel group residual based nonlocal low-rank regularization for image CS recovery, namely GRNLR-CS, which performs nonlocal low-rank modeling based on the residual of patch group rather than on the patch group itself as conventional methods. In our GRNLR-CS, the weighted Schatten p-norm is adopted as a non-convex surrogate function to estimate the rank of the group residual. To make our GRNLR-CS method tractable and robust, the split Bregman iteration technique is employed to develop an efficient algorithm to solve the optimization problem of GRNLR-CS. Extensive experimental results demonstrate that our GRNLR-CS method outperforms some state-of-the-art optimization based or deep learning based methods for image CS recovery. [ABSTRACT FROM AUTHOR]

Robust low-rank tensor completion plays an important role in multidimensional data analysis against different degradations, such as Gaussian noise, sparse noise, and missing entries, and has a variety of applications in image processing and computer vision. In this paper, we investigate the problem of low-rank tensor completion with different degradations for third-order tensors, and propose a transformed tensor nuclear norm method combined the tensor ℓ 1 norm with total variational (TV) regularization. Our model is based on a recently proposed algebraic framework in which the transformed tensor nuclear norm is introduced to capture lower transformed multi-rank by using suitable unitary transformations. We adopt the tensor ℓ 1 norm to detect the sparse noise, and the TV regularization to preserve the piecewise smooth structure along the spatial and tubal dimensions. Moreover, a symmetric Gauss–Seidel based alternating direction method of multipliers is developed to solve the resulting model and its global convergence is established under very mild conditions. Extensive numerical examples on both hyperspectral images and video datasets are carried out to demonstrate the superiority of the proposed model compared with several existing state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Metric learning plays an important role in various machine learning tasks. Particularly, local metric learning is prevailing since it can learn more flexible metrics on complex datasets. However it may not be robust for scene recognition because of intra-class diversity and inter-class similarity in the scene images. To address this issue, we propose a novel method, called robust local metric learning via least square regression regularization (RLML-LSR), to learn a more robust distance metric for scene recognition. We first formulate a local discriminative metric function, aimed to pull same class neighbors closer and push different classes ones farther away simultaneously. Then taking advantage of the least square regression, we minimize the regression errors of same class neighbors, such that the local geometry structure can be preserved as much as possible. Finally, the local discriminative metric function and least square regression regularization are integrated into a unified framework, which jointly promotes the robustness of local metric learning and enhances the recognition performance of scene images. Extensive experiments on both natural scene and remote sensing scene datasets demonstrate the effectiveness and robustness of the proposed RLML-LSR method for scene recognition. [ABSTRACT FROM AUTHOR]

Infrared search and track (IRST) system is widely used in many fields, however, it's still a challenging task to detect infrared small targets in complex background. This paper proposed a novel detection method called self-regularized weighted sparse (SRWS) model. The algorithm is designed for the hypothesis that data may come from multi-subspaces. And the overlapping edge information (OEI), which can detect the background structure information, is applied to constrain the sparse item and enhance the accuracy. Furthermore, the self-regularization item is applied to mine the potential information in background, and extract clutter from multi-subspaces. Therefore, the infrared small target detection problem is transformed into an optimization problem. By combining the optimization function with alternating direction method of multipliers (ADMM), we explained the solution method of SRWS and optimized its iterative convergence condition. A series of experimental results show that the proposed method outperforms state-of-the-art baselines. [ABSTRACT FROM AUTHOR]

Sentiment analysis aims to automatically detect the underlying attitudes that users express. For the documents with complex unstructured data, such as reviews, emojis and surveys, it is usually hard to precisely identify the real emotion. Thus, it becomes urgent, yet challenging, to develop a technique that can process and make use of the unstructured information. In this article, we consider sentiment classification for those unstructured features extracted from texts. We propose a regularization-based framework to pursue better classification performance by (1) introducing polarity shifters assembled with sentiment words to create novel bigram features and (2) simultaneously constructing a constraint graph to encode the relative polarity among unstructured features to improve the parameter estimation procedure. Under these settings, our approach can uncover the intrinsic semantic information from the unstructured text data. Theoretically, we justify its underlying equivalent connection with the standard Bayes classifier, which is ideally optimal when the sample distribution is known. Moreover, we show that our new method yields better generalization ability due to the reduced solution search space and the appealing asymptotic consistency. The superior performance from real data experiments demonstrates the robustness and effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Continual learning of deep neural networks is a key requirement for scaling them up to more complex applicative scenarios and for achieving real lifelong learning of these architectures. Previous approaches to the problem have considered either the progressive increase in the size of the networks, or have tried to regularize the network behavior to equalize it with respect to previously observed tasks. In the latter case, it is essential to understand what type of information best represents this past behavior. Common techniques include regularizing the past outputs, gradients, or individual weights. In this work, we propose a new, relatively simple and efficient method to perform continual learning by regularizing instead the network internal embeddings. To make the approach scalable, we also propose a dynamic sampling strategy to reduce the memory footprint of the required external storage. We show that our method performs favorably with respect to state-of-the-art approaches in the literature, while requiring significantly less space in memory and computational time. In addition, inspired by to recent works, we evaluate the impact of selecting a more flexible model for the activation functions inside the network, evaluating the impact of catastrophic forgetting on the activation functions themselves. [ABSTRACT FROM AUTHOR]

Clustering is a widely applied method in data analysis. As a novel framework of clustering analysis, penalized clustering bases itself on the sparsity of solution, which contributes to its ability of determining the best number of clusters automatically rather than specified in advance. Moreover, ℓ 1/2 regularization has been recognized extensively in recent studies. Compared with other ℓ p (0 < p < 1) regularization, it can always obtain sparser solution. Motivated by these two points, we propose a ℓ 1/2 -based penalized clustering model, and further transform it into a more general form by introducing the vector comprising pairwise differences between centroids and an auxiliary transfer matrix composed of identity matrixes and null matrixes. Finally, the transformed model is solved by the efficient half thresholding algorithm, which can not only obtain an exact analytic expression of ℓ 1/2 solutions, but also provide an effective parameter selection strategy. We also prove the convergence of the proposed half thresholding algorithm solving ℓ 1/2 -based penalized clustering model. Lastly but not least importantly, benchmark experiments on both synthesis and real data sets from UCI have been conducted to prove the superiority of the proposed method. [ABSTRACT FROM AUTHOR]

We investigate in this paper the architecture of deep convolutional networks. Building on existing state of the art models, we propose a reconfiguration of the model parameters into several parallel branches at the global network level, with each branch being a standalone CNN. We show that this arrangement is an efficient way to significantly reduce the number of parameters while at the same time improving the performance. The use of branches brings an additional form of regularization. In addition to splitting the parameters into parallel branches, we propose a tighter coupling of these branches by averaging their log-probabilities. The tighter coupling favours the learning of better representations, even at the level of the individual branches, as compared to when each branch is trained independently. We refer to this branched architecture as "coupled ensembles". The approach is generic and can be applied to almost any neural network architecture. With coupled ensembles of DenseNet-BC and parameter budget of 25 M, we obtain error rates of 2.92%, 15.68% and 1.50% on CIFAR-10, CIFAR-100 and SVHN, respectively. For the same parameter budget, DenseNet-BC has an error rate of 3.46%, 17.18%, and 1.8%, respectively. With ensembles of coupled ensembles, of DenseNet-BC networks, with 50M total parameters, we obtain error rates of 2.72%, 15.13% and 1.42%, respectively on these tasks. [ABSTRACT FROM AUTHOR]

• A weakly-supervised multi-label feature selection method called FSLCLC is proposed for feature selection with incomplete label information. • FSLCLC recovers the missing labels of partially labeled training instances by label compressing and local feature correlations. • FSLCLC simultaneously recovers the missing labels and selects features in a joint objective function with sparsity regularization. • FSLCLC outperforms other related methods across various multi-label learning evaluation metrics. Feature selection can efficiently alleviate the issue of curse of dimensionality , especially for multi-label data with multiple features to embody diverse semantics. Although many supervised feature selection methods have been proposed, they generally assume the labels of training data are complete, whilst we only have data with incomplete labels in many real applications. Some methods try to select features with missing labels of training data, they still can not handle feature selection with a large and sparse label space. In addition, these approaches focus on global feature correlations, but some feature correlations are local and shared by a subset of data. In this paper, we introduce an approach called Feature Selection with missing labels based on Label Compression and Local feature Correlation (FSLCLC for short). FSLCLC adopts the low-rank matrix factorization on the sparse sample-label association matrix to compress labels and recover the missing labels in the compressed label space. In addition, it utilizes sparsity regularization and local feature correlation induced manifold regularizations to select the discriminative features. To solve the joint optimization objective for label compression, recovering missing labels and feature selection, we develop an iterative algorithm with guaranteed convergence. Experimental results on benchmark datasets show that the proposed FSLCLC outperforms the state-of-the-art multi-label feature selection algorithms. [ABSTRACT FROM AUTHOR]

Regularization is a very effective algorithm to solve overfitting problem in neural network, which improves the generalization ability of the model. However, their working mechanisms and the impact on the model performance have not been fully explored. In this paper, we study and analyze them using information bottleneck theory and one theory from human brain sensory system. We propose a metric to characterise the encoding length of hidden layers, named as AEntry value. Then, we implement extensive experiments on MNIST and FashionMNIST datasets with several commonly used regularization algorithms, and calculate the corresponding AEntry values. We analyze these results and obtain three conclusions. (1) The introduction of regularization influences the encoding of relative features with prediction task in neural network. The early stopping technique avoids introducing unrelated information with the task into the model by stopping the training process as an appropriate iterations. Laplace, Gaussian and Sparse Response regularizations compress the related representation and improve the performance of neural network by introducing the prior information into the model. In contrast, Dropout, Batch Normalization, and Layer Normalization increase the encoding length of features by adopting redundant representation to improve the performance. (2) The encoding of neural network does not satisfy the data processing inequality of information theory, which is mainly caused by redundant coding of extracted features. (3) The overfitting is caused by introducing irrelative information with the target. These results can give us insight into building more efficient regularization algorithm to improve the performance of neural network model. [ABSTRACT FROM AUTHOR]

Object tracking is a problem about semi-supervised learning with insufficient data set. In the field of military navigation and security of public life, it is widely used to take the place of human beings. In this paper, we come up with a new algorithm based on Bayesian, CNN and PLK optical flow, which is called COB method, for object tracking problems. With the idea of track-by-detect, we cascade CNN after PLK optical flow and integrate them in a Bayesian method. Most importantly our method is proposed with an adaptive integrating method to reduce the influence of over-fitting. The integrator also introduces the competition mechanism between tracker and detector, so that the algorithm is able to update the classifier with online learning. Besides, the regularization of deep learning is used to solve the blind spots of classifier. The experimental results show that the algorithm is more robust than the previous work. [ABSTRACT FROM AUTHOR]

The goal of dynamic magnetic resonance imaging (dynamic MRI) is to visualize tissue properties and their local changes over time that are traceable in the MR signal. Compressed sensing enables the accurate recovery of images from highly under-sampled measurements by exploiting the sparsity of the images or image patches in a transform domain or dictionary. In this work, we focus on blind compressed sensing (BCS), where the underlying sparse signal model is a priori unknown, and propose a framework to simultaneously reconstruct the underlying image as well as the unknown model from highly under-sampled measurements. Specifically, in our model, the patches of the under-sampled images are approximately sparse in a transform domain. Transform learning that combines wavelet and gradient sparsity is considered as regularization in our model for dynamic MR images. The original complex problem is decomposed into several simpler subproblems, then each of the subproblems is efficiently solved with a variable splitting iterative scheme. The results of numerous experiments show that the proposed algorithm outperforms the state-of-the-art compressed sensing MRI algorithms and yields better reconstructions results. [ABSTRACT FROM AUTHOR]

Multiple instance learning (MIL) usually plays an important role in weakly labeled learning. It is able to reduce cost noted by accurate annotations and remove noisy samples with useless labels in large scale databases. However, in many real applications, the training data always contains many redundant features. In order to remove those residual features and improve the computational efficiency, some convex and nonconvex regularization terms are applied. In this paper we apply the MCP function to propose an efficient sparse modle SyMIL, and then use the proximal stochastic subgradient method (PSSG) to solve it. The convergence analysis of the algorithm is given in details. Extensive experiments show that our model can reduce redundant features effectively while ensuring the accuracy. Quantitative comparisons against several regularization terms demonstrate the advantages of the MCP regularizer in this model. Moreover, PSSG can be observed to have slightly faster convergence rate than some other methods. [ABSTRACT FROM AUTHOR]

Multi-view clustering is an unsupervised method which aims to enhance the clustering performance by combining the knowledge from multiple view data. Non-negative matrix factorization (NMF) is one of the most favourable multi-view clustering methods due to its strong representation ability of non-negative data. However, NMF only factorizes the data matrix into two non-negative factor matrices, which may limit its ability to learn higher level and more complex hierarchical information. To overcome this shortcoming, in this paper, we propose a multi-view clustering method based on deep graph regularized non-negative matrix factorization (MvDGNMF). MvDGNMF is able to extract more abstract representation by constructing a multilayer NMF model with graph Laplacian regularization and drive the last layer representation from each view to a common consensus representation. Meanwhile, an efficient algorithm using alternating multiplicative update rules is developed. Furthermore, in order to demonstrate the effectiveness of this proposed method, we employ several open datasets including image and text datasets to evaluate the clustering performance of MvDGNMF and the state-of-art methods. [ABSTRACT FROM AUTHOR]

Estimating the extents and localizations of extended sources from noninvasive EEG/MEG signals is challenging. In this paper, we have proposed a fully data driven source imaging method, namely V ariation S parse S ource I maging based on A utomatic R elevance D etermination (VSSI-ARD), to reconstruct extended cortical activities. VSSI-ARD explores the sparseness of current sources on the variation domain by employing ARD prior under empirical Bayesian framework. With convex analysis, the sources are efficiently obtained by solving a series of reweighting L 21 -norm regularization problems with ADMM. By virtue of the iterative reweighting process and sparse signal processing techniques, VSSI-ARD gets rid of the small amplitude dipoles that are more probably outside the extent of underlying sources. With the sparsity enforced on the edges using ARD prior, the estimations show clear boundaries between active and background regions without subjective thresholds. Validation with both simulated and human experimental data indicates that VSSI-ARD not only estimates the localizations of sources, but also provides relatively useful and accurate information about the extents of cortical activities. [ABSTRACT FROM AUTHOR]

For data with various complicated distribution in the original feature space, it is difficult to find the clusters of the data. Extreme learning machine (ELM) is famous for its universal approximation capability and the hidden space created by random nonlinear feature mapping. Existing ELM based clustering methods address this by constructing an embedding space, in which the cluster are easily revealed. A commonality of them is the final results have to be subsequently derived by k -means clustering. In this paper, we propose an unsupervised feature selection based extreme learning machine (UFSELM) for clustering, which integrates ELM with L2,1 norm regularization to remove the worthless hidden neurons and clusters the data directly without building an embedding. Specifically, the proposed method conducts feature selection by minimizing the L2,1 norm of output weights, and the clustering results is computed by eigendecomposition. By solving the formulated optimization problem in an iterative fashion, we improved the accuracy of clustering. We conducted experiments on several public datasets to demonstrate the effectiveness of the proposed method and further analyzed the properties of the proposed method. [ABSTRACT FROM AUTHOR]

Many recommendation methods have introduced item correlation information to alleviate the data sparsity and cold-start problems. However, existing approaches exploit either global or local item correlations, rarely consider both global and local item correlations, and thus they cannot provide advanced recommendation performance. Inspired by this, we propose a novel collaborative deep framework called GLICR to simultaneously incorporate the global and local item correlations into the model. More specifically, our proposed GLICR model tightly couples deep neural network with matrix factorization (MF), and jointly learns the deep feature representations of item content information in deep neural network and the rating matrix in MF. In addition, we introduce manifold regularization to learn the global and local item correlations directly from data. We conduct comprehensive experiments on real-world datasets at three different degrees of sparsity to confirm that our approach can effectively alleviate data sparsity problem and is superior to existing state-of-the-art recommendation techniques. This work is the first attempt that considers the global and local item correlations by manifold regularization in recommendation scenario. [ABSTRACT FROM AUTHOR]

Side information has been shown useful for improving the performance of matrix completion applications. However, in most cases, only partial side information of either the column or row space is available. In this work, we propose a novel regularization based model to incorporate partial side information in matrix completion. We provide theoretical guarantees to ensure the success of the proposed model. It is proved that the proposed model achieves the state-of-the-art sample complexity when the given partial side information is exact, and an error bound for inexact partial side information is also provided. Moreover, we provide a deterministic rule for the selection of regularization parameter. We conduct extensive experiments on both synthetic and real-world data-sets. Experimental results show that our model succeeds to incorporate partial side information, and outperforms the state-of-the-art models on most data-sets. [ABSTRACT FROM AUTHOR]

• A novel graph regularized NMF model with adaptive neighbors is presented for clustering. An efficient updating algorithm is proposed to address the optimization problem, and the convergence of this algorithm is also theoretically guaranteed. • Our model simultaneously performs intrinsic local structure learning and the factorization process. The local manifold structure is learned from the results of factorization adaptively, and the factors are reformulated to preserve the refined local structure. • Our model assigns adaptive neighbors to each of data points, which means that the similarity between data points will change. The similarity matrix will be modified by incorporating the ANs regularization constraint into NMF. Due to the effectiveness of Nonnegative Matrix Factorization (NMF) and its graph regularized extensions, these methods have been received much attention from various researchers. Generally, these methods are performed in two separate steps including Laplacian graph construction and the subsequent matrix decomposition.However, the similarity measurement for Laplacian graph is challenging since it's often affected by several factors such as the neighborhood size, choice of similarity metric, etc. As a result, the learned graph may be not suitable, let alone the subsequent matrix decomposition. In this paper, we propose adaptive graph regularized NMF. Different from existing methods, the similarity matrix is automatically learned from the data. The proposed model can simultaneously performs matrix decomposition and similarity learning. By balancing the interactions between both of the two subtasks in our model, each subtask is improved iteratively based on the result of another. Experimental results on benchmark data sets illustrate the effectiveness of our model. [ABSTRACT FROM AUTHOR]

• Both labeled data and unlabeled images are used to reduce the cost of labeling the flame hole image artificially. Making full use of the efficiency of CNN feature extraction and the comprehensiveness of fusion, as well as the high computation efficiency of ELM training process. • Laplacian regularization is utilized in the paper for local geometry information extraction, the proposed CNN-LapsELM can fully excavate information which contained in all available data. • The fusion of original calculated features for top-level ELM training process can greatly improve the accuracy for SD identification, and the experiment result indicates the SD accuracy is high to 87%. The superheat degree (SD) in industrial aluminum electrolysis cell is a critical index that can maintain the energy balance, improve the current efficiency and improve production. However, the existing SD identification is mainly relying on artificial experience and the accuracy of SD is far from satisfactory. Further, artificial costs and physical equipment are expensive and time-consuming. In this paper, we propose a deep soft sensor method for SD detection. First, CNN is utilized for flame hole image feature extraction. Second, a semi-supervised extreme learning machine (ELM) that integrates Laplacian regularization is further used for SD classification. The main contributions of the paper are: (1) The proposed CNN-LapsELM utilizes the CNN for flame hole image feature extraction and then ELM for further classification, which fully takes advantage of CNN's ability for complex feature extraction, ELM's excellent generalization ability, and high computation efficiency. (2) Both the labeled and unlabeled samples are utilized for the CNN-LapsELM training process. It fully leverages the information contained in unlabeled data. At the same time, Laplacian regularization is utilized for learning the manifold structure of hole image samples, so the performance of the proposed CNN-LapsELM are improved. (3) The proposed CNN-LapsELM algorithm improves the generalization ability and robustness. The comparison result demonstrates that the CNN-LapsELM is superior to the existing SD identification and the accuracy is 87%. [ABSTRACT FROM AUTHOR]

Person re-identification (PRID) is a rather challenging task due to the ambiguity of visual appearance. In this paper, we develop a dictionary-based projection transformation learning approach, where the idea of metric learning and dictionary learning are introduced into a unified framework to make full use of their respective advantages. More specifically, to cope with the challenge caused by dramatic changes in visual appearance, we first project the image features of pedestrian into a discriminative subspace to make the same person from different views with the same coding coefficients. Moreover, we develop a new stretch regularization to make the distance between different pedestrian images larger than that of the same pedestrian images so as to reduce the similarity exhibited by different pedestrian images. Additionally, we develop a label consistency constraint and integrate it into the dictionary learning and then we obtain the ensemble learning model of identity discriminator and dictionary. As a result, the coding coefficient and the corresponding label are bridged and the supervision from the labeled samples is also better exploited. Experimental results on five popular person re-identification benchmarks indicate that the approach developed in this paper has higher identification performance than some state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Low-rank matrix recovery (LRMR) has recently been applied to saliency detection by decomposing image features into a low-rank component associated with background and a sparse component associated with visual salient regions. Despite its great potential, existing LRMR-based saliency detection methods seldom consider the inter-relationship among elements within these two components, thus are prone to generating scattered or incomplete saliency maps. In this paper, we introduce a novel and efficient LRMR-based saliency detection model under a coarse-to-fine framework to circumvent this limitation. First, we roughly measure the saliency of image regions with a baseline LRMR model that integrates a ℓ 1 -norm sparsity constraint and a Laplacian regularization smooth term. Given samples from the coarse saliency map, we then learn a projection that maps image features to refined saliency values, to significantly sharpen the object boundaries and to preserve the object entirety. We evaluate our framework against existing LRMR-based methods on three benchmark datasets. Experimental results validate the superiority of our method as well as the effectiveness of our suggested coarse-to-fine framework, especially for images containing multiple objects. MATLAB code for implementing our method is available at https://github.com/qizhust/HLRSaliency. [ABSTRACT FROM AUTHOR]

The Wasserstein distance originated from the optimal transport theory is a general and flexible statistical metric in a variety of image processing problems. In this paper, we propose a novel Wasserstein driven low-dimensional manifold model (W-LDMM), which tactfully embraces the Wasserstein distance between two noise histograms and low-dimensional manifold regularization for noisy image restoration. Specifically, with the Wasserstein distance, the discrepancy between the estimated noise histogram and the reference noise histogram is minimized to improve the noise estimation accuracy. Moreover, based on the fact that the patch manifold structures of many natural images are low dimensional, the dimension of the image patch maniflod is utilized as regularization constraint. The key idea of W-LDMM is that the Wasserstein distance for noise constraint and the low-dimensional manifold for image regularization are complementary to each other, rather than isolated and uncorrelated. Together, they promote the W-LDMM to exhibit excellent restored results in Gaussian denoising and noisy image inpainting. Finally, extensive experiments confirm that our W-LDMM can greatly improve visual and quantitative performance compared to several popular image restoration models. [ABSTRACT FROM AUTHOR]

Graph-based semi-supervised learning (SSL) is one of the most popular topics in the past decades. Most graph-based SSL methods utilize two stage-approach to infer the class labels of the unlabeled data, where the first stage is to construct a graph in an unsupervised way for characterizing the geometry of data manifold while the second stage is to perform SSL for classification. However, the graph construction and SSL stages are usually separate. They do not share any common information to enhance the performance of classification. In this paper, we aim to solve the above problem by proposing a unified framework for SSL. In detail, we first adopt an adaptive low-rank model for graph construction, where the coefficient matrix is calculated through an efficient procedure so that the corresponding constructed graph can capture the global structure of data manifold. Meriting from such a graph, we then propose a unified framework for SSL, where we have involved the graph construction, the SSL strategy into the whole objective function. As a result, the class labels learned by SSL can provide more discriminative information for graph construction, while the updated graph can further enhance the classification accuracies of SSL. Simulation indicates that the proposed method can achieve better classification performance compared with other state-of-the-art graph-based methods. [ABSTRACT FROM AUTHOR]

• A novel link prediction method Graph Regularization weighted Nonnegative Matrix Factorization(GWNMF) is proposed. • GWNMF integrates two types of information: local topology information and links weight information. • We derive the multiplicative updating rules to learn the parameter of GWNMF. • Conducting experiments on seven real-world weighted network demonstrates that GWNMF outperforms the state-of-the-arts methods for weighted networks task. In weight networks, both link weights and topological structure are important features for link prediction. Currently, almost all existing weighted network link prediction algorithms only focused on naturally existed link weight but ignored the topological structure information. Therefore, these methods will suffer from the challenge of network sparsity and insufficient topology information. In this paper, we propose a novel Graph Regularization Weighted Nonnegative Matrix Factorization(GWNMF) model to integrate local topology information with link weights information for link prediction. Specifically, this model integrates two types of information: local topology and link weight information, and utilizes the weighted cosine similarity(WCS) method to calculate the weight similarity between local nodes. The WCS score matrix as the indicator weighted matrix to capture more useful link weight information. While graph regularization technology combines WCS score matrix to capture the local information. Besides, we derive the multiplicative updating rules to learn the parameter of this model. Empirically, we conduct the experiments on eight real-world weighted networks demonstrate that GWNMF remarkably outperforms the state-of-the-arts methods for weighted link prediction tasks. [ABSTRACT FROM AUTHOR]

Global low-rank methods have achieved great successes in tensor completion. However, these methods neglected the abundant non-local self-similarities, which exist in a wide range of multi-dimensional imaging data. To integrate the global and non-local property of the underlying tensor, we propose a novel low-rank tensor completion model via combined non-local self-similarity and low-rank regularization, which is named as NLS-LR. We adopt the parallel low-rank matrix factorization to guarantee the global low-rankness while plugging in non-local based denoisers to promote the non-local self-similarity instead of tailoring regularizers. To tackle the proposed model, we develop an efficient block successive upper-bound minimization (BSUM) based algorithm. Numerical experiment results demonstrate that the proposed method outperforms many state-of-the-art tensor completion methods in terms of quality metrics and visual effects. [ABSTRACT FROM AUTHOR]

In this paper, we propose two algorithms of the Nonnegative Tucker decomposition that contain manifold regularization terms used for capturing the underlying geometric structures, which are applied in clustering of image databases. In general, based on Tucker model in the form of least square estimation, that is to find interpretable basis factors, and achieve the compressed of original data. It usually supposes the structures of the processing databases are global linearity. However, in practical applications, such as the processing arguments of image, video sequence and Electroencephalography data, they are usually within richness structures of nonlinear. Therefore, it is necessary to uncover the underlying structures for achieving an effective application. Hence, in this article, we incorporate Laplacian Eigenmaps i.e., LE and Locally Linear Embedding i.e., LLE as the manifold regularization terms into the least square form of Tucker model. In addition, for the physical meaning of reasonable interpretation consistent with original data, thus, the basis factors and core tensor are constrained to be nonnegative. Finally, the numerical results have demonstrated both two our proposed algorithms can achieve an effective performance among state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• A novel stacked Wasserstein autoencoder (SWAE) is proposed to approximate high-dimensional data distribution. • The transport is minimized at two stages to approximate the data space while learning the encoded latent distribution. • Experiments show that the SWAE model learns semantically meaningful latent variables of the observed data. • The proposed SWAE model enables the interpolation of the latent representation and semantic manipulation. Approximating distributions over complicated manifolds, such as natural images, are conceptually attractive. The deep latent variable model, trained using variational autoencoders and generative adversarial networks, is now a key technique for representation learning. However, it is difficult to unify these two models for exact latent-variable inference and parallelize both reconstruction and sampling, partly due to the regularization under the latent variables, to match a simple explicit prior distribution. These approaches are prone to be oversimplified, and can only characterize a few modes of the true distribution. Based on the recently proposed Wasserstein autoencoder (WAE) with a new regularization as an optimal transport. The paper proposes a stacked Wasserstein autoencoder (SWAE) to learn a deep latent variable model. SWAE is a hierarchical model, which relaxes the optimal transport constraints at two stages. At the first stage, the SWAE flexibly learns a representation distribution, i.e., the encoded prior; and at the second stage, the encoded representation distribution is approximated with a latent variable model under the regularization encouraging the latent distribution to match the explicit prior. This model allows us to generate natural textual outputs as well as perform manipulations in the latent space to induce changes in the output space. Both quantitative and qualitative results demonstrate the superior performance of SWAE compared with the state-of-the-art approaches in terms of faithful reconstruction and generation quality. [ABSTRACT FROM AUTHOR]

Total variation (TV) regularization model has the excellent performance in noise-removing and edge-preserving. However, it often yields staircase artifacts in the smooth region of the restorations. To attack this problem, we propose a generalized hybrid nonconvex variational regularization model in this paper, which utilizes two nonconvex regularizers to impose the priors to the two different components of the images, respectively. One is nonconvex TV regularizer that measures the piecewise constant component of the input image, and the other is nonconvex Laplacian regularizer that models the piecewise smooth component. New model inherits the advantages of the nonconvex regularization and the first- and second-order hybrid variational regularization, which can well remove the noises while preserving edges and reducing staircase artifacts. To solve this nonconvex minimization problem efficiently, we propose a first-order algorithm based on alternating direction method of multipliers (ADMM) combining with majorization–minimization (MM) scheme. In addition, a sufficient condition for the convergence of the proposed algorithm is provided. In the experiments, we compare the proposed model with several state-of-the-art image denoising models, numerical results illustrate the effectiveness of the proposed model and algorithm for both synthetic and real images in terms of peak signal to noise ratio (PSNR) and structural similarity (SSIM) indexes. [ABSTRACT FROM AUTHOR]

This paper proposes a new stacked denoising autoencoders (SDAE), called manifold regularized SDAE (MRSDAE) based on particle swarm optimization (PSO), where manifold regularization and feature selection are embedded in the deep network. This study concentrates on using PSO to simultaneously optimize structure and parameters of SDAEs through a specific particle representation and learning method. MRSDAE aims to generate discriminant features from the data based on the integration of these effective techniques, i.e., structure and parameter optimization, manifold regularization and feature selection. The experimental results on a number of benchmark classification datasets demonstrate that MRSDAE can construct compact SDAEs with high generalization performance. Finding from this study can be used as effective guideline in learning both the structure and parameters of deep neural networks (DNNs) with manifold regularization and feature selection techniques. [ABSTRACT FROM AUTHOR]

This paper aims to propose a distributed semi-supervised learning (SSL) algorithm using extreme learning machine (ELM) over time-varying communication network, whose topology changes over time rather than being fixed. In distributed SSL problems, training data including labeled and unlabeled samples is separately stored on each node over the communication network and cannot be centrally processed. In order to solve these problems, we propose an algorithm combining the semi-supervised ELM (SS-ELM) algorithm with the zero-gradient-sum (ZGS) distributed optimization strategy. The SS-ELM algorithm, based on the manifold regularization (MR) framework, is used to approximate the mapping of samples on each node over the communication network. Then, the ZGS strategy is used to train the globally optimal coefficients of the single layer feed-forward neural network (SLFNN) corresponding to the SS-ELM algorithm. Thus, we denote the proposed algorithm as the distributed SS-ELM (DSS-ELM) algorithm. During the training process, all nodes over the communication network exchange updated coefficients rather than raw data with their neighboring nodes. It means that the DSS-ELM algorithm is a fully distributed and privacy-preserving algorithm. The convergence of the proposed DSS-ELM is guaranteed by the Lyapunov method. At last, some simulations are presented to show the efficiency of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Research on 3D human pose estimation based on deep neural networks has recently witnessed substantial progress in both accuracy and execution efficiency. Many methods combine the deep neural network-based 2D pose estimation and 3D pose matching. However, (1) multiscale analysis has the potential to improve inference accuracy, and (2) isometric constraint is not considered in the 3D matching stage. In this paper, a new module named the Densely Connected Attentional Pyramid Residual Module (DCAPRM) in the bottom-up mapping stage is presented to effectively increase the inference accuracy. A new isometric regularization term is also proposed to punish limb extension or shrinkage in the top-down fitting phase. The performance of our approach on 3D human pose datasets is evaluated. The experimental results show that our approach provides better results than other approaches in terms of the accuracy of 3D human pose estimation. [ABSTRACT FROM AUTHOR]

Vehicle re-identification (re-ID) is a vital technique to the urban intelligent video surveillance system and smart city. Given a query vehicle image, the vehicle re-ID aims to search and retrieve the images of the same vehicle that have been captured by different surveillance cameras with various viewing angles. Based on the observation that essential vehicle attributes, like vehicle's color and types (e.g., sedan, bus, truck, and so on), could be used as important traits to recognize vehicle, an effective multi-label learning (MLL) method is proposed in this paper that can simultaneously learn three labels: vehicle's ID, type, and color. With three labels, a multi-label smoothing regularization (MLSR) is further proposed, which can allocate a uniform label distribution to the multi-labeled training images to regularize MLL model and improve vehicle re-ID performance. Extensive experiments conducted on the VeRi and VehicleID datasets have demonstrated that the proposed MLL with MLSR approach can effectively improve the performance delivered by the baseline and outperform multiple state-of-the-art vehicle re-ID methods as well. [ABSTRACT FROM AUTHOR]

Abstract In this paper, a bottom-up saliency detection algorithm is proposed by introducing the manifold regularized Support Vector Machine (SVM) model. The local manifold regularization can well capture local saliency cues. However, its local-to-global mechanism often ignores the completeness of salient objects especially when the contrast between the foreground and background is low, while the global regularization can discover the long-range semantic structure but easily neglect the details. To comprehensively utilize their advantages, we respectively construct a local regularizer (LR) and a global regularizer (GR), and incorporate them into the SVM models, which are then self-trained across superpixels by pseudo labeled ones. The LR-SVM and GR-SVM models are further refined by updating training sample with higher-level object proposals. Extensive experiments on five benchmark datasets demonstrate that the proposed algorithm performs favorably against the state-of-the-art saliency detection methods. In addition, we show that the manifold regularized SVM model can be easily applied to some existing saliency models and achieve significant performance improvement. [ABSTRACT FROM AUTHOR]

Abstract Multi-task learning (MTL), with the help of the relationship among tasks, is able to improve the generalization performance of all tasks by learning multiple tasks simultaneously. Multi-task sparse feature learning, formulated under the regularization framework, is one of the main approaches for MTL. Hence, the regularization term is crucial for multi-task sparse feature models. While most of the existing models utilize convex sparse regularization, a non-convex capped-ℓ 1 regularization is extended into MTL and proven as a powerful sparse term. In this paper, we propose a novel regularization term for multi-task learning by extending the non-convex ℓ 1 − 2 regularization to multi-task learning. The regularization term can not only realize group sparsity to extract the common features shared by all tasks, but also learn task-specific features through the relaxation of the second term in regularization. Although the model formulation is similar to a proposed one for multi-class problem, we first extend ℓ 1 − 2 regularization to multi-task learning so that both common features and task-specific features can be extracted. A classical multi-task learning model (Multi-task Feature Selection, MTFS) can be viewed as a special case of our proposed model. Due to the complexity of regularization, we approximate the original problem by a locally linear subproblem and then use the Alternating Direction Method of Multipliers (ADMM) to solve this subproblem. The theoretical analysis shows the convergence of the proposed algorithm and the time complexity of the algorithm is provided. Experimental results demonstrate the effectiveness of our proposed method. [ABSTRACT FROM AUTHOR]

Abstract Spatially-regularized correlation filters have achieved great successes in visual object tracking, with excellent tracking accuracy and robustness to various interferences. Performance improvement is mainly attributed to spatial regularization (SR), which is a powerful tool to alleviate the boundary effects of correlation filters (CF) based tracking. However, SR also causes high complexity, which cannot achieve real-time tracking. Furthermore, the fixed and handcrafted SR map, disregarding object's appearance during the whole sequence, seems incapable of handling changeable scenes. In this paper, we propose a fast and object-adaptive spatial regularization (FOSR) model to alleviate those drawbacks. By introducing FOSR method, more discriminative filters can be efficiently obtained by jointly learning in spatial and frequency domain. Besides, an object-adaptive SR map that contains object information can be offline and online learned within a data-driven manner. Extensive experiments on two benchmarks, OTB-2015 and VOT-2016, validate the effectiveness and generality of our model in helping state-of-the-art SR based trackers to achieve more than 5 times of speedup and a relative gain of 3.7% and 3.3% in success and precision plots on OTB-2015, respectively. Additionally, FOSR can help pure CF based trackers to remarkably improve their accuracy with comparable speed. [ABSTRACT FROM AUTHOR]

Abstract Linear discriminant analysis (LDA) and two dimensional LDA (2DLDA) are widely applied methods for dimensionality reduction. However, both of them lack of robustness and sparseness. Recent studies show that the elastic net and the L1-norm would improve the learning ability of dimensionality reduction. In this paper, we propose a generalized elastic net and apply it into the L1-norm based LDA and 2DLDA to extend LDA and 2DLDA with robustness and sparseness simultaneously (named LDAL1-S and 2DLDAL1-S, respectively). The generalized elastic net also helps LDAL1-S and 2DLDAL1-S avoiding the singularity problem. Moreover, the Lp-norm (0 < p ≤ 1) is used in the generalized elastic net, which makes LDAL1-S and 2DLDAL1-S realize the desired sparseness of the discriminant vectors by selecting proper p. Both LDAL1-S and 2DLDAL1-S are solved through a series of convex problems with equality constraints, with a closed solution for each single problem. Experimental results on three contaminated human face databases show the effectiveness of the proposed methods. [ABSTRACT FROM AUTHOR]

Abstract Matrix completion is arguably one of the most studied problems in machine learning and data analysis. Inspired by the closed-form formulas for L 2/3 regularization, we employ the Schatten 2/3 quasi-norm to approximate the rank of a matrix, which can provide a better approximation than traditional ways. We also establish a necessary optimal condition and propose a fixed point iterative scheme for solving L 2/3 regularization problem. Through analysing the monotonicity and the accumulation point of L 2/3 regularization problem, the convergence of this iteration is analysed. By discussing the optimal selection of the regularization parameter together with a fast Monte Carlo algorithm and an approximate singular value decomposition (SVD) procedure, we build a fast and efficient algorithm that solves the induced optimization problem well. Extensive experiments have been conducted and the results show that the proposed algorithm is fast, efficient and robust. Specifically, we compare the proposed algorithm with state-of-the-art matrix completion algorithms on many synthetic data and large recommendation datasets. Our proposed algorithm is able to achieve similar or better prediction performance, while being faster and more efficient than alternatives. Furthermore, we demonstrate the effectiveness of our proposed algorithm by solving image inpainting problems. [ABSTRACT FROM AUTHOR]