Search

There have been a lot of methods to address the recognition of complete face images. However, in real applications, the images to be recognized are usually incomplete, and it is more difficult to realize such a recognition. In this paper, a novel convolution neural network frame, named a low-rank-recovery network (LRRNet), is proposed to conquer the difficulty effectively inspired by matrix completion and deep learning techniques. The proposed LRRNet first recovers the incomplete face images via an approach of matrix completion with the truncated nuclear norm regularization solution, and then extracts some low-rank parts of the recovered images as the filters. With these filters, some important features are obtained by means of the binaryzation and histogram algorithms. Finally, these features are classified with the classical support vector machines (SVMs). The proposed LRRNet method has high face recognition rate for the heavily corrupted images, especially for the images in the large databases. The proposed LRRNet performs well and efficiently for the images with heavily corrupted, especially in the case of large databases. Extensive experiments on several benchmark databases demonstrate that the proposed LRRNet performs better than some other excellent robust face recognition methods., (Copyright © 2017 Elsevier Ltd. All rights reserved.)

As we know, feedforward neural networks (FNNs) with sigmoidal activation function are universal approximators. Theoretically, for any continuous function defined on a compact set, there exists an FNN such that the FNN can approximate the function with arbitrary accuracy, which constitutes a theoretical guarantee that FNN can be used as an efficient learning machine. This paper addresses the construction and approximation for FNNs. We construct an FNN with sigmoidal activation function and estimate its approximation error. In particular, an inverse theorem of the approximation is established, which implies the equivalence characterization theorem of the approximation and reveals the relationship between the topological structure of the FNN and its approximation ability. As keys in this study, the concepts of modulus of continuity of function, K-functional, and their relationship are utilized, and two Bernstein-type inequalities are established. [ABSTRACT FROM AUTHOR]

Summary: This paper aims to solve the robust convex optimization (RCO) problem, where the constraints of RCO are parameterized with uncertainties, and the scenario approach is applied to transform RCO into standard convex optimization with a finite number of constraints through probabilistic approximation. The transformed problem is called a scenario problem (SP). Two consensus‐based distributed learning algorithms for SP are designed in consideration of a large number of sampled constraints. One is based on the distributed average consensus (DAC), and the other is based on the alternating direction method of multipliers (ADMM). It has regulated that data distributed to nodes are not allowed to communicate. Simulation results indicate that the proposed algorithms are suitable for handling large‐scale data and achieve excellent performance, with the ADMM‐based algorithm performing the best. Furthermore, the DAC‐based algorithm has certain advantages in terms of computational time and complexity. In addition, to improve the communicative efficiency based on a DAC, an efficient distributed average consensus (EDAC) is put forward. The average time for every node when using an EDAC is less than that of a DAC, despite the exact same performance. [ABSTRACT FROM AUTHOR]

Recently, various convolutional neural networks (CNNs) based single image super-resolution (SR) methods have been vigorously explored, and a lot of impressive results have emerged. However, more or less unfortunately, most of the methods mainly focused on increasing the depth of network to improve reconstruction performance. As a matter of fact, deeper depth of network usually means an increase in parameters and computations, or worse still, the increase in parameters or computations often results in the difficulty to train the network. This paper develops a new SR approach called multi-scale residual channel attention network (MSRCAN), which is comparative shallow two-stage neural network structure, and can extract more details to effectively ameliorate the quality of SR. Specifically, a multi-scale residual channel attention block (MSRCAB) is designed to plenarily exploit the image features with convolutional kernels of different sizes. At the same time, a channel attention mechanism is introduced to recalibrate the channel significance of feature mappings adaptively. Furthermore, multiple short skip connections and a long skip connection are presented in each MSRCAB to complement information loss. Moreover, the two-stage design contributes to fully uncover low-level and high-level information. Evaluation on the benchmark data set indicates that the proposed method can rival the state-of-the-art convolutional methods. [ABSTRACT FROM AUTHOR]

With the explosive growth in size of datasets, it becomes more significant to develop effective learning schemes for neural networks to deal with large scale data modelling. This paper proposes an iterative approximate Newton-type learning algorithm to build neural networks with random weights (NNRWs) for problem solving, where the whole training samples are divided into some small subsets under certain assumptions, and each subset is employed to construct a local learner model for integrating a unified classifier. The convergence of the output weights of the unified learner model is given. Experimental results on UCI datasets with comparisons demonstrate that the proposed algorithm is promising for large scale datasets. [ABSTRACT FROM AUTHOR]

It is well known that the support vector machine (SVM) is an effective learning algorithm. The alternating direction method of multipliers (ADMM) algorithm has emerged as a powerful technique for solving distributed optimisation models. This paper proposes a distributed SVM algorithm in a master–slave mode (MS-DSVM), which integrates a distributed SVM and ADMM acting in a master–slave configuration where the master node and slave nodes are connected, meaning the results can be broadcasted. The distributed SVM is regarded as a regularised optimisation problem and modelled as a series of convex optimisation sub-problems that are solved by ADMM. Additionally, the over-relaxation technique is utilised to accelerate the convergence rate of the proposed MS-DSVM. Our theoretical analysis demonstrates that the proposed MS-DSVM has linear convergence, meaning it possesses the fastest convergence rate among existing standard distributed ADMM algorithms. Numerical examples demonstrate that the convergence and accuracy of the proposed MS-DSVM are superior to those of existing methods under the ADMM framework. [ABSTRACT FROM AUTHOR]

Most recent learning-based image super-resolution (SR) methods have attracted much interest. This paper proposes a new SR method with multi-channel constraints, which integrates clustering, collaborative representation, and progressive multi-layer mapping relationships to reconstruct high-resolution (HR) color image. In order to collect chrominance information, the training patches from RGB channels are clustered into different subspaces, and a number of neighbor subsets are grouped. Then the optimization problem with color channel constraints is solved by using the classical gradient technique. Finally, a continuous reconstructive structure, which learns multi-layer mapping relationships between intermediate output and corresponding original HR image, is designed to obtain the desired HR image. Extensive experiments on several commonly used image SR testing datasets indicate that the proposed method achieves state-of-the-art image SR results. [ABSTRACT FROM AUTHOR]

Extreme learning machine (ELM) is widely used in training single-hidden layer feedforward neural networks (SLFNs) because of its good generalization and fast speed. However, most improved ELMs usually discuss the approximation problem for sample data with output noises, not for sample data with noises both in input and output values, i.e., error-in-variable (EIV) model. In this paper, a novel algorithm, called (regularized) TLS-ELM, is proposed to approximate the EIV model based on ELM and total least squares (TLS) method. The proposed TLS-ELM uses the idea of ELM to choose the hidden weights, and applies TLS method to determine the output weights. Furthermore, the perturbation quantities of hidden output matrix and observed values are given simultaneously. Comparison experiments of our proposed TLS-ELM with least square method, TLS method and ELM show that our proposed TLS-ELM has better accuracy and less training time. [ABSTRACT FROM AUTHOR]

This paper proposes a classifier named ensemble of polyharmonic extreme learning machine, whose part weights are randomly assigned, and it is harmonic between the feedforward neural network and polynomial. The proposed classifier provides a method for human face recognition integrating fast discrete curvelet transform (FDCT) with 2-dimension principal component analysis (2DPCA). FDCT is taken to be a feature extractor to obtain facial features, and then these features are dimensionality reduced by 2DPCA to decrease the computational complexity before they are input to the classifier. Comparison experiments of the proposed method with some other state-of-the-art approaches for human face recognition have been carried out on five well-known face databases, and the experimental results show that the proposed method can achieve higher recognition rate. [ABSTRACT FROM AUTHOR]

Abstract: In this article, we address the problem of compressed classification learning. A generalization bound of the support vector machines (SVMs) compressed classification algorithm with uniformly ergodic Markov chain samples is established. This bound indicates that the accuracy of the SVM classifier in the compressed domain is close to that of the best classifier in the data domain. In a sense, the fact that the compressed learning can avoid the curse of dimensionality in the learning process is shown. In addition, we show that compressed classification learning reduces the learning time at the price of decreasing the classification accuracy, but the decrement can be controlled. The numerical experiments further verify the results claimed in this article. [Copyright &y& Elsevier]

We establish a general oracle inequality for regularized risk minimizers with strongly mixing observations, and apply this inequality to support vector machine (SVM) type algorithms. The obtained main results extend the previous known results for independent and identically distributed samples to the case of exponentially strongly mixing observations. [ABSTRACT FROM AUTHOR]

Abstract: In this work, a new image classification method is proposed based on extreme k-means (EKM) and effective extreme learning machine (EELM). The proposed method has image decomposition with curvelet transform, reduces dimensionality with discriminative locality alignment (DLA), generates a set of distinctive features with EKM, and has a classification with EELM. Since EKM has a better clustering performance than k-means and EELM has a better accuracy than ELM, the proposed EKM-EELM algorithm has a significant improvement in classification rate. Extensive experiments are performed using challenging databases and results are compared against state of the art techniques. Experimental results show that the proposed method has superior performances on classification rate than some other traditional methods for image classification. [Copyright &y& Elsevier]

Abstract: The ranking problem is to learn a real-valued function which gives rise to a ranking over an instance space, which has gained much attention in machine learning in recent years. This article gives analysis of the convergence performance of neural networks ranking algorithm by means of the given samples and approximation property of neural networks. The upper bounds of convergence rate provided by our results can be considerably tight and independent of the dimension of input space when the target function satisfies some smooth condition. The obtained results imply that neural networks are able to adapt to ranking function in the instance space. Hence the obtained results are able to circumvent the curse of dimensionality on some smooth condition. [Copyright &y& Elsevier]

Functional data learning is an extension of traditional data learning, that is, learning the data chosen from the Euclidean space $${\mathbb{R}^{n}}$$ to a metric space. This paper focuses on the functional data learning with generalized single-hidden layer feedforward neural networks (GSLFNs) acting on some metric spaces. In addition, three learning algorithms, named Hilbert parallel overrelaxation backpropagation (H-PORBP) algorithm, ν-generalized support vector regression (ν-GSVR) and generalized extreme learning machine (G-ELM) are proposed to train the GSLFNs acting on some metric spaces. The experimental results on some metric spaces indicate that GELM with additive/RBF hidden-nodes has a faster learning speed, a better accuracy, and a better stability than HPORBP algorithm and ν-GSVR for training the functional data. The idea of GELM can be used to extend those improved extreme learning machines (ELMs) that act on the Euclidean space $${\mathbb{R}^{n}, }$$ such as online sequential ELM, incremental ELM, pruning ELM and so on, to some metric spaces. [ABSTRACT FROM AUTHOR]

Abstract: Extreme learning machine (ELM), proposed by Huang et al., has been shown a promising learning algorithm for single-hidden layer feedforward neural networks (SLFNs). Nevertheless, because of the random choice of input weights and biases, the ELM algorithm sometimes makes the hidden layer output matrix H of SLFN not full column rank, which lowers the effectiveness of ELM. This paper discusses the effectiveness of ELM and proposes an improved algorithm called EELM that makes a proper selection of the input weights and bias before calculating the output weights, which ensures the full column rank of H in theory. This improves to some extend the learning rate (testing accuracy, prediction accuracy, learning time) and the robustness property of the networks. The experimental results based on both the benchmark function approximation and real-world problems including classification and regression applications show the good performances of EELM. [Copyright &y& Elsevier]

Abstract: In this paper, we give several results of learning errors for linear programming support vector regression. The corresponding theorems are proved in the reproducing kernel Hilbert space. With the covering number, the approximation property and the capacity of the reproducing kernel Hilbert space are measured. The obtained result () shows that the learning error can be controlled by the sample error and regularization error. The mentioned sample error is summarized by the errors of learning regression function and regularizing function in the reproducing kernel Hilbert space. After estimating the generalization error of learning regression function (), the upper bound () of the regularized learning algorithm associated with linear programming support vector regression is estimated. [Copyright &y& Elsevier]

Abstract: It is well-known that single hidden layer feed-forward neural networks (SLFNs) with at most n hidden neurons can learn n distinct samples with zero error, and the weights connecting the input neurons and the hidden neurons and the hidden node thresholds can be chosen randomly. Namely, for n distinct samples, there exist SLFNs with n hidden neurons that interpolate them. These networks are called exact interpolation networks for the samples. However, for some approximated target functions (as continuous or integrable functions) not all exact interpolation networks have good approximation effect. This paper, by using a functional approach, rigorously proves that for given distinct samples there exists an SLFN which not only exactly interpolates samples but also near best approximates the target function. [ABSTRACT FROM AUTHOR]

In recent years, constructing various deep convolutional neural networks (CNNs) for single-image super-resolution (SISR) tasks has made significant progress. Despite their high performance, numerous CNNs are limited in practical applications, owing to the requirement of heavy computation. This paper proposes a lightweight network for SISR, known as attention-based multi-scale residual network (AMSRN). In detail, a residual atrous spatial pyramid pooling (ASPP) block as well as a spatial and channel-wise attention residual (SCAR) block is stacked alternately to support the main framework of the entire network. The residual ASPP block utilizes parallel dilated convolutions of different dilation rates to achieve the purpose of capturing multi-scale features. The SCAR block adds the channel attention (CA) and spatial attention (SA) mechanisms based on a double-layer convolution residual block. In addition, group convolution is introduced in the SCAR block to further reduce the parameters while preventing over-fitting. Moreover, a multi-scale feature attention module is designed to provide instructive multi-scale attention information for shallow features. Particularly, we propose a novel upscale module, which adopts dual paths to upscale the features by jointly using sub-pixel convolution and nearest interpolation layers, instead of using deconvolution layer or sub-pixel convolution layer alone. The experimental results demonstrate that our method achieves comparable performance to the state-of-the-art methods, both quantitatively and qualitatively. [ABSTRACT FROM AUTHOR]

• Propose a new method for solving the hybrid regularization model. • An efficient hybrid regularization algorithm is developed. • Algorithms convergence, sparsity, and stability are proved theoretically. • Some efficient comparisons experiments are studied. Neural networks have been widely applied to expert and intelligent systems in the fields of business, fault diagnosis, and forecasting. Especially, random vector functional-link networks (RVFL), an important structure, have also been considerably studied and used in recent years. This paper addresses in the investigation of regularization model for RVFL, and proposes a hybrid regularization approach for RVFL by adding the ℓ 2 and ℓ 1 norm penalty terms simultaneously. A novel iterative learning algorithm is developed by using a fixed point contractive map. Further, some theoretical properties, including the convergence, sparsity, and stability of the proposed algorithm, are discussed and analyzed concretely under reasonable assumptions. The proposed method greatly improves the learner's sparsity and stability, guaranteeing the feasibility and effectiveness of network training. Experimental results on some benchmarks as well as face recognition database collected from the expert and intelligent systems, particularly the use of the statistical analysis strategy, verify the effectiveness and superiority of the proposed method, i.e. this new algorithm systematically outperforms the original RVFL and its variants in terms of the accuracy, sparsity, and stability of the solution. [ABSTRACT FROM AUTHOR]

The main purpose of negative correlation learning (NCL) is to produce ensembles with sound generalization capability through controlling the disagreement among base learners’ outputs. This paper uses neural networks with random weights (NNRWs) to implement such learning scheme in the study of face recognition. Particularly, two-dimensional (2D) feed-forward neural networks (2D-FNNs) with random weights (2D-NNRWs) are employed as base components, and which are incorporated with the NCL strategy for building neural network ensembles, where the basis functions of the base networks are generated randomly and the free parameters of the 2D-FNNs can be determined by solving a linear equation system. Also, an analytical solution is derived for these parameters. To examine the merits of the proposed algorithm, a series of comparative experiments are performed. The experimental results indicate that the proposed approach outperforms existing approaches. [ABSTRACT FROM AUTHOR]

Robust modelling is significant to deal with complex systems with uncertainties. This paper aims to develop a novel learning algorithm for training regularized local random weights networks (RWNs). The learner model, terms as RL-RWN, is built on regularized moving least squares method and generalizes the solution obtained from the standard least square technique. Simulations are carried out using two benchmark datasets, including Auto-MPG data and surface reconstruction data. Results demonstrate that our proposed RL-RWN outperforms the original RWN and radial basis function networks. [ABSTRACT FROM AUTHOR]

Abstract: Extreme learning machine (ELM) is one of the most popular and important learning algorithms. It comes from single-hidden-layer feedforward neural networks. It has been proved that ELM can achieve better performance than support vector machine (SVM) in regression and classification. In this paper, mathematically, with regression problem, the step 3 of ELM is studied. First of all, the equation are reformulated as an optimal model. With the optimality, the necessary conditions of optimal solution are presented. The equation is replaced by . We can prove that the latter must have one solution at least. Second, optimal approximation solution is discussed in cases of is column full rank, row full rank, neither column nor row full rank. In the last case, the rank-1 and rank-2 methods are used to get optimal approximation solution. In theory, this paper present a better algorithm for ELM. [Copyright &y& Elsevier]

Class imbalance is a challenge in image segmentation. Deep learning has demonstrated potential advantages as a promising approach in solving such challenging problems because of its powerful ability to extract features. To treat class imbalance in medical image segmentation effectively, in this study, deep learning technology is employed. Further, a new approach, referred to as multiscale fused network with additive channel–spatial attention (MSF-ACSA), is developed. In particular, using the U-shape convolutional network as the structure of the main body network, an additive channel–spatial attention (ACSA) module, which exploits high-level features to guide low-level features for optimal responses, is designed. Therefore, the proposed MSF-ACSA can availably focus on regions of interest while adaptively recalibrating the channel significance of feature maps. Simultaneously, a multiscale fusion with deep supervision module is introduced to incorporate fine-grained details with coarse-grained semantics from diverse scale features. Furthermore, a focal weighted Tversky loss function is proposed to mitigate class imbalance issues. Numerous experimental results indicate that the proposed MSF-ACSA achieves the best performance when compared with state-of-the-art approaches for medical image segmentation. [ABSTRACT FROM AUTHOR]

Recent advances in graph convolutional networks (GCNs), which mainly focus on how to exploit information from different hops of neighbors in an efficient way, have brought substantial improvement to many graph data modeling tasks. Most of the existing GCN-based models however are built on the basis of a fixed adjacency matrix, i.e., a single view topology of the underlying graph. That inherently limits the expressive power of the developed models especially when the raw graphs are often noisy or even incomplete due to the inevitably error-prone data measurement or collection. In this paper, we propose a novel framework, termed Multi-View Graph Convolutional Networks with Attention Mechanism (MAGCN), by incorporating multiple views of topology and an attention-based feature aggregation strategy into the computation of graph convolution. As an advanced variant of GCNs, MAGCN is fed with multiple "trustable" topologies, which already exist for a given task or are empirically generated by some classical graph construction methods, which has good potential to produce a better learning representation for downstream tasks. Furthermore, we present some theoretical analysis about the expressive power and flexibility of MAGCN, which provides a general explanation as to why multi-view based methods can potentially outperform those relying on a single view. Our experimental study demonstrates the state-of-the-art accuracies of MAGCN on Cora, Citeseer, and Pubmed datasets. Robustness analysis is also undertaken to show the advantage of MAGCN in handling some uncertainty issues in node classification tasks. [ABSTRACT FROM AUTHOR]

Graph neural networks (GNNs), with their promising potential to learn effective graph representation, have been widely used for recommender systems, in which the given graph data contains abundant users, items, and their historical interaction information. How to obtain preferable latent representations for both users and items is one of the key issues for GNN-based recommendation. This paper develops a novel deep GNN model with multi-graph attention fusion, MAF-GNN. This framework constructs two feature graph attention modules and a multi-scale latent features module, to generate better user and item latent features from input information. Specifically, the dual-branch residual graph attention (DBRGA) module is presented to extract neighbors' similar features from user and item graphs effectively and easily. Then multi-scale latent matrices are captured by applying non-linear transformations which are embedded to reduce the cost of dimension selection. Furthermore, a hybrid fusion graph attention (HFGA) module is designed to obtain valuable collaborative information from the user–item interaction graph, aiming to further refine the latent embedding of users and items. Finally, the whole MAF-GNN framework is optimized by a geometric factorized regularization loss. Extensive experiment results on both synthetic and real-world datasets illustrate that MAF-GNN can achieve better recommendation performance with a certain level of interpretability than some existing approaches. • A novel deep GNN model with multi-graph attention fusion mechanism is proposed. • A dual-branch residual graph attention module is developed. • A hybrid fusion graph attention module is designed to obtain valuable information. • Multi-scale latent matrices are designed to reduce significantly the time cost. [ABSTRACT FROM AUTHOR]