Search

Recently, various convolutional neural networks (CNNs) based single image super-resolution (SR) methods have been vigorously explored, and a lot of impressive results have emerged. However, more or less unfortunately, most of the methods mainly focused on increasing the depth of network to improve reconstruction performance. As a matter of fact, deeper depth of network usually means an increase in parameters and computations, or worse still, the increase in parameters or computations often results in the difficulty to train the network. This paper develops a new SR approach called multi-scale residual channel attention network (MSRCAN), which is comparative shallow two-stage neural network structure, and can extract more details to effectively ameliorate the quality of SR. Specifically, a multi-scale residual channel attention block (MSRCAB) is designed to plenarily exploit the image features with convolutional kernels of different sizes. At the same time, a channel attention mechanism is introduced to recalibrate the channel significance of feature mappings adaptively. Furthermore, multiple short skip connections and a long skip connection are presented in each MSRCAB to complement information loss. Moreover, the two-stage design contributes to fully uncover low-level and high-level information. Evaluation on the benchmark data set indicates that the proposed method can rival the state-of-the-art convolutional methods. [ABSTRACT FROM AUTHOR]

More existing methods of single image super-resolution (SR) often direct super-resolving the details, but when the upsampling factor is larger, it is challenging to reconstruct high-frequency details. Lately, deep convolution neural networks have made significant progress with regard to SR. However, with an increase in networks width and depth, the information used for reconstruction is becoming increasingly weaker, and the training of neural networks is becoming more difficult. This paper proposes a novel architecture of a deep recursive convolution neural networks used to reconstruct a high-resolution image from an original low-resolution (LR) image in a step-by-step manner. The architecture consists of three parts: an embedding network, cascaded fine extraction blocks, and reconstruction networks. Concretely, a wide convolution is used to extract more features from the original LR images, cascaded fine extraction blocks are employed to extract more useful information through a step-by-step approach and remove redundant information, and a deconvolution operation is utilized to restore the features. The proposed networks adopt a residual-feature learning scheme, and the Caffe framework is chosen for training the networks. The experimental results show that the proposed method exhibits a superior performance compared with various other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

With the explosive growth in size of datasets, it becomes more significant to develop effective learning schemes for neural networks to deal with large scale data modelling. This paper proposes an iterative approximate Newton-type learning algorithm to build neural networks with random weights (NNRWs) for problem solving, where the whole training samples are divided into some small subsets under certain assumptions, and each subset is employed to construct a local learner model for integrating a unified classifier. The convergence of the output weights of the unified learner model is given. Experimental results on UCI datasets with comparisons demonstrate that the proposed algorithm is promising for large scale datasets. [ABSTRACT FROM AUTHOR]

Recovering the low-rank, sparse components of a given matrix is a challenging problem that arises in many real applications. Existing traditional approaches aimed at solving this problem are usually recast as a general approximation problem of a low-rank matrix. These approaches are based on the nuclear norm of the matrix, and thus in practice the rank may not be well approximated. This paper presents a new approach to solve this problem that is based on a new norm of a matrix, called the truncated nuclear norm (TNN). An efficient iterative scheme developed under the linearized alternating direction method multiple framework is proposed, where two novel iterative algorithms are designed to recover the sparse and low-rank components of matrix. More importantly, the convergence of the linearized alternating direction method multiple on our matrix recovering model is discussed and proved mathematically. To validate the effectiveness of the proposed methods, a series of comparative trials are performed on a variety of synthetic data sets. More specifically, the new methods are used to deal with problems associated with background subtraction (foreground object detection), and removing shadows and peculiarities from images of faces. Our experimental results illustrate that our new frameworks are more effective and accurate when compared with other methods. [ABSTRACT FROM AUTHOR]

In this paper, some feed-forward neural networks (FNNs) interpolation operators based on scattered data are introduced. Further, these operators are used as approximators to approximate bivariate continuous target function. By means of the translations and dilates of logistic function, some FNNs quasi-interpolation and exact interpolation operators are constructed, respectively. Using the modulus of continuity of function and the mesh norm of scattered data as measures, the corresponding approximation errors of the constructed operators are estimated. In addition, the well-known central B-splines are used to construct FNNs interpolation operators with compact support, and the corresponding approximation errors are also estimated. [ABSTRACT FROM AUTHOR]

Approximation capabilities of the spherical neural networks (SNNs) are considered in this paper. Based on a known Taylor formula, we prove that, for non-polynomial target function, rates of simultaneously approximating the function itself and its (Laplace–Beltrami) derivatives by SNNs is not slower than those by the spherical polynomials (SPs). Then, the simultaneous approximation rates of SPs automatically derive the rates of SNNs. [ABSTRACT FROM AUTHOR]

Abstract: The ranking problem is to learn a real-valued function which gives rise to a ranking over an instance space, which has gained much attention in machine learning in recent years. This article gives analysis of the convergence performance of neural networks ranking algorithm by means of the given samples and approximation property of neural networks. The upper bounds of convergence rate provided by our results can be considerably tight and independent of the dimension of input space when the target function satisfies some smooth condition. The obtained results imply that neural networks are able to adapt to ranking function in the instance space. Hence the obtained results are able to circumvent the curse of dimensionality on some smooth condition. [Copyright &y& Elsevier]

Abstract: We consider the optimal rate of approximation by single hidden feed-forward neural networks on the unit sphere. It is proved that there exists a neural network with neurons, and an analytic, strictly increasing, sigmoidal activation function such that the deviation of a Sobolev class from the class of neural networks , behaves asymptotically as . Namely, we prove that the essential rate of approximation by spherical neural networks is . [Copyright &y& Elsevier]

Abstract: Extreme learning machine (ELM), proposed by Huang et al., has been shown a promising learning algorithm for single-hidden layer feedforward neural networks (SLFNs). Nevertheless, because of the random choice of input weights and biases, the ELM algorithm sometimes makes the hidden layer output matrix H of SLFN not full column rank, which lowers the effectiveness of ELM. This paper discusses the effectiveness of ELM and proposes an improved algorithm called EELM that makes a proper selection of the input weights and bias before calculating the output weights, which ensures the full column rank of H in theory. This improves to some extend the learning rate (testing accuracy, prediction accuracy, learning time) and the robustness property of the networks. The experimental results based on both the benchmark function approximation and real-world problems including classification and regression applications show the good performances of EELM. [Copyright &y& Elsevier]

Abstract: There have been many studies on the simultaneous approximation capability of feed-forward neural networks (FNNs). Most of these, however, are only concerned with the density or feasibility of performing simultaneous approximations. This paper considers the simultaneous approximation of algebraic polynomials, employing Taylor expansion and an algebraic constructive approach, to construct a class of FNNs which realize the simultaneous approximation of any smooth multivariate function and all of its derivatives. We also present an upper bound on the approximation accuracy of the FNNs, expressed in terms of the modulus of continuity of the functions to be approximated. [Copyright &y& Elsevier]

Abstract: It is well-known that single hidden layer feed-forward neural networks (SLFNs) with at most n hidden neurons can learn n distinct samples with zero error, and the weights connecting the input neurons and the hidden neurons and the hidden node thresholds can be chosen randomly. Namely, for n distinct samples, there exist SLFNs with n hidden neurons that interpolate them. These networks are called exact interpolation networks for the samples. However, for some approximated target functions (as continuous or integrable functions) not all exact interpolation networks have good approximation effect. This paper, by using a functional approach, rigorously proves that for given distinct samples there exists an SLFN which not only exactly interpolates samples but also near best approximates the target function. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we construct two types of feed-forward neural networks (FNNs) which can approximately interpolate, with arbitrary precision, any set of distinct data in the metric space. Firstly, for analytic activation function, an approximate interpolation FNN is constructed in the metric space, and the approximate error for this network is deduced by using Taylor formula. Secondly, for a bounded sigmoidal activation function, exact interpolation and approximate interpolation FNNs are constructed in the metric space. Also the error between the exact and approximate interpolation FNNs is given. [Copyright &y& Elsevier]

Abstract: There have been many studies on the simultaneous approximation capability of feed-forward neural networks (FNNs). Most of these, however, are only concerned with the density or feasibility of performing simultaneous approximations. This paper considers the simultaneous approximation of algebraic polynomials, employing Taylor expansion and an algebraic constructive approach, to construct a class of FNNs which realize the simultaneous approximation of any smooth univariate function and all its derivatives. We also present an upper bound on the approximation accuracy of the FNNs, expressed in terms of the modulus of continuity of the functions to be approximated. The results obtained in this paper reveal the complexity of simultaneous approximation by FNNs. [Copyright &y& Elsevier]

Abstract: Two classes of feedforward neural networks (FNNs) with one hidden layer are constructed to approximate integrable functions in this paper. We not only show that the constructed FNNs can approximate any arbitrarily in the metric as long as the number of hidden nodes is sufficiently large, but also reveal the relation among the approximation speed, the number of hidden nodes and the smoothness of the target function to be approximated by designing a novel method, which is originated from the Steklov mean function and the modulus of smoothness of . The obtained results are helpful in studying the problem of approximation complexity of FNNs in the metric. [Copyright &y& Elsevier]

Abstract: This paper presents a type of feedforward neural networks (FNNs), which can be used to approximately interpolate, with arbitrary precision, any set of distinct data in multidimensional Euclidean spaces. They can also uniformly approximate any continuous functions of one variable or two variables. By using the modulus of continuity of function as metric, the rates of convergence of approximate interpolation networks are estimated, and two Jackson-type inequalities are established. [Copyright &y& Elsevier]

Abstract: Neural networks are widely used in many applications including astronomical physics, image processing, recognition, robotics and automated target tracking, etc. Their ability to approximate arbitrary functions is the main reason for this popularity. The main result of this paper is a constructive proof of a formula for the upper bound of the approximation error by feedforward neural networks with one hidden layer of sigmoidal units and a linear output. The result can also be used to estimate complexity of the maximum error network. An example to demonstrate the theoretical result is given. [Copyright &y& Elsevier]

• Filters are visualized by the activation maximization to explain functions of filters. • DNNs are compressed based on the visualization results. • The redundant filters are measured based on the color and texture similarities. • The repetitive and invalid filters can be pruned by optimization. This paper proposes a method to compress deep neural networks (DNNs) based on interpretability. For a trained DNN model, the activation maximization technique is first used to visualize every filter of the DNN model. Then, a single-layer filter pruning approach is introduced from what is learned by visualization. The entire DNN model is compressed layer by layer by using the single-layer filter pruning method in which the compression of the current layer is based on the compression of the preceding layers. Importantly, in addition to effective compression, the proposed method renders a better interpretation of the deep learning process. With a 60 % compression rate of the VGG-16, our method achieves 0.8429 Top-1 accuracy under CIFAR-10, with a slight accuracy drop of only 0.0322, and the storage space of the model can be compressed to 9.42 Mb. For a modern DNN model such as ResNet50, our visualization-based filter pruning method is significantly better than other pruning strategies in different convolutional layers under different compression rates and the larger ImageNet dataset. After pruning, the computation cost and storage requirement of the DNN can be significantly reduced, which means that complex DNN models can be easily implemented in small mobile devices, thus enabling the efficient use of DNNs in the Internet of Things technologies. [ABSTRACT FROM AUTHOR]

In recent years, deep neural networks have exhibited numerous advantages in hyperspectral image classification (HIC). However, owing to the limited number of training samples of hyperspectral images (HSIs), the network structure should not be designed too deep to retard the overfitting phenomenon. This study proposes a cascaded dual-scale crossover network for HIC, which not only could extract rich features, but also does not make the network deeper. It continuously connects two different cascaded dual-scale crossover blocks, and automatically extracts the spectral–spatial features of HSIs. Moreover, for the limited training samples, the proposed network could flexibly capture more discriminant contextual features by using different spectral-size and spatial-size convolution kernels. Furthermore, two different cross-merge methods are designed to improve the information flow and contrast of the images to obtain parts of interest for the images. Two skip structures are also used for alleviating overfitting and accelerating the network training. Additional experimental results on some datasets, including Indian Pines, Kennedy Space Center, and University of Pavia, verify the feasibility of the proposed network. Namely, the classification accuracy of the proposed network is superior to that of other existing networks. [ABSTRACT FROM AUTHOR]

The main purpose of negative correlation learning (NCL) is to produce ensembles with sound generalization capability through controlling the disagreement among base learners’ outputs. This paper uses neural networks with random weights (NNRWs) to implement such learning scheme in the study of face recognition. Particularly, two-dimensional (2D) feed-forward neural networks (2D-FNNs) with random weights (2D-NNRWs) are employed as base components, and which are incorporated with the NCL strategy for building neural network ensembles, where the basis functions of the base networks are generated randomly and the free parameters of the 2D-FNNs can be determined by solving a linear equation system. Also, an analytical solution is derived for these parameters. To examine the merits of the proposed algorithm, a series of comparative experiments are performed. The experimental results indicate that the proposed approach outperforms existing approaches. [ABSTRACT FROM AUTHOR]

Abstract: Face recognition is always a hot topic in the field of pattern recognition and computer vision. Generally, images or features are often converted into vectors in the process of recognition. This method usually results in the distortion of correlative information of the elements in the vectorization of an image matrix. This paper designs a classifier called two dimensional neural network with random weights (2D-NNRW) which can use matrix data as direct input, and can preserve the image matrix structure. Specifically, the proposed classifier employs left and right projecting vectors to replace the usual high dimensional input weight in the hidden layer to keep the correlative information of the elements, and adopts the idea of neural network with random weights (NNRW) to learn all the parameters. Experiments on some famous databases validate that the proposed classifier 2D-NNRW can embody the structural character of the face image and has good performance for face recognition. [Copyright &y& Elsevier]

Although deep neural networks have shown good performance on grid data, it is challenging to design deep neural networks for point cloud processing due to the irregular domain and disordering of point cloud. This paper presents a novel convolution block named edge-point features mixed convolution (EPFM-Conv) which effectively integrates the graph-based method and the point-based strategy. The entire EPFM-Conv block constructs the edge branch and point branch to extract the local and global features of point cloud, respectively. In the edge branch, we first construct the local dynamic graphs on point cloud and thereby extract rich detailed features through edge convolution. Next, the features of the edge are further abstracted and aggregated through the grouped residual learning followed by a symmetric function. In the point branch, we directly extract the features of the point by a point-wise multi-layer perceptron. Finally, the features of edge and point are mixed adaptively to obtain the feature representation of point cloud. Compared with the existing methods, the main difference is that the proposed method dexterously constructs the local dynamic graphs on point cloud for extracting rich detailed features, and synchronously designs the edge and point branches to extract the local and global features. Meanwhile, the proposed method uses the grouped residual learning to improve the information flow, thereby effectively deepening the depth of the network. Extensive experiments demonstrate that the proposed EPFM-Conv achieves the excellent performances on shape classification as well as part segmentation tasks. [ABSTRACT FROM AUTHOR]

Highlights • Propose a new super-resolution method based on CSC and CNN. • Use CNN to reconstruct the high resolution image for the low-frequency part. • Propose an ACSC to reconstruct high resolution image for high-frequency part. • Integrate and develope the advantages of the CSC method and CNN. Abstract The convolutional sparse coding-based super-resolution (CSC-SR) method has shown its good performance in single image super-resolution. It divides the low-resolution (LR) image into low-frequency part and the high-frequency part, and reconstructs their corresponding high-resolution (HR) image with bicubic interpolation and convolutional sparse coding (CSC) method, respectively. This paper is devoted to improve the performance of CSC-SR method. As convolutional neural network (CNN) can reveal the mapping relation between the LR image and the HR image for the low-frequency part better, we replace the bicubic interpolation with CNN to reconstruct the HR image for the low-frequency part. In addition, we propose an adaptive CSC method to reconstruct the HR image for the high-frequency part. We name our proposed super-resolution method as hybrid adaptive convolutional sparse coding-based super-resolution (HACSC-SR) method. Many comparison experiments illustrate that our proposed HACSC-SR method is superior to CSC-SR, CNN as well as several existing super-resolution methods. [ABSTRACT FROM AUTHOR]

There have been a lot of methods to address the recognition of complete face images. However, in real applications, the images to be recognized are usually incomplete, and it is more difficult to realize such a recognition. In this paper, a novel convolution neural network frame, named a low-rank-recovery network (LRRNet), is proposed to conquer the difficulty effectively inspired by matrix completion and deep learning techniques. The proposed LRRNet first recovers the incomplete face images via an approach of matrix completion with the truncated nuclear norm regularization solution, and then extracts some low-rank parts of the recovered images as the filters. With these filters, some important features are obtained by means of the binaryzation and histogram algorithms. Finally, these features are classified with the classical support vector machines (SVMs). The proposed LRRNet method has high face recognition rate for the heavily corrupted images, especially for the images in the large databases. The proposed LRRNet performs well and efficiently for the images with heavily corrupted, especially in the case of large databases. Extensive experiments on several benchmark databases demonstrate that the proposed LRRNet performs better than some other excellent robust face recognition methods. [ABSTRACT FROM AUTHOR]