Search

Recently, the focus of functional connectivity analysis of human brain has shifted from merely revealing the inter-regional functional correlation over the entire scan duration to capturing the time-varying information of brain networks and characterizing time-resolved reoccurring patterns of connectivity. Much effort has been invested into developing approaches that can track changes in re-occurring patterns of functional connectivity over time. In this paper, we propose a sparse deep dictionary learning method to characterize the essential differences of reoccurring patterns of time-varying functional connectivity between different age groups. The proposed method combines both the interpretability of sparse dictionary learning and the capability of extracting sparse nonlinear higher-level features in the latent space of sparse deep autoencoder. In other words, it learns a sparse dictionary of the original data by considering the nonlinear representation of the data in the encoder layer based on a sparse deep autoencoder. In this way, the nonlinear structure and higher-level features of the data can be captured by deep dictionary learning. The proposed method is applied to the analysis of the Philadelphia Neurodevelopmental Cohort. It shows that there exist essential differences in the reoccurrence patterns of function connectivity between child and young adult groups. Specially, children have more diffusive functional connectivity patterns while young adults possess more focused functional connectivity patterns, and the brain function transits from undifferentiated systems to specialized neural networks with the growth., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2020 Elsevier Ltd. All rights reserved.)

Extreme learning machines (ELMs) have been proposed for generalized single-hidden-layer feedforward networks which need not be neuron alike and perform well in both regression and classification applications. The problem of determining the suitable network architectures is recognized to be crucial in the successful application of ELMs. This paper first proposes a dynamic ELM (D-ELM) where the hidden nodes can be recruited or deleted dynamically according to their significance to network performance, so that not only the parameters can be adjusted but also the architecture can be self-adapted simultaneously. Then, this paper proves in theory that such D-ELM using Lebesgue p-integrable hidden activation functions can approximate any Lebesgue p-integrable function on a compact input set. Simulation results obtained over various test problems demonstrate and verify that the proposed D-ELM does a good job reducing the network size while preserving good generalization performance.

The online backpropagation (BP) training procedure has been extensively explored in scientific research and engineering applications. One of the main factors affecting the performance of the online BP training is the learning rate. This paper proposes a new dynamic learning rate which is based on the estimate of the minimum error. The global convergence theory of the online BP training procedure with the proposed learning rate is further studied. It is proved that: 1) the error sequence converges to the global minimum error; and 2) the weight sequence converges to a fixed point at which the error function attains its global minimum. The obtained global convergence theory underlies the successful applications of the online BP training procedure. Illustrative examples are provided to support the theoretical analysis.

Extreme learning machines (ELMs) have been proposed for generalized single-hidden-layer feedforward networks which need not be neuron-like and perform well in both regression and classification applications. In this brief, we propose an ELM with adaptive growth of hidden nodes (AG-ELM), which provides a new approach for the automated design of networks. Different from other incremental ELMs (I-ELMs) whose existing hidden nodes are frozen when the new hidden nodes are added one by one, in AG-ELM the number of hidden nodes is determined in an adaptive way in the sense that the existing networks may be replaced by newly generated networks which have fewer hidden nodes and better generalization performance. We then prove that such an AG-ELM using Lebesgue p-integrable hidden activation functions can approximate any Lebesgue p-integrable function on a compact input set. Simulation results demonstrate and verify that this new approach can achieve a more compact network architecture than the I-ELM.

The backpropogation (BP) neural networks have been widely applied in scientific research and engineering. The success of the application, however, relies upon the convergence of the training procedure involved in the neural network learning. We settle down the convergence analysis issue through proving two fundamental theorems on the convergence of the online BP training procedure. One theorem claims that under mild conditions, the gradient sequence of the error function will converge to zero (the weak convergence), and another theorem concludes the convergence of the weight sequence defined by the procedure to a fixed value at which the error function attains its minimum (the strong convergence). The weak convergence theorem sharpens and generalizes the existing convergence analysis conducted before, while the strong convergence theorem provides new analysis results on convergence of the online BP training procedure. The results obtained reveal that with any analytic sigmoid activation function, the online BP training procedure is always convergent, which then underlies successful application of the BP neural networks.

Simultaneous approximation of a function and its derivatives are required in many science and engineering applications. There have been many studies on the simultaneous approximation capability of feedforward neural networks (FNNs). Most of the studies are, however, only concerned with density or feasibility of performing simultaneous approximation with FNNs, and no quantitative estimation on approximation accuracy of the simultaneous approximation is given. Moreover, all existing density or feasibility results are established in the uniform metric only, and provide no solution to topology specification of the FNNs used. In this paper, by means of the Bernstein-Durrmeyer operator, a class of FNNs is constructed which realize the simultaneous approximation of any smooth multivariate function and all its existing partial derivatives. We present, by making use of multivariate approximation tools, a quantitative upper bound estimation on approximation accuracy of the simultaneous approximation of the FNNs in terms of the modulus of smoothness of the functions to be approximated. The obtained results reveals that the approximation speed of the constructed FNNs depends not only on the number of hidden units used, but also on the smoothness of functions to be approximated.

The neuron state modeling and the local field modeling provides two fundamental modeling approaches to neural network research, based on which a neural network system can be called either as a static neural network model or as a local field neural network model. These two models are theoretically compared in terms of their trajectory transformation property, equilibrium correspondence property, nontrivial attractive manifold property, global convergence as well as stability in many different senses. The comparison reveals an important stability invariance property of the two models in the sense that the stability (in any sense) of the static model is equivalent to that of a subsystem deduced from the local field model when restricted to a specific manifold. Such stability invariance property lays a sound theoretical foundation of validity of a useful, cross-fertilization type stability analysis methodology for various neural network models.

As the health impact of air pollutants existing in ambient addresses much attention in recent years, forecasting of air pollutant parameters becomes an important and popular topic in environmental science. Airborne pollution is a serious, and will be a major problem in Hong Kong within the next few years. In Hong Kong, Respirable Suspended Particulate (RSP) and Nitrogen Oxides NOx and NO2 are major air pollutants due to the dominant diesel fuel usage by public transportation and heavy vehicles. Hence, the investigation and prediction of the influence and the tendency of these pollutants are of significance to public and the city image. The multi-layer perceptron (MLP) neural network is regarded as a reliable and cost-effective method to achieve such tasks. The works presented here involve developing an improved neural network model, which combines the principal component analysis (PCA) technique and the radial basis function (RBF) network, and forecasting the pollutant levels and tendencies based in the recorded data. In the study, the PCA is firstly used to reduce and orthogonalize the original input variables (data), these treated variables are then used as new input vectors in RBF neural network model established for forecasting the pollutant tendencies. Comparing with the general neural network models, the proposed model possesses simpler network architecture, faster training speed, and more satisfactory predicting performance. This improved model is evaluated by using hourly time series of RSP, NOx and NO2 concentrations collected at Mong Kok Roadside Gaseous Monitory Station in Hong Kong during the year 2000. By comparing the predicted RSP. NOx and NO2 concentrations with the actual data of these pollutants recorded at the monitory station, the effectiveness of the proposed model has been proven. Therefore, in authors' opinion, the model presented in the paper is a potential tool in forecasting air quality parameters and has advantages over the traditional neural network methods.

The stability of neural networks is a prerequisite for successful applications of the networks as either associative memories or optimization solvers. Because the integration and communication delays are ubiquitous, the stability of neural networks with delays has received extensive attention. However, the approach used in the previous investigation is mainly based on Liapunov's direct method. Since the construction of Liapunov function is very skilful, there is little compatibility among the existing results. In this paper, we develop a new approach to stability analysis of Hopfield-type neural networks with time-varying delays by defining two novel quantities of nonlinear function similar to the matrix norm and the matrix measure, respectively. With the new approach, we present sufficient conditions of the stability, which are either the generalization of those existing or new. The developed approach may be also applied for any general system with time delays rather than Hopfield-type neural networks.