Search

This paper presents a new learning algorithm, Multi-Population Cooperative Particle Swarm Optimizer (MCPSO), for neural network training. MCPSO is based on a master-slave model, in which a population consists of a master group and several slave groups. The slave groups execute a single PSO or its variants independently to maintain the diversity of particles, while the master group evolves based on its own information and also the information of the slave groups. The particles both in the master group and the slave groups are co-evolved during the search process by employing a parameter, termed migration factor. The MCPSO is applied for training a multilayer feed-forward neural network, for three benchmark classification problems. The performance of MCPSO used for neural network training is compared to that of Back Propagation (BP), genetic algorithm (GA) and standard PSO (SPSO), demonstrating its effectiveness and efficiency. [ABSTRACT FROM AUTHOR]

Recently, Particle Swarm Optimization(PSO) has been widely applied for training neural network. To improve the performance of PSO for high-dimensional solution space which always occurs in training NN, this paper introduces a new paradigm of particle swarm optimization named stochastic PSO (S-PSO). The feature of the S-PSO is its high ability for exploration. Consequently, when swarm size is relatively small, S-PSO performs much better than traditional PSO in training of NN. Hence if S-PSO is used to realize training of NN, computational cost of training can be reduced significantly. [ABSTRACT FROM AUTHOR]

In this paper we propose a new stable learning algorithm for Cellular Neural Networks. Our approach is based on the input-to-state stability theory, so to obtain learning laws that do not need robust modifications. Here we present only a theoretical study, letting experimental evidences for further works. [ABSTRACT FROM AUTHOR]

In this paper, a node pruning algorithm based on optimal brain surgeon is proposed for feedforward neural networks. First, the neural network is trained to an acceptable solution using the standard training algorithm. After the training process, the orthogonal factorization is applied to the output of the nodes in the same hidden layer to identify and prune the dependant nodes. Then, a unit-based optimal brain surgeon(UB-OBS) pruning algorithm is proposed to prune the insensitive hidden units to further reduce the size of the neural network, and no retraining is needed. Simulations are presented to demonstrate the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

Elman presented a network with a context layer for the time-series processing. The context layer is connected to the hidden layer for the next calculation of the time series, which keeps the output of the hidden layer. In this paper, the context layer is reformed to the internal memory layer, which is connected from the hidden layer with the connection weights to make the internal memory. Then, the internal memory plays an important role of the learning of the time series. We developed a new learning algorithm, called the time-delayed back-propagation learning, for the internal memory. The ability of the network with the internal memory layer is demonstrated by applying the simple sinusoidal time-series. [ABSTRACT FROM AUTHOR]

In this paper, iterative learning control is presented to a dynamic neural network modelling process for nonlinear (stochastic) dynamical systems. When (B-spline) neural networks are used to model a nonlinear dynamical functional such as the output stochastic distributions in the repetitive processes or the batch processes, the parameters in the basis functions will be tuned using the generalized iterative learning control (GILC) scheme. The GILC scheme differs from the classical model-based ILC laws, with no dynamical input-output differential or difference equations given a prior. For the "model free" GILC problem, we propose an optimal design algorithm for the learning operators by introducing an augmented difference model in state space. A sufficient and necessary solvable condition can be given where a robust optimization solution with an LMI-based design algorithm can be provided. [ABSTRACT FROM AUTHOR]

In this paper, we consider the problem of combining the labeled and unlabeled examples to boost the performance of semi-supervised learning. We first define the label information graph, and then incorporate it with neighborhood graph. We propose a new regularized semi-supervised classification algorithm, in which the regularization term is based on this modified Graph Laplacian. According to the properties of Reproducing Kernel Hilbert Space (RKHS), the representer theorem holds, so the solution can be expressed by the Mercer kernel of examples. Experimental results show that our algorithm can use unlabeled and labeled examples effectively. [ABSTRACT FROM AUTHOR]

The eligibility trace is one of the basic mechanisms in reinforcement learning to handle delayed reward. In this paper, we have used meta-heuristic method to solve hard combinatorial optimization problems. Our proposed solution introduce Ant-Q learning method to solve Traveling Salesman Problem (TSP). The approach is based on population that use positive feedback as well as greedy search and suggest ant reinforcement learning algorithms using eligibility traces which is called replace-trace methods(Ant-TD(λ)). Although replacing traces are only slightly, they can produce a significant improvement in learning rate. We could know through an experiment that proposed reinforcement learning method converges faster to optimal solution than ACS and Ant-Q. [ABSTRACT FROM AUTHOR]

Self-organizing neural networks are typically associated with unsupervised learning. This paper presents a self-organizing neural architecture, known as TD-FALCON, that learns cognitive codes across multi-modal pattern spaces, involving states, actions, and rewards, and is capable of adapting and functioning in a dynamic environment with external evaluative feedback signals. We present a case study of TD-FALCON on a mine avoidance and navigation cognitive task, and illustrate its performance by comparing with a state-of-the-art reinforcement learning approach based on gradient descent backpropagation algorithm. [ABSTRACT FROM AUTHOR]

In Gaussian mixture (GM) modeling, it is crucial to select the number of Gaussians for a sample data set. In this paper, we propose a gradient entropy regularized likelihood (ERL) algorithm on Gaussian mixture to solve this problem under regularization theory. It is demonstrated by the simulation experiments that the gradient ERL learning algorithm can select an appropriate number of Gaussians automatically during the parameter learning on a sample data set and lead to a good estimation of the parameters in the actual Gaussian mixture, even in the cases of two or more actual Gaussians overlapped strongly. [ABSTRACT FROM AUTHOR]

In this paper, we present a new learning method called hyperbolic quotient feature map for competitive learning neural networks. The previous neural network learning algorithms didn't consider their topological properties, and thus their dynamics were not clearly defined. We show that the weight vectors obtained by competitive learning decompose the input vector space and map it to the quotient space X/R. In addition, we define a quotient function which maps [1, ∞) ⊂ Rn to [0,1) and induce the proposed algorithm from the performance measure with the quotient function. Experimental results for pattern recognition of remote sensing data indicate the superiority of the proposed algorithm in comparison to the conventional competitive learning methods. [ABSTRACT FROM AUTHOR]

In this paper, a new competitive learning algorithm called Dynamic Competitive Learning (DCL) is presented. DCL is a supervised learning method that dynamically generates output neurons and initializes automatically the weight vectors from training patterns. It introduces a new parameter called LOG (Limit of Grade) to decide whether an output neuron is created or not. If the class of at least one among LOG number of nearest output neurons is the same as the class of the present training pattern, then DCL adjusts the weight vector associated with the output neuron to learn the pattern. If the classes of all the nearest output neurons are different from the class of the training pattern, a new output neuron is created and the given training pattern is used to initialize the weight vector of the created neuron. The proposed method is significantly different from the previous competitive learning algorithms in the point that the selected neuron for learning is not limited only to the winner and the output neurons are dynamically generated during the learning process. In addition, the proposed algorithm has a small number of parameters, which are easy to be determined and applied to real-world problems. Experimental results demon- strate the superiority of DCL in comparison to the conventional competitive learning methods. [ABSTRACT FROM AUTHOR]

The rival penalized competitive learning (RPCL) algorithm has been developed to make the clustering analysis on a set of sample data in which the number of clusters is unknown, and recent theoretical analysis shows that it can be constructed by minimizing a special kind of cost function on the sample data. In this paper, we use the Mahalanobis distance instead of the Euclidean distance in the cost function computation and propose the Mahalanobis distance based rival penalized competitive learning (MDRPCL) algorithm. It is demonstrated by the experiments that the MDRPCL algorithm can be successful to determine the number of elliptical clusters in a data set and lead to a good classification result. [ABSTRACT FROM AUTHOR]

In this paper, vector field learning is proposed as a new application of manifold learning to vector field. We also provide a learning framework to extract significant features from vector data. Vector data containing position, direction and magnitude information is different from common point data only containing position information. The algorithm of locally linear embedding (LLE) is extended to deal with vector data. The learning ability of the extended version has been tested on synthetic data sets and experimental results demonstrate that the method is very helpful and promising. Manifold features of vector data obtained by learning methods can be used for next work such as classification, clustering, visualization, or segmentation of vectors. [ABSTRACT FROM AUTHOR]

In this paper, a learning algorithm for a single Quadratic Integrate-and-Fire Neuron (QIFN) is proposed and tested for various applications in which a multilayer perceptron neural network is conventionally used. It is found that a single QIFN is sufficient for the applications that require a number of neurons in different layers of a conventional neural network. Several benchmark and real-life problems of classification and function-approximation have been illustrated. [ABSTRACT FROM AUTHOR]

Quick extraction of the extremum eigenpairs of a real symmetric matrix is very important in engineering. Using neural networks to complete this operation is in a parallel manner and can achieve high performance. So, this paper proposes a very concise functional neural network (FNN) to compute the largest (or smallest) eigenvalue and one corresponding eigenvector. After transforming the FNN into a differential equation, and obtaining the analytic solution, the convergence properties are completely analyzed. By this FNN, the method that can compute the extremum eigenpairs whether the matrix is non-definite, positive definite or negative definite is designed. Finally, three examples show the validity. Comparing with the other ones used in the same field, the proposed FNN is very simple and concise, so it is very easy to realize. [ABSTRACT FROM AUTHOR]

This paper presents a technique that uses the Wang Recurrent Neural Network with the "Winner Takes All" principle to solve the Traveling Salesman Problem (TSP). When the Wang Neural Network presents solutions for the Assignment Problem with all constraints satisfied, the "Winner Takes All" principle is applied to the values in the Neural Network's decision variables, with the additional constraint that the new solution must form a feasible route for the TSP. The results from this new technique are compared to other heuristics, with data from the TSPLIB (TSP Library). The 2-opt local search technique is applied to the final solutions of the proposed technique and shows a considerable improvement of the results. [ABSTRACT FROM AUTHOR]

In this paper, we first review Chen's chaotic neural network model and then propose a novel wavelet chaotic neural network. Second,we apply them to search global minima of a continuous function, respectively. Meanwhile, the time evolution figures of the corresponding most positive Lyapunov exponent are given. Third, 10-city traveling salesman problem (TSP) is given to make a comparison between them. Finally we conclude that the novel wavelet chaotic neural network is more valid. [ABSTRACT FROM AUTHOR]

In this paper, a delayed Lagrangian network is presented for solving quadratic programming problems. Based on some known results, the delay interval is determined to guarantee the asymptotic stability of the delayed neural network at the optimal solution. One simulation example is provided to show the effectiveness of the approach. [ABSTRACT FROM AUTHOR]

This paper presents a novel recurrent time continuous neural network model which performs linear fractional optimization subject to bound constraints on each of the optimization variables. The network is proved to be complete in the sense that the set of optima of the objective function to be minimized with bound constraints coincides with the set of equilibria of the neural network. It is also shown that the network is primal and globally convergent in the sense that its trajectory cannot escape from the feasible region and will converge to an exact optimal solution for any initial point chosen in the feasible bound region. Simulation results are given to demonstrate further the global convergence and the good performance of the proposed neural network for linear fractional programming problems with bound constraints. [ABSTRACT FROM AUTHOR]

In this paper we present a model of autapses which are synapses connecting axons and dendrites of the same neuron and feeding back axonal action potentials to the own dendritic tree. The timely physiological self-inhibitory function they may serve, indicated by the spike-timing dependent plasticity rule, provides a potential negative feedback mechanism to control the dynamical properties of neurons. The model autapse is applied to a neuron with conductance-based minimal model to construct a fast-slow buster. Three types of bursting that the burster exhibits are analyzed geometrically through phase portraits to illustrate the modulation competence of the autapse model on bursting. [ABSTRACT FROM AUTHOR]

Complex dynamical behavior in a four-neuron recurrent neural network with discrete delays is investigated in this paper. The dissipativity of the system and stability of equilibrium point are studied by mens of Lyapunov theory. Stable fixed point, periodic and quasi-periodic orbits, and chaotic motion are observed in system via numerical calculation. With the change of the slope and threshold of activation function, as well as time delay and synaptic weight, the system passes from stable to periodic and then to chaotic regimes. Interestingly, the system returns to periodic or stable regimes by further changing these parameter values. Furthermore, some numerical evidences, such as phase portraits, bifurcation diagrams, power spectrum density, are given to confirm chaos. [ABSTRACT FROM AUTHOR]

In the paper, the orthogonal function neural network is utilized to realize the synchronization of chaotic system. The orthogonal function neural network is Legendred orthogonal neural network. First of all, the orthogonal function neural network is trained to learn the uncertain information. The parameters of Legendred orthogonal neural network are adjusted to accomplish the synchronization of two chaotic systems with the perturbation by Lyapunov steady theorem. At last, the result of numerical example is shown to illustrate the validity of the proposed method. [ABSTRACT FROM AUTHOR]

This paper studies the synchronization of a class of coupled discrete recurrent neural networks with time delay. The local and global conditions are obtained by using Lyapunov functional and linear matrix inequality methods. [ABSTRACT FROM AUTHOR]

This paper formulates and studies a model of impulsive coupled delayed neural networks. Based on stability theory of impulsive dynamical systems, a simple but less-conservative criterion is derived for global synchronization of such coupled neural networks. Furthermore, the theoretical result is applied to a typical chaotic delayed Hopfied neural networks, and is also illustrated by numerical simulations. [ABSTRACT FROM AUTHOR]

Multistable neural networks have attracted much interesting in recent years, since the monostable networks are computationally restricted. This paper studies a class of discrete-time two-neurons networks with unsaturating piecewise linear activation functions. Some interesting results for the convergence and the periodicity of solutions of the system are obtained. [ABSTRACT FROM AUTHOR]

In this paper, the control of Hopf bifurcations in BAM neural network with delayed self-feedback is presented. The asymptotic stability theorem and the relevant corollary for linearized nonlinear dynamical systems are stated. For BAM neural network with delayed self-feedback, a control model based on washout filter is proposed and analyzed. By applying the stability lemma, we investigate the stability of the control system and state the relevant theorem for choosing the parameters of the stabilized control system. Some numerical results are also given to illustrate the correctness of our results. [ABSTRACT FROM AUTHOR]

Based on the linear matrix inequality (LMI), new sufficient conditions on the global exponential stability and asymptotic stability of bidirectional associative memory neural networks with variable delay are presented, and exponential converging velocity index is estimated. Furthermore, the results in this paper are less conservative than the ones reported so far in the literature. One example is given to illustrate the feasibility of our main results. [ABSTRACT FROM AUTHOR]

The convergence of recurrent neural networks is known to be bases of successful applications of the networks. In this paper, the convergence of discrete delayed Hopfield neural networks is mainly investigated. One new sufficient and necessary condition for the delayed network having no one stable state is given. Also, several new sufficient conditions for the delayed networks converging towards a limit cycle with 2-period and with 4-period are respectively obtained. All results established here partly extend the previous results on the convergence of both discrete Hopfield neural networks and discrete delayed Hopfield neural networks in parallel updating mode. [ABSTRACT FROM AUTHOR]

This paper studies the problem of asymptotic stability of second-order discrete-time Hopfield neural networks with variable delays. By utilizing inequality techniques, we obtain sufficient conditions for the existence and asymptotic stability of an equilibrium point and estimate the region of existence and the attraction domain of the equilibrium point. A numerical example is given to illustrate our theoretical results. [ABSTRACT FROM AUTHOR]

This paper presents some new results for the absolute stability of Hopfield neural networks with activation functions chosen from sigmoidal functions which have unbounded derivatives. Detailed discussions are also given to the relation and difference of absolute stabilities between neural networks and Lurie systems with multiple nonlinear controls. Although the basic idea of the absolute stability of neural networks comes from that of Lurie control systems, it provides a very useful practical model for the study of Lurie control systems. [ABSTRACT FROM AUTHOR]

In this paper, we study impulsive fuzzy cellular neural networks. Criteria are obtained for the existence and exponential stability of a unique equilibrium of fuzzy cellular neural networks impulsive state displacements at fixed instants of time. [ABSTRACT FROM AUTHOR]

In this paper, the global exponential stability of fuzzy cellular neural networks with time-varying delays is studied. Without assuming the boundedness and differentiability of the activation functions, based on the properties of M-matrix, by constructing vector Liapunov functions and applying differential inequalities, the sufficient conditions ensuring existence, uniqueness, and global exponential stability of the equilibrium point of fuzzy cellular neural networks with variable delays are obtained. [ABSTRACT FROM AUTHOR]

In this paper, global exponential stability of cellular neural networks with time-varying delays and impulses is studied. By estimating the delay differential inequality with impulses, some sufficient conditions for global exponential stability of impulsive neural networks are obtained. The obtained results are new and they complement previously known results. Finally, an example is given to illustrate the theory. [ABSTRACT FROM AUTHOR]

In this paper, a sufficient condition is presented to ensure the complete stability of a cellular neural networks (CNNs) that output functions are a set of piecewise sigmoid nonlinear functions. The convergence theorem of the Gauss-Seidel method and Gauss-Seidel method, which is an iterative technique for solving a linear algebraic equation, plays an important role in our discussion. [ABSTRACT FROM AUTHOR]

In this paper, we investigate a large class of periodic Cohen-Grossberg neural networks and prove that under some mild conditions, there is a unique almost-periodic solution, which is globally stable. As special examples, we derive many results given in literature. [ABSTRACT FROM AUTHOR]

In this paper, the robust periodicity for recurrent neural networks with time delays and impulses is investigated. Based on Lyapunov method and fixed point theorem, a sufficient condition of global exponential robust stability of periodic solution is obtained. [ABSTRACT FROM AUTHOR]

This paper is devoted to global robust stability analysis of the general recurrent neural networks with time-varying parametric uncertainty and time-varying delays. To remove the dependence on the size of time-delays, Lyapunov-Razumikhin stability theorem and LMI approach are applied to derive the global robust stability conditions for the neural networks. Then delay-dependent global robust stability criteria are developed based on integrating Lyapunov-Krasovskii functional method and LMI approach. These stability criteria are in term of the solvability of linear matrix inequalities. [ABSTRACT FROM AUTHOR]

The problem of stochastic robust stability for Markovian jump discrete-time delayed neural networks with multiplicative nonlinear perturbation is investigated via Lyapunov stability theory in this paper. Based on the linear matrix inequality (LMI) methodology, a novel analysis approach is developed. The sufficient conditions of stochastically robust stable are given in terms of coupled linear matrix inequalities. The stable criteria represented in LMI setting are less conservative and more computationally efficient than the methods reported in the literature. [ABSTRACT FROM AUTHOR]

This paper deals with the stochastic robust stability problem for Markovian jump nonlinear uncertain neural networks (MJNUNNs) with Wiener process. Some criteria for stochastic robust stability of Markovian jump nonlinear uncertain neural networks are derived, even if the system contains Wiener process. All the derived results are presented in terms of linear matrix inequality. [ABSTRACT FROM AUTHOR]

Because of VLSI realization of artificial neural networks and measuring the elements of the circuits, noises coming from the circuits and the errors of the parameters of the network systems are therefore unavoidable. Making use of the stochastic version of Razumikhin theorem of stochastic functional differential equation, Lyapunov direct methods and matrix analysis,almost sure exponential stability on interval neural networks perturbed by white noises with time varying delays is examined, and some sufficient algebraic criteria which only depend on the systems' parameters are given. For well designed deterministic neural networks, the results obtained in the paper also imply that how much tolerance against perturbation they have. [ABSTRACT FROM AUTHOR]

In this paper the authors analyze problems of existence and global asymptotic stability of the equilibrium for the neural networks with mixed delays. Some new sufficient conditions ensuring the existence, uniqueness, and global asymptotic stability of the equilibrium are established by means of Leray-Schauder principle, arithmetic-mean-geometric-mean inequality and vector delay differential inequality technique. These conditions are less restrictive than previously known criteria. [ABSTRACT FROM AUTHOR]

The paper introduces a general class of neural networks with periodic inputs. By constructing a Lyapunov functional and the Halanay-type inequality separately, we obtain easily verifiable sufficient conditions ensuring that every solutions of the delayed neural networks converge exponentially to the unique periodic solutions. The results obtained can be regarded as a generalization to the discrete case of previous results. [ABSTRACT FROM AUTHOR]

In this paper, we essentially drop the requirement of Lipschitz condition on the activation functions. By employing Lyapunov functional and several new inequalities, some new criteria concerning global exponential stability for a class of generalized neural networks with time-varying delays are obtained, which only depend on physical parameters of neural networks. Since these new criteria do not require the activation functions to be differentiable, bounded or monotone nondecreasing and the connection weight matrices to be symmetric, they are mild and more general than previously known criteria. [ABSTRACT FROM AUTHOR]

In this paper, global exponential stability in Lagrange sense is further studied for continuous recurrent neural network with three different activation functions. According to the parameters of the system itself, detailed estimation of global exponential attractive set, and positive invariant set is presented without any hypothesis on existence. It is also verified that outside the global exponential attracting set; i.e., within the global attraction domain, there is no equilibrium point, periodic solution, almost periodic solution, and chaos attractor of the neural network. These theoretical analysis narrowed the search field of optimization computation and associative memories, provided convenience for application. [ABSTRACT FROM AUTHOR]

In this paper, a class of non-autonomous neural networks with distributed delays and reaction-diffusion terms is considered. Employing the properties of diffusion operator and the techniques of inequality, we investigate positive invariant set, global exponential stability, and then obtain the exponential dissipativity of the neural networks under consideration. Our results can extend and improve earlier ones. An example is given to demonstrate the effectiveness of these results. [ABSTRACT FROM AUTHOR]

In this paper, a new type of WNN is proposed to enhance the function approximation capability. In the proposed WNN, the nonlinear activation function is a linear combination of wavelets, that can be updated during the networks training process. As a result the approximate error is significantly decreased. The BP algorithm and the QR decomposition based training method for the proposed WNN is derived. The obtained results indicate that this new type of WNN exhibits excellent learning ability compared to the conventional ones. [ABSTRACT FROM AUTHOR]

There have been various studies on approximation ability of feedforward neural networks. The existing studies are, however, only concerned with the density or upper bound estimation on how a multivariate function can be approximated by the networks, and consequently, the essential approximation ability of networks cannot be revealed. In this paper, by establishing both upper and lower bound estimations on approximation order, the essential approximation ability of a class of feedforward neural networks with trigonometric hidden layer units is clarified in terms of the second order modulus of smoothness of approximated function. [ABSTRACT FROM AUTHOR]

Blood oxygenation level dependent (BOLD) contrast based functional magnetic resonance imaging (fMRI) has been widely utilized to detect brain neural activities and great efforts are now stressed on the hemodynamic processes of different brain regions activated by a stimulus. The focus of this paper is Gaussian dynamic convolution models of the fMRI BOLD response. The convolutions are between the perfusion function of the neural response to a stimulus and a Gaussian function. The parameters of the models are estimated by a nonlinear least-squares optimal algorithm for the fMRI data of eight subjects collected in a visual stimulus experiment. The results show that the Gaussian model is better in fitting the data. [ABSTRACT FROM AUTHOR]

Stochastic resonance phenomenon in a biological sensory system has been studied through signal detection theories and psychophysical experiments. There is a conflict between the real experiments and the traditional signal detection theory for stochastic resonance because the latter treats the receiver as linear model. This paper presents a two-layer summing network of Hodgkin-Huxley (HH) neurons and a summing network of threshold devices to model the receiver, respectively. The simulation results indicate that the relevant index of signal detectability exhibit the stochastic resonance characteristics. [ABSTRACT FROM AUTHOR]