Search

In safety critical applications, control architectures based on adaptive neural networks (NNs) must satisfy strict design specifications. This paper presents a practical approach for designing a mixed linear/adaptive model reference controller that recovers the performance of a reference model, and guarantees the boundedness of the tracking error within an a priori specified compact domain, in the presence of bounded uncertainties. The linear part of the controller results from the solution of an optimization problem where specifications are expressed as linear matrix inequality constraints. The linear controller is then augmented with a general adaptive NN that compensates for the uncertainties. The only requirement for the NN is that its output must be confined within pre-specified saturation limits. Toward this end a specific NN output confinement algorithm is proposed in this paper. The main advantages of the proposed approach are that requirements in terms of worst-case performance can be easily defined during the design phase, and that the design of the adaptation mechanism is largely independent from the synthesis of the linear controller. A numerical example is used to illustrate the design methodology.

Most partitioning algorithms iteratively partition a space into cells that contain underlying linear or nonlinear structures using linear partitioning strategies. The compactness of each cell depends on how well the (locally) linear partitioning strategy approximates the intrinsic structure. To partition a compact structure for complex data in a nonlinear context, this paper proposes a nonlinear partition strategy. This is a principal curve tree (PC-tree), which is implemented iteratively. Given that a PC passes through the middle of the data distribution, it allows for partitioning based on the arc length of the PC. To enhance the partitioning of a given space, a residual version of the PC-tree algorithm is developed, denoted here as the principal component analysis tree (PCR-tree) algorithm. Because of its residual property, the PCR-tree can yield the intrinsic dimension of high-dimensional data. Comparisons presented in this paper confirm that the proposed PC-tree and PCR-tree approaches show a better performance than several other competing partitioning algorithms in terms of vector quantization error and nearest neighbor search. The comparison also shows that the proposed algorithms outperform competing linear methods in total average coverage which measures the nonlinear compactness of partitioning algorithms.

This paper is concerned with the problem of exponential stability analysis of continuous-time switched delayed neural networks. By using the average dwell time approach together with the piecewise Lyapunov function technique and by combining a novel Lyapunov-Krasovskii functional, which benefits from the delay partitioning method, with the free-weighting matrix technique, sufficient conditions are proposed to guarantee the exponential stability for the switched neural networks with constant and time-varying delays, respectively. Moreover, the decay estimates are explicitly given. The results reported in this paper not only depend upon the delay but also depend upon the partitioning, which aims at reducing the conservatism. Numerical examples are presented to demonstrate the usefulness of the derived theoretical results.

This paper presents a subgradient-based neural network to solve a nonsmooth nonconvex optimization problem with a nonsmooth nonconvex objective function, a class of affine equality constraints, and a class of nonsmooth convex inequality constraints. The proposed neural network is modeled with a differential inclusion. Under a suitable assumption on the constraint set and a proper assumption on the objective function, it is proved that for a sufficiently large penalty parameter, there exists a unique global solution to the neural network and the trajectory of the network can reach the feasible region in finite time and stay there thereafter. It is proved that the trajectory of the neural network converges to the set which consists of the equilibrium points of the neural network, and coincides with the set which consists of the critical points of the objective function in the feasible region. A condition is given to ensure the convergence to the equilibrium point set in finite time. Moreover, under suitable assumptions, the coincidence between the solution to the differential inclusion and the ldquoslow solutionrdquo of it is also proved. Furthermore, three typical examples are given to present the effectiveness of the theoretic results obtained in this paper and the good performance of the proposed neural network.

The self-organizing network (SON)-based multiple model system is a recently proposed method for identifying the dynamics of a general nonlinear system. It has been observed by researchers that cooperative learning among neighboring regions is sometimes important for the success of identification of a nonlinear system under the multiple model system framework. In this paper, we intend to formally evaluate the effects of cooperative learning and topology preservation in identification of multiple model system based on SON. The results of the mathematical analysis supports the heuristic that a good learning strategy for identifying the local model parameters of a SON-based multiple model system is to choose a neighborhood function whose effective region is initially wider and is reduced gradually during learning. An example of nonlinear function approximation is also provided at the end of this paper to demonstrate the results of the mathematical analysis.

As is well known in statistics, the resulting linear regressors by using the rank-based Wilcoxon approach to linear regression problems are usually robust against (or insensitive to) outliers. This motivates us to introduce in this paper the Wilcoxon approach to the area of machine learning. Specifically, we investigate four new learning machines, namely Wilcoxon neural network (WNN), Wilcoxon generalized radial basis function network (WGRBFN), Wilcoxon fuzzy neural network (WFNN), and kernel-based Wilcoxon regressor (KWR). These provide alternative learning machines when faced with general nonlinear learning problems. Simple weights updating rules based on gradient descent will be derived. Some numerical examples will be provided to compare the robustness against outliers for various learning machines. Simulation results show that the Wilcoxon learning machines proposed in this paper have good robustness against outliers. We firmly believe that the Wilcoxon approach will provide a promising methodology for many machine learning problems.

This paper deals with adaptive tracking for discrete-time multiple-input-multiple-output (MIMO) nonlinear systems in presence of bounded disturbances. In this paper, a high-order neural network (HONN) structure is used to approximate a control law designed by the backstepping technique, applied to a block strict feedback form (BSFF). This paper also includes the respective stability analysis, on the basis of the Lyapunov approach, for the whole controlled system, including the extended Kalman filter (EKF)-based NN learning algorithm. Applicability of the scheme is illustrated via simulation for a discrete-time nonlinear model of an electric induction motor.

The existing approaches to the discrete-time nonlinear output regulation problem rely on the offline solution of a set of mixed nonlinear functional equations known as discrete regulator equations. For complex nonlinear systems, it is difficult to solve the discrete regulator equations even approximately. Moreover, for systems with uncertainty, these approaches cannot offer a reliable solution. By combining the approximation capability of the feedforward neural networks (NNs) with an online parameter optimization mechanism, we develop an approach to solving the discrete nonlinear output regulation problem without solving the discrete regulator equations explicitly. The approach of this paper can be viewed as a discrete counterpart of our previous paper on approximately solving the continuous-time nonlinear output regulation problem.

The inverse controller is traditionally assumed to be a deterministic function. This paper presents a pedagogical methodology for estimating the stochastic model of the inverse controller. The proposed method is based on Bayes' theorem. Using Bayes' rule to obtain the stochastic model of the inverse controller allows the use of knowledge of uncertainty from both the inverse and the forward model in estimating the optimal control signal. The paper presents the methodology for general nonlinear systems and is demonstrated on nonlinear single-input-single-output (SISO) and multiple-input-multiple-output (MIMO) examples.

In this paper, an adaptive wavelet-network-based control approach is proposed for highly nonlinear uncertain dynamical systems. Wavelet network, as a kind of universal approximator, has two novel properties--orthonormality and multiresolution. The orthonormal property ensures that adding a new resolution (new wavelets) does not affect the existing wavelet network that may have been well tuned. In the sequel, the online adjustment of the structure of the nonlinear adaptive wavelet controller (AWC) can be done in a constructive manner by gradually increasing the network resolution. The multiresolution property, on the other hand, assures a guaranteed improvement of the approximation precision when a new resolution is added. In real life problems we are unable to know the adequate size of a network, either a neural network (NN) or a wavelet network, to produce the required approximation precision. By virtue of the novel wavelet network properties, a coarse or very simple structure can be selected first. If the system fails to converge after the elapse of a dwell time, a new wavelet resolution is considered to be necessary and added directly. In this manner, the AWC can be easily constructed and tuned from the coarse to finer levels until the performance requirement is satisfied. The trial and error way of selecting the network structure, which may lead to either an inadequate or a highly redundant structure, can be avoided. In this paper, the proposed adaptive wavelet network is applied first to a class of nonlinear dynamical systems with a partially known model and an affine-in-input structure. Then, the adaptive wavelet network is applied to a class of nonlinear nonaffine dynamical systems.

In pattern recognition, feature extraction techniques are widely employed to reduce the dimensionality of data and to enhance the discriminatory information. Principal component analysis (PCA) and linear discriminant analysis (LDA) are the two most popular linear dimensionality reduction methods. However, PCA is not very effective for the extraction of the most discriminant features, and LDA is not stable due to the small sample size problem . In this paper, we propose some new (linear and nonlinear) feature extractors based on maximum margin criterion (MMC). Geometrically, feature extractors based on MMC maximize the (average) margin between classes after dimensionality reduction. It is shown that MMC can represent class separability better than PCA. As a connection to LDA, we may also derive LDA from MMC by incorporating some constraints. By using some other constraints, we establish a new linear feature extractor that does not suffer from the small sample size problem, which is known to cause serious stability problems for LDA. The kernelized (nonlinear) counterpart of this linear feature extractor is also established in the paper. Our extensive experiments demonstrate that the new feature extractors are effective, stable, and efficient.

In this paper, we consider a new periodic activation function for the multivalued neuron (MVN). The MVN is a neuron with complex-valued weights and inputs/output, which are located on the unit circle. Although the MVN outperforms many other neurons and MVN-based neural networks have shown their high potential, the MVN still has a limited capability of learning highly nonlinear functions. A periodic activation function, which is introduced in this paper, makes it possible to learn nonlinearly separable problems and non-threshold multiple-valued functions using a single multivalued neuron. We call this neuron a multivalued neuron with a periodic activation function (MVN-P). The MVN-Ps functionality is much higher than that of the regular MVN. The MVN-P is more efficient in solving various classification problems. A learning algorithm based on the error-correction rule for the MVN-P is also presented. It is shown that a single MVN-P can easily learn and solve those benchmark classification problems that were considered unsolvable using a single neuron. It is also shown that a universal binary neuron, which can learn nonlinearly separable Boolean functions, and a regular MVN are particular cases of the MVN-P.

Recurrent neural networks for solving constrained least absolute deviation (LAD) problems or L(1)-norm optimization problems have attracted much interest in recent years. But so far most neural networks can only deal with some special linear constraints efficiently. In this paper, two neural networks are proposed for solving LAD problems with various linear constraints including equality, two-sided inequality and bound constraints. When tailored to solve some special cases of LAD problems in which not all types of constraints are present, the two networks can yield simpler architectures than most existing ones in the literature. In particular, for solving problems with both equality and one-sided inequality constraints, another network is invented. All of the networks proposed in this paper are rigorously shown to be capable of solving the corresponding problems. The different networks designed for solving the same types of problems possess the same structural complexity, which is due to the fact these architectures share the same computing blocks and only differ in connections between some blocks. By this means, some flexibility for circuits realization is provided. Numerical simulations are carried out to illustrate the theoretical results and compare the convergence rates of the networks.

In this paper, we consider delayed bidirectional associative memory (BAM) neural networks (NNs) with Lipschitz continuous activation functions. By applying Young's inequality and Hoelder's inequality techniques together with the properties of monotonic continuous functions, global exponential stability criteria are established for BAM NNs with time delays. This is done through the use of a new Lyapunov functional and an M-matrix. The results obtained in this paper extend and improve previous results.

Adaptive approximation-based control typically uses approximators with a predefined set of basis functions. Recently, spatially dependent methods have defined self-organizing approximators where new locally supported basis elements were incorporated when existing basis elements were insufficiently excited. In this paper, performance-dependent self-organizing approximators will be defined. The designer specifies a positive tracking error criteria. The self-organizing approximation-based controller then monitors the tracking performance and adds basis elements only as needed to achieve the tracking specification. The method of this paper is applicable to general nth-order input-state feedback linearizable systems. This paper includes a complete stability analysis and a detailed simulation example.

In this paper, we study cellular neural networks with almost periodic variable coefficients and time-varying delays. By using the existence theorem of almost periodic solution for general functional differential equations, introducing many real parameters and applying the Lyapunov functional method and the technique of Young inequality, we obtain some sufficient conditions to ensure the existence, uniqueness, and global exponential stability of almost periodic solution. The results obtained in this paper are new, useful, and extend and improve the existing ones in previous literature.

The convergence of Oja's principal component analysis (PCA) learning algorithms is a difficult topic for direct study and analysis. Traditionally, the convergence of these algorithms is indirectly analyzed via certain deterministic continuous time (DCT) systems. Such a method will require the learning rate to converge to zero, which is not a reasonable requirement to impose in many practical applications. Recently, deterministic discrete time (DDT) systems have been proposed instead to indirectly interpret the dynamics of the learning algorithms. Unlike DCT systems, DDT systems allow learning rates to be constant (which can be a nonzero). This paper will provide some important results relating to the convergence of a DDT system of Oja's PCA learning algorithm. It has the following contributions: 1) A number of invariant sets are obtained, based on which we can show that any trajectory starting from a point in the invariant set will remain in the set forever. Thus, the nondivergence of the trajectories is guaranteed. 2) The convergence of the DDT system is analyzed rigorously. It is proven, in the paper, that almost all trajectories of the system starting from points in an invariant set will converge exponentially to the unit eigenvector associated with the largest eigenvalue of the correlation matrix. In addition, exponential convergence rate are obtained, providing useful guidelines for the selection of fast convergence learning rate. 3) Since the trajectories may diverge, the careful choice of initial vectors is an important issue. This paper suggests to use the domain of unit hyper sphere as initial vectors to guarantee convergence. 4) Simulation results will be furnished to illustrate the theoretical results achieved.

In this paper, an observer-based direct adaptive fuzzy-neural control scheme is presented for nonaffine nonlinear systems in the presence of unknown structure of nonlinearities. A direct adaptive fuzzy-neural controller and a class of generalized nonlinear systems, which are called nonaffine nonlinear systems, are instead of the indirect one and affine nonlinear systems given by Leu et al. By using implicit function theorem and Taylor series expansion, the observer-based control law and the weight update law of the fuzzy-neural controller are derived for the nonaffine nonlinear systems. Based on strictly-positive-real (SPR) Lyapunov theory, the stability of the closed-loop system can be verified. Moreover, the overall adaptive scheme guarantees that all signals involved are bounded and the output of the closed-loop system will asymptotically track the desired output trajectory. To demonstrate the effectiveness of the proposed method, simulation results are illustrated in this paper.

In this paper, we analyze and establish the stability and convergence of the dynamical system proposed by Xia and Feng, whose equilibria solve variational inequality and related problems. Under the pseudo-monotonicity and other conditions, this system is proved to be stable in the sense of Lyapunov and converges to one of its equilibrium points for any starting point. Meanwhile, the global exponential stability of this system is also shown under some mild conditions without the strong monotonicity of the mapping. The obtained results improve and correct some existing ones. The validity and performance of this system are demonstrated by some numerical examples.

In this paper, the direct neural dynamic programming technique is utilized to solve the Hamilton-Jacobi-Bellman equation forward-in-time for the decentralized near optimal regulation of a class of nonlinear interconnected discrete-time systems with unknown internal subsystem and interconnection dynamics, while the input gain matrix is considered known. Even though the unknown interconnection terms are considered weak and functions of the entire state vector, the decentralized control is attempted under the assumption that only the local state vector is measurable. The decentralized nearly optimal controller design for each subsystem consists of two neural networks (NNs), an action NN that is aimed to provide a nearly optimal control signal, and a critic NN which evaluates the performance of the overall system. All NN parameters are tuned online for both the NNs. By using Lyapunov techniques it is shown that all subsystems signals are uniformly ultimately bounded and that the synthesized subsystems inputs approach their corresponding nearly optimal control inputs with bounded error. Simulation results are included to show the effectiveness of the approach.

Regularization path algorithms have been proposed to deal with model selection problem in several machine learning approaches. These algorithms allow computation of the entire path of solutions for every value of regularization parameter using the fact that their solution paths have piecewise linear form. In this paper, we extend the applicability of regularization path algorithm to a class of learning machines that have quadratic loss and quadratic penalty term. This class contains several important learning machines such as squared hinge loss support vector machine (SVM) and modified Huber loss SVM. We first show that the solution paths of this class of learning machines have piecewise nonlinear form, and piecewise segments between two breakpoints are characterized by a class of rational functions. Then we develop an algorithm that can efficiently follow the piecewise nonlinear path by solving these rational equations. To solve these rational equations, we use rational approximation technique with quadratic convergence rate, and thus, our algorithm can follow the nonlinear path much more precisely than existing approaches such as predictor-corrector type nonlinear-path approximation. We show the algorithm performance on some artificial and real data sets.

In this paper, we have three goals: the first is to delineate the advantages of a variably delayed system, the second is to find a more intuitive Lyapunov function for a delayed neural network, and the third is to design a delayed neural network for a quadratic cost function. For delayed neural networks, most researchers construct a Lyapunov function based on the linear matrix inequality (LMI) approach. However, that approach is not intuitive. We provide a alternative candidate Lyapunov function for a delayed neural network. On the other hand, if we are first given a quadratic cost function, we can construct a delayed neural network by suitably dividing the second-order term into two parts: a self-feedback connection weight and a delayed connection weight. To demonstrate the advantage of a variably delayed neural network, we propose a transiently chaotic neural network with variable delay and show numerically that the model should possess a better searching ability than Chen-Aihara's model, Wang's model, and Zhao's model. We discuss both the chaotic and the convergent phases. During the chaotic phase, we simply present bifurcation diagrams for a single neuron with a constant delay and with a variable delay. We show that the variably delayed model possesses the stochastic property and chaotic wandering. During the convergent phase, we not only provide a novel Lyapunov function for neural networks with a delay (the Lyapunov function is independent of the LMI approach) but also establish a correlation between the Lyapunov function for a delayed neural network and an objective function for the traveling salesman problem.

This paper investigates a neural network (NN) state observer-based adaptive control for a class of time-varying delays nonlinear systems with unknown control direction. An adaptive neural memoryless observer, in which the knowledge of time-delay is not used, is designed to estimate the system states. Furthermore, by applying the property of the function tanh(2)(ϑ/ε)/ϑ (the function can be defined at ϑ = 0) and introducing a novel type appropriate Lyapunov-Krasovskii functional, an adaptive output feedback controller is constructed via backstepping method which can efficiently avoid the problem of controller singularity and compensate for the time-delay. It is highly proven that the closed-loop systems controller designed by the NN-basis function property, new kind parameter adaptive law and Nussbaum function in detecting the control direction is able to guarantee the semi-global uniform ultimate boundedness of all signals and the tracking error can converge to a small neighborhood of zero. The characteristic of the proposed approach is that it relaxes any restrictive assumptions of Lipschitz condition for the unknown nonlinear continuous functions. And the proposed scheme is suitable for the systems with mismatching conditions and unmeasurable states. Finally, two simulation examples are given to illustrate the effectiveness and applicability of the proposed approach.

Local model networks approximate a nonlinear system through multiple local models fitted within a partition space. The main advantage of this approach is that the identification of complex nonlinear processes is alleviated by the integration of structured knowledge about the process. This paper extends these concepts by the integration of quantitative process knowledge into the identification procedure. Quantitative knowledge describes explicit dependences between inputs and outputs and is integrated in the parameter estimation process by means of equality constraints. For this purpose, a constrained generalized total least squares algorithm for local parameter estimation is presented. Furthermore, the problem of proper integration of constraints in the partitioning process is treated where an expectation-maximization procedure is combined with constrained parameter estimation. The benefits and the applicability of the proposed concepts are demonstrated by means of two illustrative examples and a practical application using real measurement data.

In blind source separation, many methods have been proposed to estimate the mixing matrix by exploiting sparsity. However, they often need to know the source number a priori, which is very inconvenient in practice. In this paper, a new method, namely nonlinear projection and column masking (NPCM), is proposed to estimate the mixing matrix. A major advantage of NPCM is that it does not need any knowledge of the source number. In NPCM, the objective function is based on a nonlinear projection and its maxima just correspond to the columns of the mixing matrix. Thus a column can be estimated first by locating a maximum and then deflated by a masking operation. This procedure is repeated until the evaluation of the objective function decreases to zero dramatically. Thus the mixing matrix and the number of sources are estimated simultaneously. Because the masking procedure may result in some small and useless local maxima, particle swarm optimization (PSO) is introduced to optimize the objective function. Feasibility and efficiency of PSO are also discussed. Comparative experimental results show the efficiency of NPCM, especially in the cases where the number of sources is unknown and the sources are relatively less sparse.

In this paper, we study the finite-horizon optimal control problem for discrete-time nonlinear systems using the adaptive dynamic programming (ADP) approach. The idea is to use an iterative ADP algorithm to obtain the optimal control law which makes the performance index function close to the greatest lower bound of all performance indices within an ε-error bound. The optimal number of control steps can also be obtained by the proposed ADP algorithms. A convergence analysis of the proposed ADP algorithms in terms of performance index function and control policy is made. In order to facilitate the implementation of the iterative ADP algorithms, neural networks are used for approximating the performance index function, computing the optimal control policy, and modeling the nonlinear system. Finally, two simulation examples are employed to illustrate the applicability of the proposed method.

This paper presents adaptive neural tracking control for a class of non-affine pure-feedback systems with multiple unknown state time-varying delays. To overcome the design difficulty from non-affine structure of pure-feedback system, mean value theorem is exploited to deduce affine appearance of state variables x(i) as virtual controls α(i), and of the actual control u. The separation technique is introduced to decompose unknown functions of all time-varying delayed states into a series of continuous functions of each delayed state. The novel Lyapunov-Krasovskii functionals are employed to compensate for the unknown functions of current delayed state, which is effectively free from any restriction on unknown time-delay functions and overcomes the circular construction of controller caused by the neural approximation of a function of u and [Formula: see text] . Novel continuous functions are introduced to overcome the design difficulty deduced from the use of one adaptive parameter. To achieve uniformly ultimate boundedness of all the signals in the closed-loop system and tracking performance, control gains are effectively modified as a dynamic form with a class of even function, which makes stability analysis be carried out at the present of multiple time-varying delays. Simulation studies are provided to demonstrate the effectiveness of the proposed scheme.

In this paper, a class of impulsive interval neural networks with discrete and distributed time-varying delays is discussed. Several new sufficient conditions are obtained ensuring the existence, uniqueness, and global exponential stability of periodic solution (i.e., stationary oscillation) for the addressed models based on inequality analysis techniques. The obtained results can be checked easily by the linear matrix inequality control toolbox in MATLAB. Finally, two numerical examples are given to show the effectiveness of our results.

Smooth hinging hyperplane (SHH) has been proposed as an improvement over the well-known hinging hyperplane (HH) by the fact that it retains the useful features of HH while overcoming HH's drawback of nondifferentiability. This paper introduces a formal characterization of smooth hinge function (SHF), which can be used to generate SHH as a neural network. A method for the general construction of SHF is also given. Furthermore, the work proves that SHH is better than HH in functional approximation, i.e., the optimal error of SHH approximating a general function is always smaller or equal to that of HH. Particularly, in the case that the SHF is generated via the integration of a class of sigmoidal functions, it is further proven that the corresponding SHH of the 2m SHFs would outperform a neural network with m of the sigmoidal function from which the SHF is derived. Any upper bound established on the approximation error of a neural network of m sigmoidal activation functions can hence be translated to the SHH of m SHFs by replacing m with [m/2]. The work also includes an algorithm for the identification of SHH making use of its differentiability property. Simulation experiments are presented to validate the theoretical conclusions to possible extent.

The weight-decay technique is an effective approach to handle overfitting and weight fault. For fault-free networks, without an appropriate value of decay parameter, the trained network is either overfitted or underfitted. However, many existing results on the selection of decay parameter focus on fault-free networks only. It is well known that the weight-decay method can also suppress the effect of weight fault. For the faulty case, using a test set to select the decay parameter is not practice because there are huge number of possible faulty networks for a trained network. This paper develops two mean prediction error (MPE) formulae for predicting the performance of faulty radial basis function (RBF) networks. Two fault models, multiplicative weight noise and open weight fault, are considered. Our MPE formulae involve the training error and trained weights only. Besides, in our method, we do not need to generate a huge number of faulty networks to measure the test error for the fault situation. The MPE formulae allow us to select appropriate values of decay parameter for faulty networks. Our experiments showed that, although there are small differences between the true test errors (from the test set) and the MPE values, the MPE formulae can accurately locate the appropriate value of the decay parameter for minimizing the true test error of faulty networks.

This paper develops a probabilistic approach to recursive second-order training of recurrent neural networks (RNNs) for improved time-series modeling. A general recursive Bayesian Levenberg-Marquardt algorithm is derived to sequentially update the weights and the covariance (Hessian) matrix. The main strengths of the approach are a principled handling of the regularization hyperparameters that leads to better generalization, and stable numerical performance. The framework involves the adaptation of a noise hyperparameter and local weight prior hyperparameters, which represent the noise in the data and the uncertainties in the model parameters. Experimental investigations using artificial and real-world data sets show that RNNs equipped with the proposed approach outperform standard real-time recurrent learning and extended Kalman training algorithms for recurrent networks, as well as other contemporary nonlinear neural models, on time-series modeling.

The Adachi neural network (AdNN) is a fascinating neural network (NN) which has been shown to possess chaotic properties, and to also demonstrate associative memory (AM) and pattern recognition (PR) characteristics. Variants of the AdNN have also been used to obtain other PR phenomena, and even blurring. An unsurmountable problem associated with the AdNN and the variants referred to above is that all of them require a quadratic number of computations. This is essentially because the NNs in each case are completely connected graphs. In this paper, we consider how the computations can be significantly reduced by merely using a linear number of computations. To achieves this, we extract from the original completely connected graph one of its spanning trees. We then address the problem of computing the weights for this spanning tree. This is done in such a manner that the modified tree-based NN has approximately the same input-output characteristics, and thus the new weights are themselves calculated using a gradient-based algorithm. By a detailed experimental analysis, we show that the new linear-time AdNN-like network possesses chaotic and PR properties for different settings. As far as we know, such a tree-based AdNN has not been reported, and the results given here are novel.

State estimation for uncertain systems affected by external noises is an important problem in control theory. This paper deals with a state observation problem when the dynamic model of a plant contains uncertainties or it is completely unknown. Differential neural network (NN) approach is applied in this uninformative situation but with activation functions described by wavelets. A new learning law, containing an adaptive adjustment rate, is suggested to imply the stability condition for the free parameters of the observer. Nominal weights are adjusted during the preliminary training process using the least mean square (LMS) method. Lyapunov theory is used to obtain the upper bounds for the weights dynamics as well as for the mean squared estimation error. Two numeric examples illustrate this approach: first, a nonlinear electric system, governed by the Chua's equation and second the Lorentz oscillator. Both systems are assumed to be affected by external perturbations and their parameters are unknown.

An adaptive Fourier neural network (AFNN) control scheme is presented in this paper for the control of a class of uncertain nonlinear systems. Based on Fourier analysis and neural network (NN) theory, AFNN employs orthogonal complex Fourier exponentials as the activation functions. Due to the clear physical meaning of the neurons, the determination of the AFNN structure as well as the parameters of the activation functions becomes convenient. One salient feature of the proposed AFNN approach is that all the nonlinearities and uncertainties of the dynamical system are lumped together and compensated online by AFNN. It can, therefore, be applied to uncertain nonlinear systems without any a priori knowledge about the system dynamics. Derived from Lyapunov theory, a novel learning algorithm is proposed, which is essentially a frequency domain method and can guarantee asymptotic stability of the closed-loop system. The simulation results of a multiple-input-multiple-output (MIMO) nonlinear system and the experimental results of an X - Y positioning table are presented to show the effectiveness of the proposed AFNN controller.

This paper studies the problem of stability analysis for neural networks (NNs) with a time-varying delay. Unlike the previous works, the activation functions are assumed to be neither monotonic, nor differentiable, nor bounded. By defining a more general type of Lyapunov functionals, some new less conservative delay-dependent stability criteria are established in terms of linear matrix inequalities (LMIs). Meanwhile, the computational complexity of the newly obtained stability conditions is reduced because less variables are involved. Numerical examples are given to illustrate the effectiveness and the benefits of the proposed method.

This paper considers the design of robust neural network tracking controllers for nonlinear systems. The neural network is used in the closed-loop system to estimate the nonlinear system function. We introduce the conic sector theory to establish a robust neural control system, with guaranteed boundedness for both the input/output (I/O) signals and the weights of the neural network. The neural network is trained by the simultaneous perturbation stochastic approximation (SPSA) method instead of the standard backpropagation (BP) algorithm. The proposed neural control system guarantees closed-loop stability of the estimation system, and a good tracking performance. The performance improvement of the proposed system over existing systems can be quantified in terms of preventing weight shifts, fast convergence, and robustness against system disturbance.

This paper presents a Galerkin/neural-network- based guaranteed cost control (GCC) design for a class of parabolic partial differential equation (PDE) systems with unknown nonlinearities. A parabolic PDE system typically involves a spatial differential operator with eigenspectrum that can be partitioned into a finite-dimensional slow one and an infinite-dimensional stable fast complement. Motivated by this, in the proposed control scheme, Galerkin method is initially applied to the PDE system to derive an ordinary differential equation (ODE) system with unknown nonlinearities, which accurately describes the dynamics of the dominant (slow) modes of the PDE system. The resulting nonlinear ODE system is subsequently parameterized by a multilayer neural network (MNN) with one-hidden layer and zero bias terms. Then, based on the neural model and a Lure-type Lyapunov function, a linear modal feedback controller is developed to stabilize the closed-loop PDE system and provide an upper bound for the quadratic cost function associated with the finite-dimensional slow system for all admissible approximation errors of the network. The outcome of the GCC problem is formulated as a linear matrix inequality (LMI) problem. Moreover, by using the existing LMI optimization technique, a suboptimal guaranteed cost controller in the sense of minimizing the cost bound is obtained. Finally, the proposed design method is applied to the control of the temperature profile of a catalytic rod.

In this paper, we propose a powerful symmetric radial basis function (RBF) classifier for nonlinear detection in the so-called "overloaded" multiple-antenna-aided communication systems. By exploiting the inherent symmetry property of the optimal Bayesian detector, the proposed symmetric RBF classifier is capable of approaching the optimal classification performance using noisy training data. The classifier construction process is robust to the choice of the RBF width and is computationally efficient. The proposed solution is capable of providing a signal-to-noise ratio (SNR) gain in excess of 8 dB against the powerful linear minimum bit error rate (BER) benchmark, when supporting four users with the aid of two receive antennas or seven users with four receive antenna elements.

In this paper, we use complex analytic functions to achieve independent component analysis (ICA) by maximization of non-Gaussianity and introduce the complex maximization of non-Gaussianity (CMN) algorithm. We derive both a gradient-descent and a quasi-Newton algorithm that use the full second-order statistics providing superior performance with circular and noncircular sources as compared to existing methods. We show the connection among ICA methods through maximization of non-Gaussianity, mutual information, and maximum likelihood (ML) for the complex case, and emphasize the importance of density matching for all three cases. Local stability conditions are derived for the CMN cost function that explicitly show the effects of noncircularity on convergence and demonstrated through simulation examples.

It is well known that least absolute deviation (LAD) criterion or L(1)-norm used for estimation of parameters is characterized by robustness, i.e., the estimated parameters are totally resistant (insensitive) to large changes in the sampled data. This is an extremely useful feature, especially, when the sampled data are known to be contaminated by occasionally occurring outliers or by spiky noise. In our previous works, we have proposed the least absolute deviation neural network (LADNN) to solve unconstrained LAD problems. The theoretical proofs and numerical simulations have shown that the LADNN is Lyapunov-stable and it can globally converge to the exact solution to a given unconstrained LAD problem. We have also demonstrated its excellent application value in time-delay estimation. More generally, a practical LAD application problem may contain some linear constraints, such as a set of equalities and/or inequalities, which is called constrained LAD problem, whereas the unconstrained LAD can be considered as a special form of the constrained LAD. In this paper, we present a new neural network called constrained least absolute deviation neural network (CLADNN) to solve general constrained LAD problems. Theoretical proofs and numerical simulations demonstrate that the proposed CLADNN is Lyapunov stable and globally converges to the exact solution to a given constrained LAD problem, independent of initial values. The numerical simulations have also illustrated that the proposed CLADNN can be used to robustly estimate parameters for nonlinear curve fitting, which is extensively used in signal and image processing.

This paper studies the problems of global exponential stability for impulsive high-order Hopfield-type neural networks (NNs) with time-varying delays. By employing the Lyapunov-Razumikhin technique, some criteria ensuring global exponential stability are derived. Our results are then used to obtain some sufficient conditions under which some NNs can be forced to converge by impulsive control. Numerical examples are also discussed to illustrate our results.

In this paper, a neuroadaptive control framework for continuous- and discrete-time nonlinear uncertain dynamical systems with input-to-state stable internal dynamics is developed. The proposed framework is Lyapunov based and unlike standard neural network (NN) controllers guaranteeing ultimate boundedness, the framework guarantees partial asymptotic stability of the closed-loop system, that is, asymptotic stability with respect to part of the closed-loop system states associated with the system plant states. The neuroadaptive controllers are constructed without requiring explicit knowledge of the system dynamics other than the assumption that the plant dynamics are continuously differentiable and that the approximation error of uncertain system nonlinearities lie in a small gain-type norm bounded conic sector. This allows us to merge robust control synthesis tools with NN adaptive control tools to guarantee system stability. Finally, two illustrative numerical examples are provided to demonstrate the efficacy of the proposed approach.

This paper proposes a self-organizing adaptive fuzzy neural control (SAFNC) via sliding-mode approach for a class of nonlinear systems. The proposed SAFNC system is comprised of a computation controller and a supervisory controller. The computation controller including a self-organizing fuzzy neural network (SOFNN) identifier is the principal controller. The SOFNN identifier is used to online estimate the controlled system dynamics with the structure and parameter learning phases of fuzzy neural network (FNN), simultaneously. The structure learning phase possesses the ability of online generation and elimination of fuzzy rules to achieve optimal neural structure, and the parameter learning phase adjusts the interconnection weights of neural network to achieve favorable approximation performance. The supervisory controller is used to achieve the L2-norm bound tracking performance with a desired attenuation level. Moreover, all the parameter learning algorithms are derived based on Lyapunov function candidate, thus the system stability can be guaranteed. Finally, simulation results show that the SAFNC can achieve favorable tracking performances.

A neural control synthesis method is considered for a class of nonaffine uncertain single-input-single-output (SISO) systems. The method eliminates a fixed-point assumption and does not assume boundedness on the time derivative of a control effectiveness term. One or the other of these assumptions exist in earlier papers on this subject. Using Lyapunov's direct method, it is shown that all the signals of the closed-loop system are uniformly ultimately bounded, and that the tracking error converges to an adjustable neighborhood of the origin. Simulation with a Van Der Pol equation with nonaffine control terms illustrates the approach.

This paper considers the problems of global exponential stability and exponential convergence rate for impulsive high-order Hopfield-type neural networks with time-varying delays. By using the method of Lyapunov functions, some sufficient conditions for ensuring global exponential stability of these networks are derived, and the estimated exponential convergence rate is also obtained. As an illustration, an numerical example is worked out using the results obtained.

This paper considers the problem of robust exponential /spl epsiv/-regulation for a class of multi-input nonlinear systems with uncertainties. The uncertainties appear not only in the feedback channel but also in the control channel. Under some mild assumptions, an adaptive neural network control scheme is developed such that all the signals of the closed-loop system are semiglobally uniformly ultimately bounded and, under the control scheme with initial data starting in some compact set, the states of the closed-loop system is guaranteed to exponentially converge to an arbitrarily specified /spl epsi/-neighborhood about the origin. The important contributions of the present work are that a new exponential uniformly ultimately bounded performance is proposed and that the design parameters and initial condition set can be determined easily. The development generalizes and improves earlier results for the single-input case.

In this paper, an adaptive neural controller for a class of time-delay nonlinear systems with unknown nonlinearities is proposed. Based on a wavelet neural network (WNN) online approximation model, a state feedback adaptive controller is obtained by constructing a novel integral-type Lyapunov-Krasovskii functional, which also efficiently overcomes the controller singularity problem. It is shown that the proposed method guarantees the semiglobal boundedness of all signals in the adaptive closed-loop systems. An example is provided to illustrate the application of the approach.

We consider a class of nonlinear models based on mixtures of local autoregressive time series. At any given time point, we have a certain number of linear models, denoted as experts, where the vector of covariates may include lags of the dependent variable. Additionally, we assume the existence of a latent multinomial variable, whose distribution depends on the same covariates as the experts, that determines which linear process is observed. This structure, denoted as mixture-of-experts (ME), is considerably flexible in modeling the conditional mean function, as shown by Jiang and Tanner. In this paper, we present a formal treatment of conditions to guarantee the asymptotic normality of the maximum likelihood estimator (MLE), under stationarity and nonstationarity, and under correct model specification and model misspecification. The performance of common model selection criteria in selecting the number of experts is explored via Monte Carlo simulations. Finally, we present applications to simulated and real data sets, to illustrate the ability of the proposed structure to model not only the conditional mean, but also the whole conditional density.

This paper presents a novel nonlinear control strategy for a class of uncertain single-input and single-output discrete-time nonlinear systems with unstable zero-dynamics. The proposed method combines adaptive-network-based fuzzy inference system (ANFIS) with multiple models, where a linear robust controller, an ANFIS-based nonlinear controller and a switching mechanism are integrated using multiple models technique. It has been shown that the linear controller can ensure the boundedness of the input and output signals and the nonlinear controller can improve the dynamic performance of the closed loop system. Moreover, it has also been shown that the use of the switching mechanism can simultaneously guarantee the closed loop stability and improve its performance. As a result, the controller has the following three outstanding features compared with existing control strategies. First, this method relaxes the assumption of commonly-used uniform boundedness on the unmodeled dynamics and thus enhances its applicability. Second, since ANFIS is used to estimate and compensate the effect caused by the unmodeled dynamics, the convergence rate of neural network learning has been increased. Third, a “one-to-one mapping” technique is adapted to guarantee the universal approximation property of ANFIS. The proposed controller is applied to a numerical example and a pulverizing process of an alumina sintering system, respectively, where its effectiveness has been justified.

This paper presents a one-layer recurrent neural network for solving a class of constrained nonsmooth optimization problems with piecewise-linear objective functions. The proposed neural network is guaranteed to be globally convergent in finite time to the optimal solutions under a mild condition on a derived lower bound of a single gain parameter in the model. The number of neurons in the neural network is the same as the number of decision variables of the optimization problem. Compared with existing neural networks for optimization, the proposed neural network has a couple of salient features such as finite-time convergence and a low model complexity. Specific models for two important special cases, namely, linear programming and nonsmooth optimization, are also presented. In addition, applications to the shortest path problem and constrained least absolute deviation problem are discussed with simulation results to demonstrate the effectiveness and characteristics of the proposed neural network.