Search

Time-varying (TV) nonlinear systems widely exist in various fields of engineering and science. Effective identification and modeling of TV systems is a challenging problem due to the nonstationarity and nonlinearity of the associated processes. In this paper, a novel parametric modeling algorithm is proposed to deal with this problem based on a TV nonlinear autoregressive with exogenous input (TV-NARX) model. A new class of multiple beta wavelet (MBW) basis functions is introduced to represent the TV coefficients of the TV-NARX model to enable the tracking of both smooth trends and sharp changes of the system behavior. To produce a parsimonious model structure, a locally regularized ultraorthogonal forward regression (LRUOFR) algorithm aided by the adjustable prediction error sum of squares (APRESS) criterion is investigated for sparse model term selection and parameter estimation. Simulation studies and a real application to EEG data show that the proposed MBW-LRUOFR algorithm can effectively capture the global and local features of nonstationary systems and obtain an optimal model, even for signals contaminated with severe colored noise.

In model identification, the existence of uncertainty normally generates negative impact on the accuracy and performance of the identified models, especially when the size of data used is rather small. With a small data set, least squares estimates are biased, the resulting models may not be reliable for further analysis and future use. This study introduces a novel robust model structure selection method for model identification. The proposed method can successfully reduce the model structure uncertainty and therefore improve the model performances. Case studies on simulation data and real data are presented to illustrate how the proposed metric works for robust model identification.

This paper is concerned with the model selection and model averaging\ud problems in system identification and data-driven modelling for nonlinear\ud systems. Given a set of data, the objective of model selection is to evaluate\ud a series of candidate models and determine which one best presents the data.\ud Three commonly used criteria, namely, Akaike information criterion (AIC),\ud Bayesian information criterion (BIC) and an adjustable prediction error sum\ud of squares (APRESS) are investigated and their performance in model\ud selection and model averaging is evaluated via a number of case studies\ud using both simulation and real data. The results show that APRESS\ud produces better models in terms of generalization performance and model\ud complexity.

In this paper, an accurate parameter estimation model of surface permanent magnet synchronous machines (SPMSMs) is established by taking into account voltage-source-inverter (VSI) nonlinearity. A fast dynamic particle swarm optimization (DPSO) algorithm combined with a receptor editing (RE) strategy is proposed to explore the optimal values of parameter estimations. This combination provides an accelerated implementation on graphics processing unit (GPU), and the proposed method is, therefore, referred to as G-DPSO-RE. In G-DPSO-RE, a dynamic labor division strategy is incorporated into the swarms according to the designed evolutionary factor during the evolution process. Two novel modifications of the movement equation are designed to update the velocity of particles. Moreover, a chaotic-logistic-based immune RE operator is developed to facilitate the global best individual ( gBest particle) to explore a potentially better region. Furthermore, a GPU parallel acceleration technique is utilized to speed up parameter estimation procedure. It has been demonstrated that the proposed method is effective for simultaneous estimation of the PMSM parameters and the disturbance voltage ( $V_{\mathrm { {dead}}})$ due to VSI nonlinearity from experimental data for currents and rotor speed measured with inexpensive equipment. The influence of the VSI nonlinearity on the accuracy of parameter estimation is analyzed.

Objective: This study proposes a new parametric TF (time frequency) CGC (conditional Granger causality) method for high precision connectivity analysis over time and frequency in multivariate coupling nonstationary systems, and applies it to scalp and source EEG signals to reveal dynamic interaction patterns in oscillatory neocortical sensorimotor networks. Methods: The Geweke spectral measure is combined with the TVARX (time varying autoregressive with exogenous input) modelling approach, which uses multiwavelets and ultra regularized orthogonal least squares (UROLS) algorithm aided by APRESS (adjustable prediction error sum of squares), to obtain high resolution time varying CGC representations. The UROLS APRESS algorithm, which adopts both the regularization technique and the ultra least squares criterion to measure not only the signal data themselves but also the weak derivatives of them, is a novel powerful method in constructing time varying models with good generalization performance, and can accurately track smooth and fast changing causalities. The generalized measurement based on CGC decomposition is able to eliminate indirect influences in multivariate systems. Results: The proposed method is validated on two simulations and then applied to multichannel motor imagery (MI) EEG signals at scalp and source level, where the predicted distributions are well recovered with high TF precision, and the detected connectivity patterns of MI EEG data are physiologically and anatomically interpretable and yield new insights into the dynamical organization of oscillatory cortical networks. Conclusion: Experimental results confirm the effectiveness of the proposed TF CGC method in tracking rapidly varying causalities of EEG based oscillatory networks. Significance: The novel TF CGC method is expected to provide important information of neural mechanisms of perception and cognition.

This paper addresses nonlinear time series modelling and prediction problem using a type of wavelet neural networks. The basic building block of the neural network models is a ridge type function. The training of such a network is a nonlinear optimization problem. Evolutionary algorithms (EAs), including genetic algorithm (GA) and particle swarm optimization (PSO), together with a new gradient-free algorithm (called coordinate dictionary search optimization – CDSO), are used to train network models. An example for real speed wind data modelling and prediction is provided to show the performance of the proposed networks trained by these three optimization algorithms.

A new ultra-least squares (ULS) criterion is introduced for system identification. Unlike the standard least squares criterion which is based on the Euclidean norm of the residuals, the new ULS criterion is derived from the Sobolev space norm. The new criterion measures not only the discrepancy between the observed signals and the model prediction but also the discrepancy between the associated weak derivatives of the observed and the model signals. The new ULS criterion possesses a clear physical interpretation and is easy to implement. Based on this, a new Ultra-Orthogonal Forward Regression (UOFR) algorithm is introduced for nonlinear system identification, which includes converting a least squares regression problem into the associated ultra-least squares problem and solving the ultra-least squares problem using the orthogonal forward regression method. Numerical simulations show that the new UOFR algorithm can significantly improve the performance of the classic OFR algorithm.

Standard convolutional filters usually capture unnecessary overlap of features resulting in a waste of computational cost. In this paper, we aim to solve this problem by proposing a novel Learned Depthwise Separable Convolution (LdsConv) operation that is smart but has a strong capacity for learning. It integrates the pruning technique into the design of convolutional filters, formulated as a generic convolutional unit that can be used as a direct replacement of convolutions without any adjustments of the architecture. To show the effectiveness of the proposed method, experiments are carried out using the state-of-the-art convolutional neural networks (CNNs), including ResNet, DenseNet, SE-ResNet and MobileNet, respectively. The results show that by simply replacing the original convolution with LdsConv in these CNNs, it can achieve a significantly improved accuracy while reducing computational cost. For the case of ResNet50, the FLOPs can be reduced by 40.9%, meanwhile the accuracy on the associated ImageNet increases.

A new transient Granger causality detection method is proposed based on a time-varying\ud parametric modelling framework, and is applied to real EEG signals to reveal the causal information flow\ud during motor imagery (MI) tasks. The time-varying parametric modelling approach employs a nonlinear\ud autoregressive with external input (NARX) model, whose parameters are approximated by a set of multiwavelet\ud basis functions. A regularized orthogonal least squares (ROLS) algorithm is then used to produce\ud a parsimonious or sparse regression model and estimate the associated model parameters. The time-varying\ud Granger causality between nonstationary signals can be detected accurately by making use of both the good\ud approximation properties of multi-wavelets and the good generalization performance of the ROLS in the\ud presence of high-level noise. Two simulation examples are presented to demonstrate the effectiveness of\ud the proposed method for both linear and nonlinear causal detection respectively. The proposed method is\ud then applied to real EEG signals of MI tasks. It follows that transient causal information flow over the\ud time course between various sensorimotor related channels can be successfully revealed during the whole\ud reaction processes. Experiment results from these case studies confirm the applicability of the proposed\ud scheme and show its utility for the understanding of the associated neural mechanism and the potential\ud significance for developing MI tasks based brain-computer interface (BCI) systems.

A new iOFR-MF iterative orthogonal forward regression--modulating function algorithm is proposed to identify continuous-time models from noisy data by combining the MF method and the iOFR algorithm. In the new method, a set of candidate terms, which describe different dynamic relationships among the system states or between the input and output, are first constructed. These terms are then modulated using the MF method to generate the data matrix. The iOFR algorithm is next applied to build the relationships between these modulated terms, which include detecting the model structure and estimating the associated parameters. The relationships between the original variables are finally recovered from the model of the modulated terms. Both nonlinear state-space models and a class of higher order nonlinear input–output models are considered. The new direct method is compared with the traditional finite difference method and results show that the new method performs much better than the finite difference method. The new method works well even when the measurements are severely corrupted by noise. The selection of appropriate MFs is also discussed.

System identification has been applied in diverse areas over past decades. In particular, parametric modelling approaches such as linear and nonlinear autoregressive with exogenous inputs models have been extensively used due to the transparency of the model structure. Model structure detection aims to identify parsimonious models by ranking a set of candidate model terms using some dependency metrics, which evaluate how the inclusion of an individual candidate model term affects the prediction of the desired output signal. The commonly used dependency metrics such as correlation function and mutual information may not work well in some cases, and therefore, there are always uncertainties in model parameter estimates. Thus, there is a need to introduce a new model structure detection scheme to deal with uncertainties in parameter estimation. In this work, a distance correlation metric is implemented and incorporated with a bagging method. The combination of these two implementations enhances the performance of existing forward selection approaches in that it provides the interpretability of nonlinear dependency and an insightful uncertainty analysis for model parameter estimates. The new scheme is referred as bagging forward orthogonal regression using distance correlation (BFOR-dCor) algorithm. A comparison of the performance of the new BFOR-dCor algorithm with benchmark algorithms using metrics like error reduction ratio, mutual information, or the Reversible Jump Markov Chain Monte Carlo method has been carried out in dealing with several numerical case studies. For ease of analysis, the discussion is restricted to polynomial models that can be expressed in a linear-in-the-parameters form.

The identification of nonlinear time-varying systems using linear-in-the-parameter models is investigated. An efficient common model structure selection CMSS algorithm is proposed to select a common model structure, with application to EEG data modelling. The time-varying parameters for the identified common-structured model are then estimated using a sliding-window recursive least squares SWRLS approach. The new method can effectively detect and adaptively track and rapidly capture the transient variation of nonstationary signals, and can also produce robust models with better generalisation properties. Two examples are presented to demonstrate the effectiveness and applicability of the new approach including an application to EEG data.

Model selection is a crucial step in choosing a best model from a series of candidate models for data based modelling problems. The commonly used Akaike information criterion (AIC) and Bayesian information criteria (BIC) may not be effective for many real-world modelling problems when the true system model structure is unknown and therefore not included in the candidate model set. This study investigates the model selection issue using AIC, BIC and an adjustable prediction error sum of squares (APRESS) for nonlinear dynamic predictive modelling. Results from simulation and real data modelling case studies show that both BIC and APRESS produce good models for nonlinear modelling problems. The APRESS works slightly better than BIC in achieving a parsimonious representation of the studied system. In addition, a model averaging method is introduced, which is capable to provide an averaged model that is more robust in generalization (i.e. in representing future data) than any single model.

A new parametric approach is proposed for nonlinear and non-stationary system identification based on a time-varying nonlinear autoregressive with exogenous input (TV-NARX) model. The time-varying coefficients of the TV-NARX model are expanded using multi- wavelet basis functions and the model is thus transformed into a time-invariant regression problem.\ud An ultra-orthogonal forward regression (UOFR) algorithm aided by mutual information (MI) is designed to identify a parsimonious model structure and estimate the associated model parameters. The UOFR-MI algorithm which uses not only the observed data themselves but also weak derivatives of the signals is more powerful in model structure detection. The proposed approach combining the advantages of both the basis function expansion method and the UOFR-MI algorithm is proved to be capable of tracking the change of time-varying parameters effectively in both numerical simulations and the real EEG data.

In this paper, a new global time series averaging method is proposed for improving k-means clustering performance for observed signals or time series data. The proposed method is different from most commonly used time series averaging methods, such as pairwise averaging method (PA), nonlinear alignment and averaging filter (NLAAF), prioritized shape averaging (PSA) and dynamic time warping (DTW) barycentre averaging (DBA). The implementation mechanism of the new method is as follows: i) Choose an initial cluster centre sequence based on the within-cluster distance; ii) Compute the weight values and extract the optimal matching path according to the dynamic time warping distance between the centre sequence and all the rest sequences, and iii) Calculate the average sequence using the rest sequences, the optimal matching paths and the weight values. To demonstrate the performance of the proposed method, the aforementioned four existing averaging methods, along with the proposed method, are applied to a benchmark time series dataset containing 600 sequences. The experimental results show that the proposed method outperforms all the four compared methods, in terms of the sum of within-cluster distance and adjusted Rand index.

A novel iterative learning algorithm is proposed to improve the classic Orthogonal Forward Regression (OFR) algorithm in an attempt to produce an optimal solution under a purely OFR framework without using any other auxiliary algorithms. The new algorithm searches for the optimal solution on a global solution space while maintaining the advantage of simplicity and computational efficiency. Both a theoretical analysis and simulations demonstrate the validity of the new algorithm.

Background Frequency domain Granger causality measures have been proposed and widely applied in analyzing rhythmic neurophysiological and biomedical signals. Almost all these measures are based on linear time domain regression models, and therefore can only detect linear causal effects in the frequency domain. New method A frequency domain causality measure, the partial directed coherence, is explicitly linked with the frequency response function concept of linear systems. By modeling the nonlinear relationships between time series using nonlinear models and employing corresponding frequency-domain analysis techniques (i.e. generalized frequency response functions), a new nonlinear partial directed coherence method is derived. Results The advantages of the new method are illustrated via a numerical example of a nonlinear physical system and an application to electroencephalogram signals from a patient with childhood absence epilepsy. Comparison with existing methods The new method detects both linear and nonlinear casual effects between bivariate signals in the frequency domain, while the existing measures can only detect linear effects. Conclusions The proposed new method has important advantages over the classical linear measures, because detecting nonlinear dependencies has become more and more important in characterizing functional couplings in neuronal and biological systems.

This paper introduces a new approach for nonlinear and non-stationary time-varying system identification based on time-varying nonlinear autoregressive moving average with exogenous variable TV-NARMAX models. The challenging model structure selection and parameter tracking problems are solved by combining a multiwavelet basis function expansion of the time-varying parameters with an orthogonal least squares algorithm. Numerical examples demonstrate that the proposed approach can track rapid time-varying effects in nonlinear systems more accurately than the standard recursive algorithms. Based on the identified time domain model, a new frequency domain analysis approach is introduced based on a time-varying generalised frequency response function TV-GFRF concept, which enables the analysis of nonlinear, non-stationary systems in the frequency domain. Features in the TV-GFRFs which depend on the TV-NARMAX model structure and time-varying parameters are investigated. It is shown that the high-dimensional frequency features can be visualised in a low-dimensional time–frequency space.

A new mutual information based algorithm is introduced for term selection in spatio-temporal models. A generalised cross validation procedure is also introduced for model length determination and examples based on cellular automata, coupled map lattice and partial differential equations are described.

This part of the paper introduces a comparison of VAR-IM algorithm with modern techniques used for missing data analysis. Quantitative methods are usually developed based on some fundamental understanding of the statistical analysis of the missing data. Various types of quantitative methods such as K nearest neighbour (KNN), (Multivariate Autoregressive state-Space) MARRS package and EM algorithm are discussed. The relative advantages and disadvantages of these approaches are highlighted. The performance of the vector autoregressive model based imputation methods is compared with that of three existing methods, namely, KNN, MARRRS and EM for dealing with missing data in multivariate time series, where an ECG dataset is used as an a case study. The results show that VAR-IM produces a better recovering performance for missing values than the other three methods. The advantages and limitations of the VAR-IM algorithm is also discussed.

Given an observed data set, there are different methods that can be used to impute missing data. While excellent work has been done in this field, most available approaches are focused on some particular applications, such as static data and univariate time series. The primary aim of the two papers Part I VAR-IM algorithm v.s. traditional methods and Part II VAR-IM algorithm v.s. modern algorithms — is to introduce an algorithm for handling missing data in multivariate time series based on vector autoregressive (VAR) model by combining an expectation and minimization (EM) algorithm with the prediction error minimization (PEM) method. In the first part, we conduct two cases studies (one for simulation data and another for real ECG data) to compare the proposed algorithm with three traditional methods for imputing missing data: Mean substitution, list-wise deletion and linear regression substitution. In the second part, the proposed method is compared with more powerful modern techniques: MARRS Package, nearest neighbour, and the full information maximum likelihood (FIML) method. Furthermore, we demonstrate the use of the proposed method together with an empirical example of multivariate time series to ECG data and discuss its advantages and limitations.

This brief introduces a new parametric modelling and identification method for linear time-varying systems using a block least mean square (LMS) approach where the time-varying parameters are approximated using multi-wavelet basis functions. This approach can be applied to track rapidly or even sharply varying processes and is developed by combining wavelet approximation theory with a block LMS algorithm. Numerical examples are provided to show the effectiveness of the proposed method for dealing with severely nonstationary processes. Application of the proposed approach to a real mechanical system indicates better tracking capability of the multi-wavelet basis function algorithm compared with the normalized least squares or recursive least squares routines.

A novel modelling scheme that can be used to estimate and track time-varying properties of nonstationary signals is investigated. This scheme is based on a class of time-varying AutoRegressive with an eXogenous input (TVARX) models where the associated time-varying parameters are represented by multi-wavelet basis functions. The orthogonal least square (OLS) algorithm is then applied to refine the model parameter estimates of the TVARX model. The main features of the multi-wavelet approach is that it enables smooth trends to be tracked but also to capture sharp changes in the time-varying process parameters. Simulation studies and applications to real EEG data show that the proposed algorithm can provide important transient information on the inherent dynamics of nonstationary processes.

Particle swarm optimization (PSO) is introduced to implement a new constructive learning algorithm for training generalized cellular neural networks (GCNNs) for the identification of spatio-temporal evolutionary (STE) systems. The basic idea of the new PSO-based learning algorithm is to successively approximate the desired signal by progressively pursuing relevant orthogonal projections. This new algorithm will thus be referred to as the orthogonal projection pursuit (OPP) algorithm, which is in mechanism similar to the conventional projection pursuit approach. A novel two-stage hybrid training scheme is proposed for constructing a parsimonious GCNN model. In the first stage, the orthogonal projection pursuit algorithm is applied to adaptively and successively augment the network, where adjustable parameters of the associated units are optimized using a particle swarm optimizer. The resultant network model produced at the first stage may be redundant. In the second stage, a forward orthogonal regression (FOR) algorithm, aided by mutual information estimation, is applied to refine and improve the initially trained network. The effectiveness and performance of the proposed method is validated by applying the new modeling framework to a spatio-temporal evolutionary system identification problem.

A new adaptive orthogonal search (AOS) algorithm is proposed for model subset selection and non-linear system identification. Model structure detection is a key step in any system identification problem. This consists of selecting significant model terms from a redundant dictionary of candidate model terms, and determining the model complexity (model length or model size). The final objective is to produce a parsimonious model that can well capture the inherent dynamics of the underlying system. In the new AOS algorithm, a modified generalized cross-validation criterion, called the adjustable prediction error sum of squares (APRESS), is introduced and incorporated into a forward orthogonal search procedure. The main advantage of the new AOS algorithm is that the mechanism is simple and the implementation is direct and easy, and more importantly it can produce efficient model subsets for most non-linear identification problems.

A new algorithm for determining the output frequency range and the frequency components of Volterra models under multiple inputs is introduced for nonlinear system analysis. For a given Volterra model, the output frequency components corresponding to a multi-tone input can easily be calculated using the new algorithm.

The Dst index is a key parameter which characterises the disturbance of the geomagnetic field in magnetic storms. Modelling of the Dst index is thus very important for the analysis of the geomagnetic field. A data-based modelling approach, aimed at obtaining efficient models from limited input–output observational data, provides a powerful tool for analysing and forecasting geomagnetic activities including the prediction of the Dst index. In this study, the process of the Dst index is treated to be a structure-unknown system, where the solar wind parameter ( VBs ) and the solar wind dynamic pressure ( P ) are the system inputs, and the Dst index is the system output. A novel multiscale RBF (MSRBF) network is introduced to represent such a two-input and single-output system, where the Dst index is related to the solar wind parameter and the dynamic pressure, via a hybrid network model consisting of two submodels: a linear part that reflects the linear relationship between the output and the inputs, and a nonlinear part that captures the effect of the interacting contribution of past observations of the inputs and the output, on the current output. The proposed MSRBF network can easily be converted into a linear-in-the-parameters form and the training of the linear network model can easily be implemented using a forward orthogonal regression (FOR) algorithm. One advantage of the new MSRBF network, compared with traditional single scale RBF networks, is that the new network is more flexible for describing complex nonlinear dynamical systems.

A sparse representation, with satisfactory approximation accuracy, is usually desirable in any nonlinear system identification and signal processing problem. A new forward orthogonal regression algorithm, with mutual information interference, is proposed for sparse model selection and parameter estimation. The new algorithm can be used to construct parsimonious linear-in-the-parameters models

Most nonlinear modelling approaches focus on solving a model selection problem with complete data, and in many cases original data are pre-processed with some nonlinear transforms such as Box-Tidwell and fractional polynomial transformation. Often these approaches can lead to models that are better than traditional models (for example, logistic model and quadratic model). However, in the case of missing data, it is not easy to predict the relationship between the predictor and dependent variables; traditional nonlinear models in some cases of missing data analysis give poor results. This paper explains nonlinear model selection techniques for missing data. It includes an overview of nonlinear model selection with complete data, and provides accessible descriptions of Box-Tidwell and fractional polynomial methods for model selection. In particular, this paper focuses on a fractional polynomial method for nonlinear modelling in cases of missing data and presents analysis examples to illustrate performance of the method.

This paper presents a new type of neural network called ridge basis function neural network (RiBFNN), which in structure comprises two submodels, namely a linear submodel and a ridge basis function submodel. The proposed model has the following advantages. It is known that linear models are transparent and easily interpretable, the inclusion of a linear submodel can therefore determine how the overall evolution trend of the system behavior (output or response) depends on the system input (or dependent) variables. On the other hand, the inclusion of the ridge basis functions enables the identification of nonlinearities that cannot be revealed by linear models and thus can improve the overall prediction performance. The proposed network is applied to a real data modeling task in relation to the relativistic electron flux intensity prediction in space weather study, and relevant performance of the network is presented.

Wavelet based non-parametric additive NARX models are proposed for nonlinear input-output system identification. By expanding each functional component of the non-parametric NARX model into wavelet multiresolution expansions, the non-parametric estimation problem becomes a linear-in-the-parameters problem, and least-squares-based methods such as the orthogonal forward regression (OFR) approach, coupled with model size determination criteria, can be used to select the model terms and estimate the parameters. Wavelet based additive models, combined with model order determination and variable selection approaches, are capable of handling problems of high dimensionality.

This paper introduces a new approach for the identification of coupled map lattice models of complex spatio-temporal patterns from measured data. The nonlinear functionals describing the evolution of the spatio-temporal patterns are constructed using B-spline wavelet and scaling functions. This provides a multiresolution approximation for the underlying spatio-temporal dynamics. An orthogonal least squares algorithm is used to determine the suitable terms from wavelet functions to form an accurate representation of the nonlinear spatio-temporal dynamics. Three examples are used to demonstrate the application of the proposed new approach.

A new approach for the estimation of spatial derivatives and the identification of a class of continuous spatio-temporal dynamical systems from experimental data is presented in this study. The proposed identification approach is a combination of implicit Adams integration and an orthogonal forward regression algorithm (OFR), in which the operators are expanded using polynomials as basis functions. The noisy experimental data are de-noised by using biorthogonal spline wavelet filters and the spatial derivatives are estimated using a multiresolution analysis method. Finally, a bootstrap method is applied to refine the identified parameters from the OFR algorithm. The resulting identified models of the spatio-temporal evolution form a system of partial differential equations. Examples are provided to demonstrate the efficiency of the proposed method.

The long term prediction of non-linear dynamical time series, based on identified multiresolution wavelet models, from historically observed data sets is investigated and a new direct prediction approach is introduced. Prediction results based on the new direct scheme are compared with those from iterative methods and it is shown that improved predictions can be obtained using the new approach.

A new hybrid model structure combing polynomial models with multiresolution wavelet decompositions is introduced for nonlinear system identification. Polynomial models play an important role in approximation theory and have been extensively used in linear and non-linear system identification. Wavelet decompositions, in which the basis functions have the property of localization in both time and frequency, outperform many other approximation schemes and offer a flexible solution for approximating arbitrary functions. Although wavelet representations can approximate even severe nonlinearities in a given signal very well, the advantage of these representations can be lost when wavelets are used to capture linear or low-order nonlinear behaviour in a signal. In order to sufficiently utilize the global property of polynomials and the local property of wavelet representations simultaneously, in this study polynomial models and wavelet decompositions are combined together in a parallel structure to represent nonlinear input-output systems. As a special form of the NARMAX model, this hybrid model structure will be referred to as the WAvelet-NARMAX model, or simply WANARMAX. Generally, such a WANARMAX representation for an input-output system might involve a large number of basis functions and therefore a great number of model terms. Experience reveals that only a small number of these model terms are significant to the system output. A new fast orthogonal least-squares algorithm, called the matching pursuit orthogonal least squares (MPOLS) algorithm, is also introduced in this study to determine which terms should be included in the final model.

A new modelling framework for identifying and reconstructing chaotic systems is developed based on multiresolution wavelet decompositions. Qualitative model validation is used to compare the multiresolution wavelet models and it is shown that the dynamical features of chaotic systems can be captured by the identified models providing the wavelet basis functions are properly selected. Two basis selection algorithms, orthogonal least squares and a new matching pursuit orthogonal least squares, are considered and compared. Several examples are included to illustrate the results.

A new unified modelling framework based on the superposition of additive submodels, functional components, and wavelet decompositions is proposed for non-linear system identification. A non-linear model, which is often represented using a multivariate non-linear function, is initially decomposed into a number of functional components via the well-known analysis of variance (ANOVA) expression, which can be viewed as a special form of the NARX (non-linear autoregressive with exogenous inputs) model for representing dynamic input–output systems. By expanding each functional component using wavelet decompositions including the regular lattice frame decomposition, wavelet series and multiresolution wavelet decompositions, the multivariate non-linear model can then be converted into a linear-in-the-parameters problem, which can be solved using least-squares type methods. An efficient model structure determination approach based upon a forward orthogonal least squares (OLS) algorithm, which involves a stepwise o...

Identification of linear and nonlinear time-varying systems is investigated and a new wavelet model identification algorithm is introduced. By expanding each time-varying coefficient using a multiresolution wavelet expansion, the time varying problem is reduced to a time invariant problem and the identification reduces to regressor selection and parameter estimation. Several examples are included to illustrate the application of the new algorithm.

A linear and nonlinear causality detection method called the error-reduction-ratio causality (ERRC) test is introduced in this paper to investigate if linear or nonlinear models should be considered in the study of human electroencephalograph (EEG) data. In comparison to the traditional Granger methods, one significant advantage of the ERRC approach is that it can effectively detect the time-varying linear and nonlinear causalities between two signals without fitting a complete nonlinear model. Two numerical simulation examples are employed to compare the performance of the new method with other widely used methods in the presence of noise and in tracking time-varying causality. Finally, an application to measure the linear and nonlinear relationships between two EEG signals from different cortical sites for patients with childhood absence epilepsy is discussed.

A novel simple approach is presented for solving small sample size problem in nonlinear dynamic system identification where only a very small number of measured input and output data points are available. The technique is built upon an orthogonal least squares algorithm and a new resampling method called output jittering method,. The proposed algorithm is applied to the model identification of bioethanol production and it shows that a robust and reliable model can be obtained with the new system identification approach.

An important step in the identification of cellular automata (CA) is to detect the correct neighborhood before parameter estimation. Many authors have suggested procedures based on the removal of redundant neighbors from a very large initial neighborhood one by one to find the real model, but this often induces ill conditioning and overfitting. This is true particularly for a large initial neighborhood where there are few significant terms, and this will be demonstrated by an example in this paper. By introducing a new criteria and three new techniques, this paper proposes a new adaptive fast CA orthogonal-least-square (Adaptive-FCA-OLS) algorithm, which cannot only adaptively search for the correct neighborhood without any preset tolerance but can also considerably reduce the computational complexity and memory usage. Several numerical examples demonstrate that the Adaptive-FCA-OLS algorithm has better robustness to noise and to the size of the initial neighborhood than other recently developed neighborhood detection methods in the identification of binary CA.

[1] The NARMAX OLS-ERR algorithm, which is widely used in the study of systems dynamics, is able to determine the causal relationship between the input and output variables for nonlinear systems. This technique has been applied to measurements of the solar wind from ACE at L1 and the Dst index in order to find the best solar wind–magnetosphere coupling function, i.e., which combination of solar wind parameters provides the best predictive capabilities of the Dst index. The data-deduced coupling functions were then compared to those suggested in previous analytical and data-based studies. The most appropriate coupling function was found to be n1/2VαBT sin6(θ/2), where the power of velocity, α, was inconclusive but should be in the range 2–3.

In this brief, by combining an efficient wavelet representation with a coupled map lattice model, a new family of adaptive wavelet neural networks, called lattice dynamical wavelet neural networks (LDWNNs), is introduced for spatio-temporal system identification. A new orthogonal projection pursuit (OPP) method, coupled with a particle swarm optimization (PSO) algorithm, is proposed for augmenting the proposed network. A novel two-stage hybrid training scheme is developed for constructing a parsimonious network model. In the first stage, by applying the OPP algorithm, significant wavelet neurons are adaptively and successively recruited into the network, where adjustable parameters of the associated wavelet neurons are optimized using a particle swarm optimizer. The resultant network model, obtained in the first stage, however, may be redundant. In the second stage, an orthogonal least squares algorithm is then applied to refine and improve the initially trained network by removing redundant wavelet neurons from the network. An example for a real spatio-temporal system identification problem is presented to demonstrate the performance of the proposed new modeling framework.

A new iterative orthogonal least squares forward regression (iOFR) algorithm is proposed to identify nonlinear systems which may not be persistently excited. By slightly revising the classic forward orthogonal regression (OFR) algorithm, the new iterative algorithm provides search solutions on a global solution space. Examples show that the new iterative algorithm is computationally efficient and capable of producing a good model even when the input is not completely persistently excited.

A novel modelling framework is proposed for constructing parsimonious and flexible multiscale radial basis function networks (RBF). Unlike a conventional standard single scale RBF network, where all the basis functions have a common kernel width, the new network structure adopts multiscale Gaussian functions as the bases, where each selected centre has multiple kernel widths, to provide more flexible representations with better generalization properties for general nonlinear dynamical systems. As a direct extension of the traditional single scale Gaussian networks, the new multiscale network is easy to implement and is quick to learn using standard learning algorithms. A k-means clustering algorithm and an improved orthogonal least squares (OLS) algorithm are used to determine the unknown parameters in the network model including the centres and widths of the basis functions, and the weights between the basis functions. It is demonstrated that the new network can lead to a parsimonious model with much better generalization property compared with the traditional single width RBF networks.

[1] A new identification approach is introduced for predicting the Dst index using multiresolution B-spline wavelet models based on an observational data set consisting of VBs, the solar wind parameter, as the input and the Dst index as the output. The relationship between the input VBs and output Dst is initially described using a B-spline wavelet model. This model is then simplified using an orthogonal least squares and error reduction ratio (OLS-ERR) algorithm by selecting the significant model terms to produce a parsimonious wavelet model. Forecasts of the Dst index are then computed based on this model.

A new time-varying autoregressive (TVAR) modelling approach is proposed for non-stationary signal processing and analysis, with application to EEG data modelling and power spectral estimation. In the new parametric modelling framework, the time-dependent coefficients of the TVAR model are represented using a novel multiwavelet decomposition scheme. The time-varying modelling problem is then reduced to regression selection and parameter estimation, which can be effectively resolved by using a forward orthogonal regression algorithm. Two examples, one for an artificial signal and another for an EEG signal, are given to show the effectiveness and applicability of the new TVAR modelling method.

Model structure selection plays a key role in non-linear system identification. The first step in non-linear system identification is to determine which model terms should be included in the model. Once significant model terms have been determined, a model selection criterion can then be applied to select a suitable model subset. The well known Orthogonal Least Squares (OLS) type algorithms are one of the most efficient and commonly used techniques for model structure selection. However, it has been observed that the OLS type algorithms may occasionally select incorrect model terms or yield a redundant model subset in the presence of particular noise structures or input signals. A very efficient Integrated Forward Orthogonal Search (IFOS) algorithm, which is assisted by the squared correlation and mutual information, and which incorporates a Generalised Cross-Validation (GCV) criterion and hypothesis tests, is introduced to overcome these limitations in model structure selection.