Search

Using existing production/injection rates is convenient for analysing a mature field as the access to relevant historical records does not cause any extra cost and a large amount of significant information can be extracted from the dataset by means of good analysis methods.In the proposed work, each of the production wells is considered as an output variable while injection rates from other wells, which have an effect on the outputs, are treated as input variables. Control and System Engineering Approaches motivate such considerations.Since injection and production are time dependent variables, time delays were considered in the analysis. This consideration allows recognition of how long it takes for the injection on a certain well to affect production.The Error Reduction Ratio (ERR) algorithm, along with the Orthogonal Least Squares (OLS) method is used to determine how each of the injection wells affects production; this is achieved by sequentially orthogonalising input variables according to the order of their contribution to production.The algorithm can be customised to explain a certain variance threshold in production resulting from injection wells. For the present case study, a 19-year monthly dataset from Beatrice, an operating field in the North Sea is studied.It is observed that the historical datasets contain missing values. In order to address this problem, missing values are predicted by using an autoregressive linear model.

Most nonlinear modelling approaches focus on solving a model selection problem with complete data, and in many cases original data are pre-processed with some nonlinear transforms such as Box-Tidwell and fractional polynomial transformation. Often these approaches can lead to models that are better than traditional models (for example, logistic model and quadratic model). However, in the case of missing data, it is not easy to predict the relationship between the predictor and dependent variables; traditional nonlinear models in some cases of missing data analysis give poor results. This paper explains nonlinear model selection techniques for missing data. It includes an overview of nonlinear model selection with complete data, and provides accessible descriptions of Box-Tidwell and fractional polynomial methods for model selection. In particular, this paper focuses on a fractional polynomial method for nonlinear modelling in cases of missing data and presents analysis examples to illustrate performance of the method.

This paper presents a new type of neural network called ridge basis function neural network (RiBFNN), which in structure comprises two submodels, namely a linear submodel and a ridge basis function submodel. The proposed model has the following advantages. It is known that linear models are transparent and easily interpretable, the inclusion of a linear submodel can therefore determine how the overall evolution trend of the system behavior (output or response) depends on the system input (or dependent) variables. On the other hand, the inclusion of the ridge basis functions enables the identification of nonlinearities that cannot be revealed by linear models and thus can improve the overall prediction performance. The proposed network is applied to a real data modeling task in relation to the relativistic electron flux intensity prediction in space weather study, and relevant performance of the network is presented.

System identification is a challenging and interesting engineering problem that has been studied for decades. In particular, the NARMAX methodology has been extensively used with interesting results. Such methodology identifies a deterministic parsimonious model by ranking a set of candidate terms using a linear dependency metric with respect to the output. Other metrics have been used that identify nonlinear dependencies, like the mutual information, but they are hard to interpret. In this work, the distance correlation metric is implemented together with the bagging method. These two implementations enhance the performance of the NARMAX methodology providing interpretability of nonlinear dependencies and uncertainty measures in the model identified. A comparison of the new BOFR-dCor (Bagging Orthogonal Forward Regression using distance Correlation) algorithm is done with respect to the traditional OFR (Orthogonal Forward Regression) algorithm and the OFR-MI (Orthogonal Forward Regression using Mutual Information) algorithm showing interesting results that improve interpretability and uncertainty analysis.

Spatio-temporal images are a class of complex dynamical systems that evolve over both space and time. Compared with pure temporal processes, the identification of spatio-temporal models from observed images is much more difficult and quite challenging. Starting with an assumption that there is no a priori information about the true model but only observed data are available, this work introduces a new type of wavelet network that utilizes the easy tractability and exploits the good properties of multiscale wavelet decompositions to represent the rules of the associated spatio-temporal evolutionary system. An application to a chemical reaction exhibiting a spatio-temporal evolutionary behaviour, is investigated to demonstrate the application of the proposed modeling and learning approaches. This work was supported in part by EPSRC under Grant: EP/I011056/1 and Platform Grant EP/H00453X/1

A great deal of recent methodological research on missing data analysis has focused on model parameter estimation using modern statistical methods such as maximum likelihood and multiple imputation. These approaches are better than traditional methods (for example listwise deletion and mean imputation methods). These modern techniques can lead to unbiased parametric estimation in many particular application cases. However, these methods do not work well in some cases especially for nonlinear systems that have highly nonlinear behaviour. This paper explains the linear parametric estimation in existence of missing data, which includes an overview of biased and unbiased linear parametric estimation with missing data, and provides accessible descriptions of expectation maximization (EM) algorithm and Gauss-Newton method. In particular, this paper proposes a Gauss-Newton iteration method for nonlinear parametric estimation in case of missing data. Since Gauss-Newton method needs initial values that are hard to obtain in the presence of missing data, the EM algorithm is thus used to estimate these initial values. In addition, we present two analysis examples to illustrate the performance of the proposed methods.

This paper introduces a new approach for the identification of coupled map lattice models of complex spatio-temporal patterns from measured data. The nonlinear functionals describing the evolution of the spatio-temporal patterns are constructed using B-spline wavelet and scaling functions. This provides a multiresolution approximation for the underlying spatio-temporal dynamics. An orthogonal least squares algorithm is used to determine the suitable terms from wavelet functions to form an accurate representation of the nonlinear spatio-temporal dynamics. Three examples are used to demonstrate the application of the proposed new approach.

A new approach for the estimation of spatial derivatives and the identification of a class of continuous spatio-temporal dynamical systems from experimental data is presented in this study. The proposed identification approach is a combination of implicit Adams integration and an orthogonal forward regression algorithm (OFR), in which the operators are expanded using polynomials as basis functions. The noisy experimental data are de-noised by using biorthogonal spline wavelet filters and the spatial derivatives are estimated using a multiresolution analysis method. Finally, a bootstrap method is applied to refine the identified parameters from the OFR algorithm. The resulting identified models of the spatio-temporal evolution form a system of partial differential equations. Examples are provided to demonstrate the efficiency of the proposed method.

The long term prediction of non-linear dynamical time series, based on identified multiresolution wavelet models, from historically observed data sets is investigated and a new direct prediction approach is introduced. Prediction results based on the new direct scheme are compared with those from iterative methods and it is shown that improved predictions can be obtained using the new approach.

An efficient class of nonlinear models, constructed using cardinal B-spline (CBS) basis functions, are proposed for high tide forecasts at the Venice lagoon. Accurate short term predictions of high tides in the lagoon can easily be calculated using the proposed CBS models, which can also produce good long term (up to 24 hrs ahead) forecasts for normal water levels.

A new hybrid model structure combing polynomial models with multiresolution wavelet decompositions is introduced for nonlinear system identification. Polynomial models play an important role in approximation theory and have been extensively used in linear and non-linear system identification. Wavelet decompositions, in which the basis functions have the property of localization in both time and frequency, outperform many other approximation schemes and offer a flexible solution for approximating arbitrary functions. Although wavelet representations can approximate even severe nonlinearities in a given signal very well, the advantage of these representations can be lost when wavelets are used to capture linear or low-order nonlinear behaviour in a signal. In order to sufficiently utilize the global property of polynomials and the local property of wavelet representations simultaneously, in this study polynomial models and wavelet decompositions are combined together in a parallel structure to represent nonlinear input-output systems. As a special form of the NARMAX model, this hybrid model structure will be referred to as the WAvelet-NARMAX model, or simply WANARMAX. Generally, such a WANARMAX representation for an input-output system might involve a large number of basis functions and therefore a great number of model terms. Experience reveals that only a small number of these model terms are significant to the system output. A new fast orthogonal least-squares algorithm, called the matching pursuit orthogonal least squares (MPOLS) algorithm, is also introduced in this study to determine which terms should be included in the final model.

The geomagnetic activity of the Dst index is analyzed using wavelet transforms and it is shown that the Dst index possesses properties associated with self-affine fractals. For example, the power spectral density obeys a power-law dependence on frequency, and therefore the Dst index can be viewed as a self-affine fractal dynamic process. In fact, the behaviour of the Dst index, with a Hurst exponent H≈0.5 (power-law exponent β≈2) at high frequency, is similar to that of Brownian motion. Therefore, the dynamical invariants of the Dst index may be described by a potential Brownian motion model. Characterization of the geomagnetic activity has been studied by analysing the geomagnetic field using a wavelet covariance technique. The wavelet covariance exponent provides a direct effective measure of the strength of persistence of the Dst index. One of the advantages of wavelet analysis is that many inherent problems encountered in Fourier transform methods, such as windowing and detrending, are not necessary.

A new unified modelling framework based on the superposition of additive submodels, functional components, and wavelet decompositions is proposed for non-linear system identification. A non-linear model, which is often represented using a multivariate non-linear function, is initially decomposed into a number of functional components via the well-known analysis of variance (ANOVA) expression, which can be viewed as a special form of the NARX (non-linear autoregressive with exogenous inputs) model for representing dynamic input–output systems. By expanding each functional component using wavelet decompositions including the regular lattice frame decomposition, wavelet series and multiresolution wavelet decompositions, the multivariate non-linear model can then be converted into a linear-in-the-parameters problem, which can be solved using least-squares type methods. An efficient model structure determination approach based upon a forward orthogonal least squares (OLS) algorithm, which involves a stepwise o...

The purpose of variable selection is to pre-select a subset consisting of the significant variables or to eliminate the redundant variables from all the candidate variables of a system under study prior to model term detection. It is required that the selected significant variables alone should sufficiently represent the system. Generally, not all the model terms, which are produced by combining different variables, make an equal contribution to the system output and terms, which make little contribution, can be omitted. A parsimonious representation, which contains only the significant terms, can often be obtained without the loss of representational accuracy by eliminating the redundant terms. Based on these observations, a new variable and term selection algorithm is proposed in this paper. The term detection algorithm can be applied to the general class of non-linear modelling problems which can be expressed as a linear-in-the-parameters form. The variable selection procedure is based on locally linea...

Identification of linear and nonlinear time-varying systems is investigated and a new wavelet model identification algorithm is introduced. By expanding each time-varying coefficient using a multiresolution wavelet expansion, the time varying problem is reduced to a time invariant problem and the identification reduces to regressor selection and parameter estimation. Several examples are included to illustrate the application of the new algorithm.

A linear and nonlinear causality detection method called the error-reduction-ratio causality (ERRC) test is introduced in this paper to investigate if linear or nonlinear models should be considered in the study of human electroencephalograph (EEG) data. In comparison to the traditional Granger methods, one significant advantage of the ERRC approach is that it can effectively detect the time-varying linear and nonlinear causalities between two signals without fitting a complete nonlinear model. Two numerical simulation examples are employed to compare the performance of the new method with other widely used methods in the presence of noise and in tracking time-varying causality. Finally, an application to measure the linear and nonlinear relationships between two EEG signals from different cortical sites for patients with childhood absence epilepsy is discussed.

A basic assumption on the data used for nonlinear dynamic model identification is that the data points are continuously collected in chronological order. However, there are situations in practice where this assumption does not hold and we end up with an identification problem from multiple data sets. The problem is addressed in this paper and a new cross-validation-based orthogonal search algorithm for NARMAX model identification from multiple data sets is proposed. The algorithm aims at identifying a single model from multiple data sets so as to extend the applicability of the standard method in the cases, such as the data sets for identification are obtained from multiple tests or a series of experiments, or the data set is discontinuous because of missing data points. The proposed method can also be viewed as a way to improve the performance of the standard orthogonal search method for model identification by making full use of all the available data segments in hand. Simulated and real data are used in this paper to illustrate the operation and to demonstrate the effectiveness of the proposed method.

Bioethanol production by means of anaerobic thermophilic microorganisms with pentose or hexose as the substrate are of paramount importance in sustainable fuel innovation. Manipulation of microorganisms and the associated experiment conditions by means of various ad-hoc technology is obviously the most straightforward way with the aim of maximizing bioethanol yield. However, methodology by means of mathematical modeling and analysis is often neglected among these routines. In this paper, typical input-output models are applied in the metabolic system analysis of Thermoanaerobacter sp. X514 under sole glucose substrate, sole xylose substrate and mixed glucose and xylose substrates conditions. Orthogonal Least Squares (OLS) approach is used for model parameter estimation. Model selection is proposed in order to testify the generality of the suggested model. System identification results illustrate that various forms of Nonlinear AutoRegressive with eXogenous input models (NARX) are applicable in delineating the system where different substrates (glucose or xylose) are utilized during the experiments. The proposed model structure infers that the yields of various products in X514 are mainly driven by the history information of the substrate consumption change. Moreover, the interaction between the main fermentation products of X514 is indirectly connected through the proposed models.

A novel type of discrete-time fractional-power nonlinear autoregressive with exogenous input (FPNARX) model is introduced for system identification, modeling and prediction. Parameter estimation of such a model is a nonlinear optimization problem. A harmony search algorithm is then applied to solve such fractional models. Examples of both simulated and real data are provided.

An important step in the identification of cellular automata (CA) is to detect the correct neighborhood before parameter estimation. Many authors have suggested procedures based on the removal of redundant neighbors from a very large initial neighborhood one by one to find the real model, but this often induces ill conditioning and overfitting. This is true particularly for a large initial neighborhood where there are few significant terms, and this will be demonstrated by an example in this paper. By introducing a new criteria and three new techniques, this paper proposes a new adaptive fast CA orthogonal-least-square (Adaptive-FCA-OLS) algorithm, which cannot only adaptively search for the correct neighborhood without any preset tolerance but can also considerably reduce the computational complexity and memory usage. Several numerical examples demonstrate that the Adaptive-FCA-OLS algorithm has better robustness to noise and to the size of the initial neighborhood than other recently developed neighborhood detection methods in the identification of binary CA.

This paper introduces an error reduction ratio--causality (ERR-causality) test that can be used to detect and track causal relationships between two signals. In comparison to the traditional Granger method, one significant advantage of the new ERR-causality test is that it can effectively detect the time-varying direction of linear or nonlinear causality between two signals without fitting a complete model. Another important advantage is that the ERR-causality test can detect both the direction of interactions and estimate the relative time shift between the two signals. Numerical examples are provided to illustrate the effectiveness of the new method together with the determination of the causality between electroencephalograph signals from different cortical sites for patients during an epileptic seizure.

Statistical measures such as coherence, mutual information, or correlation are usually applied to evaluate the interactions between two or more signals. However, these methods cannot distinguish directions of flow between two signals. The capability to detect causalities is highly desirable for understanding the cooperative nature of complex systems. The main objective of this work is to present a linear and nonlinear time-varying parametric modeling and identification approach that can be used to detect Granger causality, which may change with time and may not be detected by traditional methods. A numerical example, in which the exact causal influences relationships, is presented to illustrate the performance of the method for time-varying Granger causality detection. The approach is applied to EEG signals to track and detect hidden potential causalities. One advantage of the proposed model, compared with traditional Granger causality, is that the results are easier to interpret and yield additional insights into the transient directed dynamical Granger causality interactions.

[1] Recently published data from Reeves et al. (2011) on the fluxes of 1.8–3.5 MeV electrons at geostationary orbit are subjected to Error Reduction Ratio (ERR) analysis in order to identify the parameters that control variance of these fluxes. ERR shows that it is the solar wind density not the velocity that controls most of the variance of the energetic electrons fluxes. High fluxes are observed under the conditions of low density in absolute majority of cases. Under the condition of fixed density the dependence of fluxes upon the velocity is the following: fluxes increase with the velocity reaching some saturation level. Both the level of saturation and the value of the velocity where it is achieved decrease with the increase of solar wind density.

[1] The NARMAX OLS-ERR algorithm, which is widely used in the study of systems dynamics, is able to determine the causal relationship between the input and output variables for nonlinear systems. This technique has been applied to measurements of the solar wind from ACE at L1 and the Dst index in order to find the best solar wind–magnetosphere coupling function, i.e., which combination of solar wind parameters provides the best predictive capabilities of the Dst index. The data-deduced coupling functions were then compared to those suggested in previous analytical and data-based studies. The most appropriate coupling function was found to be n1/2VαBT sin6(θ/2), where the power of velocity, α, was inconclusive but should be in the range 2–3.

[1] The NARMAX (Nonlinear Autoregressive Moving Average with eXogenous inputs) modeling based approach is used to identify the relations for Solar wind Magnetosphere Coupling Functions in order to identify the one with the best Dst index forecasting abilities. It is argued that the Error Reduction Ratio which is a cornerstone concept in the NARMAX system identification methodology is more appropriate for such nonlinear system as the terrestrial magnetosphere than correlation function analysis which has often been used in the past. It is shown that the identified relationships for coupling functions are proportional to , where is the clock angle, and not to as was suggested in earlier studies. It is shown that the factor appears naturally within the analytical model of Kan and Lee (1979) as the dependence on the clock angle.

It is well known that there is a dynamic relationship between cerebral blood flow (CBF) and cerebral blood volume (CBV). With increasing applications of functional MRI, where the blood oxygen-level-dependent signals are recorded, the understanding and accurate modeling of the hemodynamic relationship between CBF and CBV becomes increasingly important. This study presents an empirical and data-based modeling framework for model identification from CBF and CBV experimental data. It is shown that the relationship between the changes in CBF and CBV can be described using a parsimonious autoregressive with exogenous input model structure. It is observed that neither the ordinary least-squares (LS) method nor the classical total least-squares (TLS) method can produce accurate estimates from the original noisy CBF and CBV data. A regularized total least-squares (RTLS) method is thus introduced and extended to solve such an error-in-the-variables problem. Quantitative results show that the RTLS method works very well on the noisy CBF and CBV data. Finally, a combination of RTLS with a filtering method can lead to a parsimonious but very effective model that can characterize the relationship between the changes in CBF and CBV.

In this brief, by combining an efficient wavelet representation with a coupled map lattice model, a new family of adaptive wavelet neural networks, called lattice dynamical wavelet neural networks (LDWNNs), is introduced for spatio-temporal system identification. A new orthogonal projection pursuit (OPP) method, coupled with a particle swarm optimization (PSO) algorithm, is proposed for augmenting the proposed network. A novel two-stage hybrid training scheme is developed for constructing a parsimonious network model. In the first stage, by applying the OPP algorithm, significant wavelet neurons are adaptively and successively recruited into the network, where adjustable parameters of the associated wavelet neurons are optimized using a particle swarm optimizer. The resultant network model, obtained in the first stage, however, may be redundant. In the second stage, an orthogonal least squares algorithm is then applied to refine and improve the initially trained network by removing redundant wavelet neurons from the network. An example for a real spatio-temporal system identification problem is presented to demonstrate the performance of the proposed new modeling framework.

Starting from the basic concept of coupled map lattices, a new family of adaptive wavelet neural networks (AWNN) is introduced for spatio-temporal system identification, by combining an efficient wavelet representation with a coupled map lattice model. A new orthogonal projection pursuit (OPP) method, coupled with a particle swarm optimization (PSO) algorithm, is proposed for augmenting the proposed network. A novel two-stage hybrid training scheme is developed for constructing a parsimonious network model. In the first stage, by applying the orthogonal projection pursuit algorithm, significant wavelet neurons are adaptively and successively recruited into the network, where adjustable parameters of the associated wavelet neurons are optimized using a particle swarm optimizer. The resultant network model, obtained in the first stage, may however be redundant. In the second stage, an orthogonal least squares algorithm is then applied to refine and improve the initially trained network by removing redundant wavelet neurons from the network. The proposed two-stage hybrid training procedure can generally produce a parsimonious network model, where a ranked list of wavelet neurons, according to the capability of each neuron to represent the total variance in the system output signal is produced. Two spatio-temporal system identification examples are presented to demonstrate the performance of the proposed new modelling framework.

A new iterative orthogonal least squares forward regression (iOFR) algorithm is proposed to identify nonlinear systems which may not be persistently excited. By slightly revising the classic forward orthogonal regression (OFR) algorithm, the new iterative algorithm provides search solutions on a global solution space. Examples show that the new iterative algorithm is computationally efficient and capable of producing a good model even when the input is not completely persistently excited.

A novel modelling framework is proposed for constructing parsimonious and flexible multiscale radial basis function networks (RBF). Unlike a conventional standard single scale RBF network, where all the basis functions have a common kernel width, the new network structure adopts multiscale Gaussian functions as the bases, where each selected centre has multiple kernel widths, to provide more flexible representations with better generalization properties for general nonlinear dynamical systems. As a direct extension of the traditional single scale Gaussian networks, the new multiscale network is easy to implement and is quick to learn using standard learning algorithms. A k-means clustering algorithm and an improved orthogonal least squares (OLS) algorithm are used to determine the unknown parameters in the network model including the centres and widths of the basis functions, and the weights between the basis functions. It is demonstrated that the new network can lead to a parsimonious model with much better generalization property compared with the traditional single width RBF networks.

[1] A new identification approach is introduced for predicting the Dst index using multiresolution B-spline wavelet models based on an observational data set consisting of VBs, the solar wind parameter, as the input and the Dst index as the output. The relationship between the input VBs and output Dst is initially described using a B-spline wavelet model. This model is then simplified using an orthogonal least squares and error reduction ratio (OLS-ERR) algorithm by selecting the significant model terms to produce a parsimonious wavelet model. Forecasts of the Dst index are then computed based on this model.

The geomagnetic activity of the Dst index is analyzed using wavelet transforms and it is shown that the Dst index possesses properties associated with self-affine fractals. For example, the power spectral density obeys a power-law dependence on frequency, and therefore the Dst index can be viewed as a self-affine fractal dynamic process. In fact, the behaviour of the Dst index, with a Hurst exponent H≈0.5 (power-law exponent β≈2) at high frequency, is similar to that of Brownian motion. Therefore, the dynamical invariants of the Dst index may be described by a potential Brownian motion model. Characterization of the geomagnetic activity has been studied by analysing the geomagnetic field using a wavelet covariance technique. The wavelet covariance exponent provides a direct effective measure of the strength of persistence of the Dst index. One of the advantages of wavelet analysis is that many inherent problems encountered in Fourier transform methods, such as windowing and detrending, are not necessary.

A new time-varying autoregressive (TVAR) modelling approach is proposed for non-stationary signal processing and analysis, with application to EEG data modelling and power spectral estimation. In the new parametric modelling framework, the time-dependent coefficients of the TVAR model are represented using a novel multiwavelet decomposition scheme. The time-varying modelling problem is then reduced to regression selection and parameter estimation, which can be effectively resolved by using a forward orthogonal regression algorithm. Two examples, one for an artificial signal and another for an EEG signal, are given to show the effectiveness and applicability of the new TVAR modelling method.

This study considers the identification problem for a class of non-linear parameter-varying systems associated with the following scenario: the system behaviour depends on some specifically prescribed parameter properties, which are adjustable. To understand the effect of the varying parameters, several different experiments, corresponding to different parameter properties, are carried out and different data sets are collected. The objective is to find, from the available data sets, a common parameter-dependent model structure that best fits the adjustable parameter properties for the underlying system. An efficient Common Model Structure Selection (CMSS) algorithm, called the Extended Forward Orthogonal Regression (EFOR) algorithm, is proposed to select such a common model structure. Two examples are presented to illustrate the application and the effectiveness of the new identification approach.

Model structure selection plays a key role in non-linear system identification. The first step in non-linear system identification is to determine which model terms should be included in the model. Once significant model terms have been determined, a model selection criterion can then be applied to select a suitable model subset. The well known Orthogonal Least Squares (OLS) type algorithms are one of the most efficient and commonly used techniques for model structure selection. However, it has been observed that the OLS type algorithms may occasionally select incorrect model terms or yield a redundant model subset in the presence of particular noise structures or input signals. A very efficient Integrated Forward Orthogonal Search (IFOS) algorithm, which is assisted by the squared correlation and mutual information, and which incorporates a Generalised Cross-Validation (GCV) criterion and hypothesis tests, is introduced to overcome these limitations in model structure selection.

Natural computation, as an exciting and emerging interdisciplinary field, has been witnessing a surge of newly developed theories, methodologies and applications in recent years. These innovations have generated a huge impact in tackling complex and challenging real world problems. Not only are the well established intelligent techniques, such as neural networks, fuzzy systems, genetic and evolutionary algorithms, and cellular automata expanding to new application areas; the new forms of natural computation that have emerged recently, for example, swarm intelligence, artificial immune systems, bio-molecular computing and membrane computing, quantum computing, and granular computing, are also providing additional tools for various applications. One attractive area that natural computation has been playing a major role in is knowledge discovery. There are many success stories on natural computation and knowledge discovery, as you will find out in this special issue