Search

In the paper, a novel kernel recursive least-squares (KRLS) algorithm named random Fourier feature kernel recursive maximum mixture correntropy (RFF-RMMC) algorithm is proposed, which improves the prediction efficiency and robustness of the KRLS algorithm. Random Fourier feature (RFF) method as well as maximum mixture correntropy criterion (MMCC) are combined and applied into KRLS algorithm afterwards. Using RFF to approximate the kernel function in KRLS with a fixed cost can greatly reduce the computational complexity and simultaneously improve the prediction efficiency. In addition, the MMCC maintains the robustness like the maximum correntropy criterion (MCC). More importantly, it can enhance the accuracy of the similarity measurement between predicted and true values by more flexible parameter settings, and then make up for the loss of prediction accuracy caused by RFF to a certain extent. The performance of the RFF-RMMC algorithm for online time series prediction is verified by the simulation results based on three datasets.

Considering disturbances, noise, and sensor faults, this paper investigates interval estimation for discrete-time Takagi-Sugeno fuzzy systems. To obtain precise estimation results and attenuate disturbances and noise in the system simultaneously, we integrate robust observers based on the H technique and reachability analysis. Two novel observer gain computation methods are proposed for different purposes. The time-invariant method relaxes the original design conditions by transforming the parameterized linear matrix inequality into a series of linear matrix inequalities to increase computational speed, while the time-varying method employs the parameterized linear matrix inequality directly and conducts calculation online. Furthermore, reachable set representations for error dynamics are formulated by making use of both time-invariant observer gain and time-varying observer gain. Two comparative numerical simulation examples are studied to illustrate the effectiveness and superiority of the developed methods.

Kernel least mean square (KLMS) algorithm is a popular method for time series online prediction. It has the advantages of good robustness, low computational complexity, model simplicity and online learning ability. Unfortunately, as input data grows, the dictionary size increases and the computational complexity raises significantly. In addition, how to improve the adaptability in time-varying environments with noise is also one of the main challenges. Therefore, we propose an improved KLMS algorithm from sparse perspective in response to the above problems, called adaptive sparse quantization kernel least mean square (ASQ-KLMS) algorithm. In the new model, sequential outlier criterion for sparsification and weights adaptive adjustment are combined with coherence criterion and quantization to form ASQ-KLMS algorithm. Firstly, it makes full use of effective information and ignores the interference of abnormal information to obtain a more accurate and compact dictionary. Then, a good balance between algorithm efficiency and accuracy can be achieved by controlling the choice of parameters. In addition, it can adaptively adjust weights in time-varying environment. At last, the Lorenz chaotic time series, the ENSO chaotic time series and the Beijing $$\text {P}{{\text {M}}_{2.5}}$$ chaotic time series are used to prove the reliability of the ASQ-KLMS algorithm.

For echo state networks, it is difficult to select suitable reservoir parameters for different applications. In this paper, we put forward a divided adaptive multi-objective differential evolution (DAMODE) algorithm to optimize the reservoir parameters of echo state network. To improve the performance of multi-objective differential evolution algorithm, the entire population is divided into several subpopulations, and each subpopulation is divided into two subsets to compromise convergence and diversity, which are updated according to certain rules. Besides, the scale factor and crossover rate of differential evolutionary algorithm are adaptively adjusted. Experiments were conducted on the Lorenz time series, hourly temperature time series and PM2.5 time series in Beijing. Experiment results show that the proposed model can improve prediction accuracy and has good generalization ability and practicability.

In this paper, the design of reduced-order and full-order interval observers for discrete-time system is investigated. First, the original system is transformed into a descriptor system with the ability to estimate the unknown noise of sensors. Next, a reduced-order interval observer design method for descriptor system is presented. The state matrix of the system is obtained by linear transformation, and a Sylvester equation is constructed to find the appropriate observer gain matrix. Then, an existing design method of full-order interval observer is introduced in this paper. Finally, a simulation example is given to demonstrate that the reduced-order interval observer has more accurate interval estimation results than the full-order interval observer.

State space reconstruction is the foundation of chaotic system modeling. Selection of reconstructed variables is essential to the analysis and prediction of multivariate chaotic time series. As most existing state space reconstruction theorems deal with univariate time series, we have presented a novel nonuniform state space reconstruction method using information criterion for multivariate chaotic time series. We derived a new criterion based on low dimensional approximation of joint mutual information for time delay selection, which can be solved efficiently through the use of an intelligent optimization algorithm with low computation complexity. The embedding dimension is determined by conditional entropy, after which the reconstructed variables have relatively strong independence and low redundancy. The scheme, which integrates nonuniform embedding and feature selection, results in better reconstructions for multivariate chaotic systems. Moreover, the proposed nonuniform state space reconstruction method shows good performance in forecasting benchmark and actual multivariate chaotic time series.

Analyzing the disease data from the view of combinatorial features may better characterize the disease phenotype. In this study, a novel method is proposed to construct feature combinations and a classification model (CFC-CM) by mining key feature relationships. CFC-CM iteratively tests for differences in the feature relationship between different groups. To do this, it uses a modified $k$k-top-scoring pair (M-$k$k-TSP) algorithm and then selects the most discriminative feature pairs in the current feature set to infer the combinatorial features and build the classification model. Compared with support vector machines, random forests, least absolute shrinkage and selection operator, elastic net, and M-$k$k-TSP, the superior performance of CFC-CM on nine public gene expression datasets validates its potential for more precise identification of complex diseases. Subsequently, CFC-CM was applied to two metabolomics datasets, it obtained accuracy rates of $88.73\pm 2.06\%$88.73±2.06% and $79.11\pm 2.70\%$79.11±2.70% in distinguishing between hepatocellular carcinoma and hepatic cirrhosis groups and between acute kidney injury (AKI) and non-AKI samples, results superior to those of the other five methods. In summary, the better results of CFC-CM show that in contrast to molecules and combinations constituted by just two features, the combinations inferred by appropriate number of features could better identify the complex diseases.

In an optical phased array (OPA) free space laser communication system, one of the main issues is to control the phase of each beam to acquire fast and stable phase-locking for phase consistency, in which the number of the elements in the array is of vital importance. In this paper, we studied the method on improving the performance of phase-locking about speed and stability of optical phased array with large number of elements. The performance of OPA with different number of emitting elements was analyzed with the simplified stochastic parallel gradient descent (SPGD) algorithm for closed-loop phase-locking. And the algorithm showed a poor performance when the elements of the array were in large amounts. An optimal algorithm, the distributed SPGD (DSPGD) algorithm, was proposed to improve the performance of the array, which revealed fast speed and stability in phase-locking of a large number of emitting elements of optical phased array, suggesting the algorithm is effective.

Training model weight is a challenging research topics in the field of chaotic time series modeling and analysis, and intelligent optimization algorithm provides an effective solution to this kind of problem. Aiming at the nonlinear characteristics of dynamic systems, this paper proposes an improved particle swarm optimization (IPSO) based neural network to forecast the chaotic time series. The novel particle swarm optimization (PSO) algorithm is used to optimize the output weight matrix of a neural network, and the hybrid model is available for the chaotic time series modeling process. To enhance the generalization ability and prevent over-fitting, the objective function of this paper is the weighted sum of the L 2 norm of the loss function and the L 1 norm of the output weights. Our work improves the original PSO algorithm by improving the initialization strategy of the population. Furthermore, some new strategies are proposed to disturb the population, which can increase population diversity and enhance search performance. To verify the efficiency, we evaluate the IPSO based forecasting model with some other intelligent optimization algorithm based models. And simulations show the IPSO based model achieves the highest forecasting accuracy on both Lorenz data and Beijing air quality index data, which testifies the validity of the proposed model in chaotic time series forecasting.

Multivariate time-series prediction is a challenging research topic in the field of time-series analysis and modeling, and is continually under research. The echo state network (ESN), a type of efficient recurrent neural network, has been widely used in time-series prediction, but when using ESN, two crucial problems have to be confronted: 1) how to select the optimal subset of input features and 2) how to set the suitable parameters of the model. To solve this problem, the modified biogeography-based optimization ESN (MBBO-ESN) system is proposed for system modeling and multivariate time-series prediction, which can simultaneously achieve feature subset selection and model parameter optimization. The proposed MBBO algorithm is an improved evolutionary algorithm based on biogeography-based optimization (BBO), which utilizes an S -type population migration rate model, a covariance matrix migration strategy, and a Lévy distribution mutation strategy to enhance the rotation invariance and exploration ability. Furthermore, the MBBO algorithm cannot only optimize the key parameters of the ESN model but also uses a hybrid-metric feature selection method to remove the redundancies and distinguish the importance of the input features. Compared with the traditional methods, the proposed MBBO-ESN system can discover the relationship between the input features and the model parameters automatically and make the prediction more accurate. The experimental results on the benchmark and real-world datasets demonstrate that MBBO outperforms the other traditional evolutionary algorithms, and the MBBO-ESN system is more competitive in multivariate time-series prediction than other classic machine-learning models.

Electroencephalogram (EEG) signals play an important role in clinical diagnosis and cognitive neuroscience. Automatic classification of EEG signals is gradually becoming the research focus, which contains two procedures: feature extraction and classification. In the phase of feature extraction, a hybrid feature extraction method is proposed and the features are derived by performing linear and nonlinear feature extraction methods, which can describe abundant properties of original EEG signals. In order to eliminate irrelevant and redundant features, feature selection based on class separability is employed to select the optimal feature subset. In the phase of classification, this paper presents a novel ensemble extreme learning machine based on linear discriminant analysis. Linear discriminant analysis is used to transform training subsets that are generated by bootstrap method, through which we can increase the differences of basic classifiers and reduce generalization errors of ensemble extreme learning machine. Experiments on two different EEG datasets are conducted in this study. Class separability is investigated to verify the effectiveness of feature extraction methods. The overall classification results show that compared with other similar studies, the proposed method can significantly enhance the performance of EEG signals classification.

Predicting for long-term dynamics of complex systems from observations is a challenging topic in the field of time series modeling and analysis, and is continually under research. Noteworthily, multi-step prediction requires accurate learning of dynamics and correlations between historical data for predicting future behavior. In this paper, we proposed a modified recurrent neural network named hierarchical delay-memory echo state network (HDESN) for solving the task of multi-step chaotic time series prediction. The HDESN uses multiple reservoirs with delay-memory capabilities, which can simultaneously discover and explore the information of short-term and long-term memory hidden in the historical sequence, and extract the valuable evolution patterns through deep topology and hierarchical processing. Moreover, to ensure high-quality prediction results and reduce the computational burden as much as possible, we further design a phase-space representation strategy which can calculate a compact topology and delay-memory coefficient according to the chaotic characteristics of the data. Compared with other improved ESN-based models, the proposed HDESN does not have a larger memory capacity to capture potential evolution law hidden in the complex system layer by layer, but can also adaptively determine a suitable network architecture to reflect the mapping relations in chaotic phase space. The experimental results on two benchmark chaotic systems and a real-world meteorological dataset demonstrate that the proposed HDESN model obtains satisfactory performance in multi-step chaotic time series prediction.

Prediction of chaotic time series has broad application prospects and becomes a research hotspot. Since chaotic time series is strongly non-stationary and nonlinear, it’s difficult to predict based on any single model. Therefore, the empirical mode decomposition (EMD)-based combination model becomes an important means of prediction. To reduce the scale of prediction models of conventional combination method, this paper proposes a high-efficiency combination model using ensemble EMD (EEMD), permutation entropy (PE) and echo state network(ESN). EEMD decomposes the original time series into a group of intrinsic mode functions (IMFs), and the number of IMFs is consistent with the number of predictors. On account of the complexity of the chaotic time series, there is a large demand of predictors. Through complexity analysis by PE, we combine some IMFs whose complexities are similar, and predict the combined signals instead of the initial ones based on ESNs. Finally, these obtained estimates are assembled as the ultimate prediction results. In the experiment, we use real-world dataset to examine the proposed model. The experimental results confirm that our combination approach outperforms existing single models, and efficiently reduces the scale of prediction models comparing to the EEMD-ESN.

With the acceleration of urbanization, there is an increasing trend of heavy pollution. PM2.5, also known as fine particulate matter, refers to particles in the atmosphere with a diameter of less than or equal to 2.5 microns. PM2.5 has a serious impact on human life, a sustainable city, national economic development, and so on. How to forecast the PM2.5 concentration accurately, and then formulate a scientific air pollution prevention and monitoring program is of great significance. This paper proposes a hybrid model based on echo state network (ESN) and an improved particle swarm optimization (IPSO) algorithm for the Beijing air pollution problem, and provides a method for PM2.5 concentration forecasting. Firstly, the PSO algorithm is improved to speed up the search performance. Secondly, the optimal subset of the original data is selected by the convergence cross-mapping (CCM) method. Thirdly, the phase space reconstruction (PSR) process is combined with the forecasting model, and some parameters are optimized by the IPSO. Finally, the optimal variable subset is used to predict PM2.5 concentration. The 11-dimensional air quality data in Beijing from January 1 to December 31, 2016 are analyzed by the proposed method. The experimental results show that the hybrid method is superior to other comparative models in several evaluation indicators, both in one-step and multi-step forecasting of PM2.5 time series. The hybrid model has good application prospects in air quality forecasting and monitoring.

Air pollution has become a global environmental problem, because it has a great adverse impact on human health and the climate. One way to explore this problem is to monitor and predict air quality index in an economical way. Accurate monitoring and prediction of air quality index (AQI), e.g., PM2.5 concentration, is a challenging task. In order to accurately predict the PM2.5 time series, we propose a supplementary leaky integrator echo state network (SLI-ESN) in this paper. It adds the historical state term of the historical moment to the calculation of leaky integrator reservoir, which improves the influence of historical evolution state on the current state. Considering the redundancy and correlation between multivariable time series, minimum redundancy maximum relevance (mRMR) feature selection method is introduced to reduce redundant and irrelevant information, and increase computation speed. A variety of evaluation indicators are used to assess the overall performance of the proposed method. The effectiveness of the proposed model is verified by the experiment of Beijing PM2.5 time series prediction. The comparison of learning time also shows the efficiency of the algorithm.

Noises and outliers commonly exist in dynamical systems because of sensor disturbations or extreme dynamics. Thus, the robustness and generalization capacity are of vital importance for system modeling. In this paper, the robust manifold broad learning system(RM-BLS) is proposed for system modeling and large-scale noisy chaotic time series prediction. Manifold embedding is utilized for chaotic system evolution discovery. The manifold representation is randomly corrupted by perturbations while the features not related to low-dimensional manifold embedding are discarded by feature selection. It leads to a robust learning paradigm and achieves better generalization performance. We also develop an efficient solution for Stiefel manifold optimization, in which the orthogonal constraints are maintained by Cayley transformation and curvilinear search algorithm. Furthermore, we discuss the common thoughts between random perturbation approximation and other mainstream regularization methods. We also prove the equivalence between perturbations to manifold embedding and Tikhonov regularization. Simulation results on large-scale noisy chaotic time series prediction illustrates the robustness and generalization performance of our method.

With the rapid development of information theoretic learning, the maximum correntropy criterion (MCC) has been widely used in time series prediction area. Especially, the kernel recursive least squares (KRLS) based on MCC is studied recently due to its online recursive form and the ability to resist noise and be robust in non-Gaussian environments. However, it is not always an optimal choice that using the correntropy, which is calculated by default Gaussian kernel function, to describe the local similarity between variables. Besides, the computational burden of MCC based KRLS will raise as data size increases, thus causing difficulties in accommodating time-varying environments. Therefore, this paper proposes a quantized generalized MCC (QGMCC) to solve the above problem. Specifically, a generalized MCC (GMCC) is utilized to enhance the accuracy and flexibility in calculating the correntropy. In order to solve the problem of computational complexity, QGMCC quantizes the input space and upper bounds the network size by vector quantization (VQ) method. Furthermore, QGMCC is applied to KRLS and forming a computationally efficient and precisely predictive algorithm. After that, the improved algorithm named quantized kernel recursive generalized maximum correntropy (QKRGMC) is set up and the derivation process is also given. Experimental results of one benchmark dataset and two real-world datasets are present to verify the effectiveness of the online prediction algorithm.

Feature selection is an important preprocessing step in data mining. Mutual information-based feature selection is a kind of popular and effective approaches. In general, most existing mutual information-based techniques are greedy methods, which are proven to be efficient but suboptimal. In this paper, mutual information-based feature selection is transformed into a global optimization problem, which provides a new idea for solving feature selection problems. First, a single-objective feature selection algorithm combining relevance and redundancy is presented, which has well global searching ability and high computational efficiency. Furthermore, to improve the performance of feature selection, we propose a multi-objective feature selection algorithm. The method can meet different requirements and achieve a tradeoff among multiple conflicting objectives. On this basis, a hybrid feature selection framework is adopted for obtaining a final solution. We compare the performance of our algorithm with related methods on both synthetic and real datasets. Simulation results show the effectiveness and practicality of the proposed method.

The causality analysis is an important research topic in time series data mining. Granger causality analysis is a powerful method that determines cause and effect based on predictability. However, the traditional Granger causality is limited to analyzing linear causality between bivariate time series, because it is based on vector autoregressive models. In this paper, we propose a novel method, named Hilbert–Schmidt independence criterion Lasso Granger causality (HSIC-Lasso-GC), for revealing nonlinear causality between multivariate time series. Firstly, for each time series, we perform stationarity test and state space reconstruction to extract the historical information. Then, we build a HSIC-Lasso model of all input variables and output variable, where the optimal model is selected by generalized information criterion. Finally, according to the significance test, we get the causality analysis results from all input variables to output variable. In the simulations, we use two benchmark datasets and two actual datasets to test the effectiveness of the proposed method. The results show that the proposed method can effectively analyze nonlinear causality between multivariate time series.

Defining important information from complex biological data is of great significance in biological study. It is known that the physiological and pathological changes in an organism are usually influenced by molecule interactions. Analyzing biological data by fusing the evaluation of the individual molecules and molecule interactions could induce a more accurate and comprehensive understanding of the organism. This study proposes an Interaction Gain - Recursive Feature Elimination (IG-RFE) method which evaluates the feature importance by combining the relevance between feature and class label and the interaction among features. Symmetrical uncertainty is adopted to measure the relevance between feature and the class label. The average normalized interaction gain of feature f, every other features and the class label is calculated to reflect the interaction of feature f with other features in the feature set F. Based on the combination of symmetrical uncertainty and normalized interaction gain, less important features are removed iteratively. To show the performance of IG-RFE, it was compared with seven efficient feature selection methods, MIFS, mRMR, CMIM, ReliefF, FCBF, PGVNS and SVM-RFE, on eleven public datasets. The experiment results showed the superiority of IG-RFE in accuracy, sensitivity, specificity and stability. Hence, integrating feature individual discriminative ability and the interaction among features could better evaluate feature importance in biological data analysis.

For modeling of multivariate time series, input variable selection is a key problem. This paper presents the estimation of joint mutual information and its application in input variable selection problems. Mutual information is a commonly used measure for variable selection. To improve the performance of input variable selection, we propose a novel high-dimensional mutual information estimator based on copula entropy, which is estimated by the truncated k -nearest neighbor method. Simulations on high dimensional Gaussian distributions substantiate the effectiveness of the proposed mutual information estimator. A relationship between the joint mutual information and the copula entropy is derived, which is used for joint mutual information estimation. Then the proposed estimator is applied to input variable selection for multivariate time series modeling based on the criterion of max dependency and max–min dependency. A stop criterion is proposed to terminate the selection process automatically. Simulation results show that the input variable selection method works well on both synthetic and real life dataset.

Feature extraction and classification for EEG signals are key technologies in medical applications. This paper proposes an efficient feature extraction framework that combines hybrid feature extraction and feature selection method. In order to fully exploit information from EEG signals, several feature extraction methods of different types are applied, which are autoregressive model, discrete wavelet transform, wavelet packet transform and sample entropy. After information fusion, feature selection methods are introduced to deal with redundant and irrelevant information, which is advantageous to classification. In this phase, global optimization strategy based on binary particle swarm optimization (BPSO) is presented to enhance the performance of feature selection. To evaluate the results of feature extraction, experiments of class separability are conducted. Classification results on EEG dataset of university of Bonn show the superiority of the proposed method.

The prediction of chaotic time series has been a popular research field in recent years. Due to the strong non-stationary and high complexity of the chaotic time series, it is difficult to directly analyze and predict depending on a single model, so the hybrid prediction model has become a promising and favorable alternative. In this paper, we put forward a novel hybrid model based on a two-layer decomposition approach and an optimized back propagation neural network (BPNN). The two-layer decomposition approach is proposed to obtain comprehensive information of the chaotic time series, which is composed of complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) and variational mode decomposition (VMD). The VMD algorithm is used for further decomposition of the high frequency subsequences obtained by CEEMDAN, after which the prediction performance is significantly improved. We then use the BPNN optimized by a firefly algorithm (FA) for prediction. The experimental results indicate that the two-layer decomposition approach is superior to other competing approaches in terms of four evaluation indexes in one-step and multi-step ahead predictions. The proposed hybrid model has a good prospect in the prediction of chaotic time series.

A new learning framework is proposed for multivariate chaotic system modeling. In order to construct suitable input variables, we put forward a scheme of input variable selection based on nonuniform state space reconstruction. A new criteria based on low dimensional approximation of joint mutual information is derived, which is solved by evolutionary computation approach efficiently with low computation complexity. Then, echo state network is adopted as prediction model, which has powerful capability for nonlinear predicting. To improve generalization performance and stability of the predictive model, we introduce feature selection in the training process. Feature selection method can control complexity of the network and prevent overfitting. The model is applied to the prediction of real world time series. The simulation results show the effectiveness and practicality of the proposed method.

In this paper, we present an echo state network model based on sparse Gaussian process regression, which has been successfully applied to multivariate time series prediction. While combining the Gaussian process with Echo State Network, the computational complexity of the model is very high. We consider using a group of limited basis functions instead of the original covariance function, which reduces the computational complexity and maintains the prediction performance of the model. In the framework of Bayesian inference, the model can combine prior knowledge and observation data perfectly and provide prediction confidence. The model realizes adaptive estimation of the hyper-parameters by using maximum likelihood approach and avoids complex computation process. Two simulation results show the effectiveness and practicality of the proposed method.