Search

Considering disturbances, noise, and sensor faults, this paper investigates interval estimation for discrete-time Takagi-Sugeno fuzzy systems. To obtain precise estimation results and attenuate disturbances and noise in the system simultaneously, we integrate robust observers based on the H technique and reachability analysis. Two novel observer gain computation methods are proposed for different purposes. The time-invariant method relaxes the original design conditions by transforming the parameterized linear matrix inequality into a series of linear matrix inequalities to increase computational speed, while the time-varying method employs the parameterized linear matrix inequality directly and conducts calculation online. Furthermore, reachable set representations for error dynamics are formulated by making use of both time-invariant observer gain and time-varying observer gain. Two comparative numerical simulation examples are studied to illustrate the effectiveness and superiority of the developed methods.

Echo state network (ESN), a type of special recurrent neural network with a large-scale randomly fixed hidden layer (called a reservoir) and an adaptable linear output layer, has been widely employed in the field of time series analysis and modeling. However, when tackling the problem of multidimensional chaotic time series prediction, due to the randomly generated rules for input and reservoir weights, not only the representation of valuable variables is enriched but also redundant and irrelevant information is accumulated inevitably. To remove the redundant components, reduce the approximate collinearity among echo-state information, and improve the generalization and stability, a new method called hierarchical ESN with sparse learning (HESN-SL) is proposed. The HESN-SL mines and captures the latent evolution patterns hidden from the dynamic system by means of layer-by-layer processing in stacked reservoirs, and leverage monotone accelerated proximal gradient algorithm to train a sparse output layer with variable selection capability. Meanwhile, we further prove that the HESN-SL satisfies the echo state property, which guarantees the stability and convergence of the proposed model when applied to time series prediction. Experimental results on two synthetic chaotic systems and a real-world meteorological dataset illustrate the proposed HESN-SL outperforms both original ESN and existing hierarchical ESN-based models for multidimensional chaotic time series prediction.

The application of deep learning in time-series prediction has developed gradually. In this paper, we propose a deep generative network model for feature extraction of multivariate time series, namely, mutual information variational autoencoders (MI-VAE). In the architecture of the proposed model, we use the latent space of VAE for feature learning, which can extract the essential features of multivariate time-series data effectively. The latent space employed directly as a feature extractor can avoid poor interpretability of model. In addition, we introduce a mutual information term into the loss function, which improves the expression capability and accuracy of model. The proposed model, combining the merits of VAE and mutual information, extracts features for multivariate time-series data from a new perspective. The Lorenz system and Beijing air quality time series are used to test performance of the proposed model and comparative models. Results show that the proposed model is superior to other similar models in terms of accuracy and expression capability of latent space.

In multivariate chaotic time series prediction, correlation analysis is important for reducing input dimensions and improving prediction performance. Grey relational analysis (GRA) has proved to be an effective method for data correlation analysis, especially for inexact data and incomplete data. In GRA, points are usually regarded as objects, and the distance between points or the concave and convex degree are mostly used to measure the correlations. However, with discrete variables, correlation analysis results always tend to have some deviations when using prior GRA methods. Furthermore, GRA methods cannot directly use vector datasets. Therefore, in this paper, an improved GRA method is proposed based on vector projections. The input and output variables are expressed as vectors by linking two adjacent points. The vectors, instants of the points, are regarded as the objects, and the projection length of input variables to output variables is used to measure the correlations. The smaller the difference between the projection length and the input variables, the higher the correlation. Then, a hybrid variable selection and prediction model is proposed based on the improved GRA method for multivariate chaotic time series predictions, in order to overcome the negative effects of irrelevant and redundant variables caused by phase-space reconstruction. The experimental results based on the gas furnace dataset and San Francisco river runoff dataset demonstrate that the improved GRA method is effective for data correlation analysis, and the prediction accuracy is better than prior GRA-based methods.

Analyzing the disease data from the view of combinatorial features may better characterize the disease phenotype. In this study, a novel method is proposed to construct feature combinations and a classification model (CFC-CM) by mining key feature relationships. CFC-CM iteratively tests for differences in the feature relationship between different groups. To do this, it uses a modified $k$k-top-scoring pair (M-$k$k-TSP) algorithm and then selects the most discriminative feature pairs in the current feature set to infer the combinatorial features and build the classification model. Compared with support vector machines, random forests, least absolute shrinkage and selection operator, elastic net, and M-$k$k-TSP, the superior performance of CFC-CM on nine public gene expression datasets validates its potential for more precise identification of complex diseases. Subsequently, CFC-CM was applied to two metabolomics datasets, it obtained accuracy rates of $88.73\pm 2.06\%$88.73±2.06% and $79.11\pm 2.70\%$79.11±2.70% in distinguishing between hepatocellular carcinoma and hepatic cirrhosis groups and between acute kidney injury (AKI) and non-AKI samples, results superior to those of the other five methods. In summary, the better results of CFC-CM show that in contrast to molecules and combinations constituted by just two features, the combinations inferred by appropriate number of features could better identify the complex diseases.

Electroencephalogram (EEG) signals play an important role in clinical diagnosis and cognitive neuroscience. Automatic classification of EEG signals is gradually becoming the research focus, which contains two procedures: feature extraction and classification. In the phase of feature extraction, a hybrid feature extraction method is proposed and the features are derived by performing linear and nonlinear feature extraction methods, which can describe abundant properties of original EEG signals. In order to eliminate irrelevant and redundant features, feature selection based on class separability is employed to select the optimal feature subset. In the phase of classification, this paper presents a novel ensemble extreme learning machine based on linear discriminant analysis. Linear discriminant analysis is used to transform training subsets that are generated by bootstrap method, through which we can increase the differences of basic classifiers and reduce generalization errors of ensemble extreme learning machine. Experiments on two different EEG datasets are conducted in this study. Class separability is investigated to verify the effectiveness of feature extraction methods. The overall classification results show that compared with other similar studies, the proposed method can significantly enhance the performance of EEG signals classification.

Predicting for long-term dynamics of complex systems from observations is a challenging topic in the field of time series modeling and analysis, and is continually under research. Noteworthily, multi-step prediction requires accurate learning of dynamics and correlations between historical data for predicting future behavior. In this paper, we proposed a modified recurrent neural network named hierarchical delay-memory echo state network (HDESN) for solving the task of multi-step chaotic time series prediction. The HDESN uses multiple reservoirs with delay-memory capabilities, which can simultaneously discover and explore the information of short-term and long-term memory hidden in the historical sequence, and extract the valuable evolution patterns through deep topology and hierarchical processing. Moreover, to ensure high-quality prediction results and reduce the computational burden as much as possible, we further design a phase-space representation strategy which can calculate a compact topology and delay-memory coefficient according to the chaotic characteristics of the data. Compared with other improved ESN-based models, the proposed HDESN does not have a larger memory capacity to capture potential evolution law hidden in the complex system layer by layer, but can also adaptively determine a suitable network architecture to reflect the mapping relations in chaotic phase space. The experimental results on two benchmark chaotic systems and a real-world meteorological dataset demonstrate that the proposed HDESN model obtains satisfactory performance in multi-step chaotic time series prediction.

With the rapid development of information theoretic learning, the maximum correntropy criterion (MCC) has been widely used in time series prediction area. Especially, the kernel recursive least squares (KRLS) based on MCC is studied recently due to its online recursive form and the ability to resist noise and be robust in non-Gaussian environments. However, it is not always an optimal choice that using the correntropy, which is calculated by default Gaussian kernel function, to describe the local similarity between variables. Besides, the computational burden of MCC based KRLS will raise as data size increases, thus causing difficulties in accommodating time-varying environments. Therefore, this paper proposes a quantized generalized MCC (QGMCC) to solve the above problem. Specifically, a generalized MCC (GMCC) is utilized to enhance the accuracy and flexibility in calculating the correntropy. In order to solve the problem of computational complexity, QGMCC quantizes the input space and upper bounds the network size by vector quantization (VQ) method. Furthermore, QGMCC is applied to KRLS and forming a computationally efficient and precisely predictive algorithm. After that, the improved algorithm named quantized kernel recursive generalized maximum correntropy (QKRGMC) is set up and the derivation process is also given. Experimental results of one benchmark dataset and two real-world datasets are present to verify the effectiveness of the online prediction algorithm.

Noises and outliers commonly exist in dynamical systems because of sensor disturbations or extreme dynamics. Thus, the robustness and generalization capacity are of vital importance for system modeling. In this paper, the robust manifold broad learning system(RM-BLS) is proposed for system modeling and large-scale noisy chaotic time series prediction. Manifold embedding is utilized for chaotic system evolution discovery. The manifold representation is randomly corrupted by perturbations while the features not related to low-dimensional manifold embedding are discarded by feature selection. It leads to a robust learning paradigm and achieves better generalization performance. We also develop an efficient solution for Stiefel manifold optimization, in which the orthogonal constraints are maintained by Cayley transformation and curvilinear search algorithm. Furthermore, we discuss the common thoughts between random perturbation approximation and other mainstream regularization methods. We also prove the equivalence between perturbations to manifold embedding and Tikhonov regularization. Simulation results on large-scale noisy chaotic time series prediction illustrates the robustness and generalization performance of our method.

Defining important information from complex biological data is of great significance in biological study. It is known that the physiological and pathological changes in an organism are usually influenced by molecule interactions. Analyzing biological data by fusing the evaluation of the individual molecules and molecule interactions could induce a more accurate and comprehensive understanding of the organism. This study proposes an Interaction Gain - Recursive Feature Elimination (IG-RFE) method which evaluates the feature importance by combining the relevance between feature and class label and the interaction among features. Symmetrical uncertainty is adopted to measure the relevance between feature and the class label. The average normalized interaction gain of feature f, every other features and the class label is calculated to reflect the interaction of feature f with other features in the feature set F. Based on the combination of symmetrical uncertainty and normalized interaction gain, less important features are removed iteratively. To show the performance of IG-RFE, it was compared with seven efficient feature selection methods, MIFS, mRMR, CMIM, ReliefF, FCBF, PGVNS and SVM-RFE, on eleven public datasets. The experiment results showed the superiority of IG-RFE in accuracy, sensitivity, specificity and stability. Hence, integrating feature individual discriminative ability and the interaction among features could better evaluate feature importance in biological data analysis.

Feature selection is an important preprocessing step in data mining. Mutual information-based feature selection is a kind of popular and effective approaches. In general, most existing mutual information-based techniques are greedy methods, which are proven to be efficient but suboptimal. In this paper, mutual information-based feature selection is transformed into a global optimization problem, which provides a new idea for solving feature selection problems. First, a single-objective feature selection algorithm combining relevance and redundancy is presented, which has well global searching ability and high computational efficiency. Furthermore, to improve the performance of feature selection, we propose a multi-objective feature selection algorithm. The method can meet different requirements and achieve a tradeoff among multiple conflicting objectives. On this basis, a hybrid feature selection framework is adopted for obtaining a final solution. We compare the performance of our algorithm with related methods on both synthetic and real datasets. Simulation results show the effectiveness and practicality of the proposed method.

For modeling of multivariate time series, input variable selection is a key problem. This paper presents the estimation of joint mutual information and its application in input variable selection problems. Mutual information is a commonly used measure for variable selection. To improve the performance of input variable selection, we propose a novel high-dimensional mutual information estimator based on copula entropy, which is estimated by the truncated k -nearest neighbor method. Simulations on high dimensional Gaussian distributions substantiate the effectiveness of the proposed mutual information estimator. A relationship between the joint mutual information and the copula entropy is derived, which is used for joint mutual information estimation. Then the proposed estimator is applied to input variable selection for multivariate time series modeling based on the criterion of max dependency and max–min dependency. A stop criterion is proposed to terminate the selection process automatically. Simulation results show that the input variable selection method works well on both synthetic and real life dataset.

Liquid chromatography-mass spectrometry (LC-MS) is an important analytical platform for metabolomics study. Peak alignment of metabolomics dataset is one of the keys for a successful metabolomics study. In this work, a MS/MS-based peak alignment method for LC-MS metabolomics data was developed. A rigorous strategy for screening endogenous reference variables was proposed. Firstly, candidate endogenous reference variables were selected based on MS, MS/MS and retention time in all samples. Multiple robust endogenous reference variables were obtained through further evaluation and confirmation. Then retention time of each metabolite feature was corrected by local linear regression using the four nearest neighbor robust reference variables. Finally, peak alignment was carried out based on corrected retention time, MS and MS/MS. Comparing with the other two peak alignment methods, the developed method showed a good performance and was suitable for metabolomics data with larger retention time drift. Our approach provides a simple and robust alignment method which is reliable to align LC-MS metabolomics dataset.

Feature extraction and classification for EEG signals are key technologies in medical applications. This paper proposes an efficient feature extraction framework that combines hybrid feature extraction and feature selection method. In order to fully exploit information from EEG signals, several feature extraction methods of different types are applied, which are autoregressive model, discrete wavelet transform, wavelet packet transform and sample entropy. After information fusion, feature selection methods are introduced to deal with redundant and irrelevant information, which is advantageous to classification. In this phase, global optimization strategy based on binary particle swarm optimization (BPSO) is presented to enhance the performance of feature selection. To evaluate the results of feature extraction, experiments of class separability are conducted. Classification results on EEG dataset of university of Bonn show the superiority of the proposed method.

In systems biology, filtering the discriminative features from complex high-dimensional data is a crucial issue. This paper proposes a feature selection algorithm based on feature overlapping and group overlapping (FS-FOGO) to calculate the feature importance. FS-FOGO weighs feature from two aspects: overlapping degree based on the ratio of overlapping area on the effective range of each class and the overlapping degree based on the proportion of heterogeneous samples in every sample's nearest neighbors. To show the validation of FS-FOGO, it is compared with effective range based gene selection (ERGS), which calculates the feature weights based on overlapping area of the effective range, on six public biological data sets and one serum metabolomics data set about liver disease. Naive Bayes and Support Vector Machine are used as classifiers, respectively. The experiment results show that the top ranked features by FS-FOGO are more discriminative and get higher classification accuracy rates than those by ERGS in most cases. And in the metabolomics data, the top ranked metabolites by FS-FOGO could separate different liver diseases well.

A complete learning framework for modeling multivariate time series is presented in this paper. First, in order to construct input variables, variable selection method based on max dependency criterion is introduced, which can remove redundant and irrelevant variables. Then, Gaussian process model is adopted as prediction model, which has powerful capability of nonlinear modeling. In addition, confidence and confidence intervals are built for the evaluation of predictive results. Finally, the model is applied to the prediction of real world multivariate time series. The simulation results show the effectiveness and practicality of the proposed method.

In this paper, we present an echo state network model based on sparse Gaussian process regression, which has been successfully applied to multivariate time series prediction. While combining the Gaussian process with Echo State Network, the computational complexity of the model is very high. We consider using a group of limited basis functions instead of the original covariance function, which reduces the computational complexity and maintains the prediction performance of the model. In the framework of Bayesian inference, the model can combine prior knowledge and observation data perfectly and provide prediction confidence. The model realizes adaptive estimation of the hyper-parameters by using maximum likelihood approach and avoids complex computation process. Two simulation results show the effectiveness and practicality of the proposed method.