Search

Recurrent neural networks (RNNs) are widely utilized in time series forecasting tasks. In practical applications, there are noises in real-life time series data. A model's generalization capacity will be diminished by model uncertainty with regard to input noises. However, the robustness of RNNs with respect to input noises has not been well studied yet. The localized stochastic sensitivity (LSS), which measures output disturbances with respect to input perturbations of learning models, has been successfully applied to improve the robustness of different neural networks on tabular and image data. But its effectiveness on time series data has not been explored. Therefore, we extend the idea of LSS and apply it to the robust RNNs training for time series forecasting problems. With the minimization of LSS, output sensitivities of RNNs with respect to small perturbations are reduced. So, the proposed robust RNNs will not be affected by slight input noises. We have used the LSTM as an example for theoretical analysis and analyzed the effectiveness of LSS on several RNN variants including the vanilla RNN, the gated recurrent unit (GRU), the long-short-term memory (LSTM), and the bi-directional long short-term memory (Bi-LSTM) in empirical studies. Experimental results confirm the efficiency of applying LSS to enhance the robustness of RNNs for time series data. For example, the robust RNNs outperform their counterpart by 53.26 % and 49.45 % on average in terms of Root Mean Squared Error and R-Square on five datasets. [ABSTRACT FROM AUTHOR]

Deep learning approaches, given their low cost and high reliability, have gained much popularity in different subjects, such as computer vision and natural language processing, and more recently in graph data types. Spatiotemporal graph-based neural networks have been more and more developed to solve problems related to spatiotemporal data, mainly for analyzing changes over time and for provisioning purposes. In this systematic literature review, we have aimed to answer the most important questions regarding spatiotemporal graph deep learning architectures in different applications domains, such as traffic related topics, medical imaging, and geographical data analysis. We have selected more than 50 papers that cover a wide range of applications and very different architectures and innovations. We have also noticed that most of them consider the spatiotemporal graphs to be quite classic graphs without any further sophisticated modeling of temporal and spatial data evolution. Moreover, core problems (such as node classification, frequent pattern recognition, etc.) and application domains are not sufficiently addressed by the state-of-the-art. This study thus opens many perspectives to new developments in spatio-temporal graph deep learning with different strategies in order to solve various end-to-end tasks and other problems related to this special kind of graph. • We have conducted a large systematic literature review about the development of spatiotemporal graph deep learning. • Our selection process reached to more than 50 articles related to various applications and innovations. • We concluded that most research works use quite classic representations of spatiotemporal data. • Moreover, we noticed that graph problems and applications are not addressed in the literature. • Both findings open the way to various perspectives in spatiotemporal graph based deep learning. [ABSTRACT FROM AUTHOR]

Chaotic time series prediction, a challenging research topic in dynamic system modeling, has drawn great attention from researchers around the world. In recent years extensive researches have been done on developing chaotic time series prediction methods, and various models have been proposed. Among them, recurrent fuzzy neural networks (RFNNs) have shown significant potential in this area. Most of the proposed RFNNs learn a single function, but when dealing with chaotic time series, different outputs may be generated for a specific input based on the system's state. So, a network is required that can learn multiple functions simultaneously. Based on this concept, a novel multi-functional recurrent fuzzy neural network (MFRFNN) is proposed in this paper. MFRFNN consists of two fuzzy neural networks with Takagi-Sugeno-Kang fuzzy rules, one is used to produce the output, and the other to determine the system's state. There is a feedback loop between these two networks, which makes MFRFNN capable of learning and memorizing historical information of past observations. Employing the states allows the proposed network to learn multiple functions simultaneously. Moreover, a new learning algorithm, which employs the particle swarm optimization algorithm, is developed to train the networks' weights. The effectiveness of MFRFNN is validated using the Lorenz and Rossler chaotic time series and four real-world datasets, including Box–Jenkins gas furnace, wind speed prediction, Google stock price prediction, and air quality index prediction. Based on the root mean square error, the proposed method shows a decrease of 35.12 % , 13.95 % , and 49.62 % from the second best methods in the Lorenz time series, Box–Jenkins gas furnace, and wind speed prediction dataset, respectively. [ABSTRACT FROM AUTHOR]

[Display omitted] Probabilistic prediction of the remaining useful life (RUL) of bearings is critically important, especially in an industrial setting where unplanned maintenance needs, unscheduled equipment downtime, or catastrophic failures can cost a company millions of dollars and threaten worker safety. Current research in the field of bearing prognostics clearly shows the advantage of a deep learning-based solution, but the reliability of purely data-driven predictions is questionable in harsh industrial environments with varying operational conditions. To make this work industrially relevant, we adopt ISO guidelines to determine bearing failure thresholds (specifically ISO 10816), which are defined in the velocity domain, while considering characteristic bearing fault frequencies defined by the geometry of each bearing. We propose a two-stage Long Short-Term Memory (LSTM) model ensemble which includes: (1) a predictor step to forecast and (2) a corrector step to offset the RUL prediction. Each LSTM model within the ensemble is customized to include a Gaussian layer that captures the aleatoric uncertainty in the forecasted parameter, and the ensemble of all the individual LSTM models provides the epistemic uncertainty in the RUL prediction. We demonstrate the implementation of the proposed model on the publicly available Xi'an Jiaotong University and Changxing Sumyoung Technology Co., Ltd. (XJTU-SY) bearing dataset and establish the superiority of the model, both in terms of accuracy as well as uncertainty quantification, when compared against other commonly used techniques in the field of bearing prognostics. The ensemble model tends to explore multiple functional/forecast modes providing better uncertainty estimates when compared to Bayesian counterparts. [ABSTRACT FROM AUTHOR]

• A methodology to forecast time series with generalized neural networks is outlined. • A clever way of feature selection is proposed. • Strategies for dealing with time series with a trend are developed. This paper discusses how to forecast time series using generalized regression neural networks. The main goal is to take advantage of their inherent properties to generate fast, highly accurate forecasts. To this end, the key modeling decisions involved in forecasting with generalized regression neural networks are described. To deal with every modeling decision, several strategies are proposed. Each strategy is analyzed in terms of forecast accuracy and computational time. Apart from the modeling decisions, any successful time series forecasting methodology has to be able to capture the seasonal and trend patterns found in a time series. In this regard, some clever techniques to cope with these patterns are also suggested. The proposed methodology is able to forecast time series in an automatic way. Additionally, the paper introduces a publicly available R package that incorporates the best presented modeling approaches and transformations to forecast time series with generalized regression neural networks. [ABSTRACT FROM AUTHOR]

This work presents a novel hybrid framework called DCEnt-PredictiveNet (deep convolutional neural network (DCNN) + entropy + support vector regressor (SVR)) that concatenate both deep and handcrafted features for time series data analysis and forecasting. From the discrete wavelet transform coefficients of input time series data, computed four different handcrafted entropy features, which were then concatenated with deep features extracted using a modified DCNN. The concatenated deep and handcrafted feature vector was then fed to a SVR for prediction. The DCEnt-PredictiveNet framework was trained and tested on three time series datasets of real-world COVID-19, stock price and traffic information, and achieved mean absolute percentage errors of 0.03 %, 1.53 % and 11.41 % for daily cumulative COVID-19 positive cases, closing stock price, and hourly traffic (vehicle numbers) at one junction predictions, respectively. In addition, we incorporated local interpretable model-agnostic explanations and Shapley additive explanations methods into DCEnt-PredictiveNet to enable visualization of significant features that contributed to the model's decision-making, thereby enhancing its explainability. Our DCEnt-PredictiveNet model yielded promising and interpretable forecasting results, which can facilitate advance resource planning in hospitals for incoming COVID-19 patients, stock market investment planning, and efficient traffic control management. • Explainable hybrid framework called DCEnt-PredictiveNet for Time Series Forecasting (TSF) is proposed. • Proposed architecture consists of Discrete Wavelet Transform, Convolutional Neural Network, Entropy and Support Vector Regression. • The proposed framework can be applied to three different areas: COVID, Traffic Flow and Stock Price predictions. • Proposed model is explainable – highlights the significant deep and hand-crafted features responsible for the prediction. • Achieved promising MAPE of 0.03 %, 1.53 %, and 11.41 % for COVID, stock price, and hourly traffic predictions respectively. [ABSTRACT FROM AUTHOR]

Time series forecasting is essential for a wide range of real-world applications. Recent studies have shown the superiority of Transformer in dealing with such problems, especially long sequence time series input (LSTI) and long sequence time series forecasting (LSTF) problems. To improve the efficiency and enhance the locality of Transformer, these studies combine Transformer with CNN in varying degrees. However, their combinations are loosely-coupled and do not make full use of CNN. To address this issue, we propose the concept of tightly-coupled convolutional Transformer (TCCT) and three TCCT architectures which apply transformed CNN architectures into Transformer: (1) CSPAttention: through fusing CSPNet with self-attention mechanism, the computation cost of self-attention mechanism is reduced by 30% and the memory usage is reduced by 50% while achieving equivalent or beyond prediction accuracy. (2) Dilated causal convolution: this method is to modify the distilling operation proposed by Informer through replacing canonical convolutional layers with dilated causal convolutional layers to gain exponentially receptive field growth. (3) Passthrough mechanism: the application of passthrough mechanism to stack of self-attention blocks helps Transformer-like models get more fine-grained information with negligible extra computation costs. Our experiments on real-world datasets show that our TCCT architectures could greatly improve the performance of existing state-of-the-art Transformer models on time series forecasting with much lower computation and memory costs, including canonical Transformer, LogTrans and Informer. [ABSTRACT FROM AUTHOR]

Despite the superiority of convolutional neural networks demonstrated in time series modeling and forecasting, it has not been fully explored on the design of the neural network architecture and the tuning of the hyper-parameters. Inspired by the incremental construction strategy for building a random multilayer perceptron, we propose a novel Error-feedback Stochastic Modeling (ESM) strategy to construct a random Convolutional Neural Network (ESM-CNN) for time series forecasting task, which builds the network architecture adaptively. The ESM strategy suggests that random filters and neurons of the error-feedback fully connected layer are incrementally added to steadily compensate the prediction error during the construction process, and then a filter selection strategy is introduced to enable ESM-CNN to extract the different size of temporal features, providing helpful information at each iterative process for the prediction. The performance of ESM-CNN is justified on its prediction accuracy of one-step-ahead and multi-step-ahead forecasting tasks respectively. Comprehensive experiments on both the synthetic and real-world datasets show that the proposed ESM-CNN not only outperforms the state-of-art random neural networks, but also exhibits stronger predictive power and less computing overhead in comparison to trained state-of-art deep neural network models. [ABSTRACT FROM AUTHOR]

• A general framework of neural grey system model is developed. • Levenberg-Marquardt algorithm and Bayesian regularization are used for the semiparametric estimation. • Bayesian regularization can tune the regularization parameters automatically. • The proposed model outperforms the other existing models in 6 real-world cases. Most machine learning models are essentially "Black-box" models, of which the performance heavily relies on large scale data sets. In this work the idea of "Grey-box" modelling is adopted in order to take most advantage of known information represented by deterministic structure, and then the neural grey system model is developed. Levenberg-Marquardt algorithm is used to train the proposed model, and the Bayesian regularization is used to tune the regularized parameter automatically. Six real world case studies are presented to show the performance of the proposed model, comparing to 6 existing machine learning models and 17 grey system models. The results show that the proposed model can significantly overperform the other models and has very good generality, illustrating its high potential in a wide variety of real world applications and the efficiency of the proposed modelling method. [ABSTRACT FROM AUTHOR]

The uncertainty cargo flow problem establishes a limitation in ports management where decision-making processes need accurate information of the future values. This work aims at predicting the future values of Ro-Ro perishable cargo flow at the Port of Algeciras Bay using a machine learning-based forecasting system. Two datasets consisting of daily records of fresh fruits and vegetables between 2010 to 2017 were analyzed. Additionally, these two--time series were pre-processed applying an exponential moving average method to obtain a smoothed version of the original ones. Multiple Linear Regression, Support Vector Machines, Long Short-Term Memory networks and an ensemble approach have been used to build a forecasting system and obtain the future values of the perishable cargo. The results of the analysis showed how this machine learning-based system achieved 14.83% better performance rate than a baseline persistence model in terms of root mean squared error in the fresh fruits dataset and 11.3% better in the vegetables one. In general, models' average performance rates are better using the smoothed version of the times series rather than the original ones. [ABSTRACT FROM AUTHOR]

For time series prediction, hybrid systems that combine linear and nonlinear models can provide more accurate performance than a single model. However, the irregularity of the error series and the unknown nature of combinations of different forecasters may strongly impact the performance of hybrid systems. Therefore, in this paper, we propose a novel method for time series prediction, in which error decomposition and a nonlinear combination of forecasters are introduced. The proposed method performs the following: (i) linear modeling to obtain the error series, (ii) error decomposition by using variational mode decomposition (VMD), (iii) nonlinear modeling and a phase fix procedure for the error subseries, and (iv) a combination of forecasters through an appropriate combination function generated by a nonlinear model. By using the proposed method, this paper constructs two hybrid systems, in which the autoregressive integrated moving average (ARIMA) is used for linear modeling, and two artificial intelligence (AI) models, namely, the multilayer perceptron (MLP) and support vector regression (SVR), are used for nonlinear modeling and combination, respectively. Finally, four time series data sets, six evaluation metrics, two single models and thirteen hybrid systems are used to assess the effectiveness of the proposed method. The empirical results show that hybrid systems based on error decomposition and a nonlinear combination of forecasters can achieve better performance than some existing systems and models. [ABSTRACT FROM AUTHOR]

Time series prediction has attracted much attention as various issues can be formulated as such a task. It is one of the critical challenges to extract the intrinsic information for estimating future trends from historical data. Long Short-Term Memory Network (LSTM) shows excellent performance in this assignment. Probabilistic information extraction, which is demonstrated effective in object recognition in recent years, has not been introduced in time series prediction. To our knowledge, there has not been any work on plugging trainable probability distributions into LSTM as feature representation in an end-to-end manner. In this work we put forward an Adaptive Log-Euclidean Gaussian embedding Network (ALGeNet) to take one step further on solving this problem. The core of the network is capturing statistical information through the Gaussian Distribution with LSTM for end-to-end learning. As the space of Gaussian Distribution is a manifold, we try to embed Gaussian layer into LSTM through mapping Gaussian space into a linear space based on the Lie group and logarithm operation. We introduce four descriptors of Gaussian, two descriptors performing direct logarithm and the others performing indirect logarithm. All of them can extract first-order and second-order statistical features while utilizing the structures of geometry and smooth group of Gaussians. Furthermore, our adaptive mechanism merges the advantages of each descriptor and works well. Experimental results on a real-world wind speed dataset and a system-level electricity load dataset show that the proposed adaptive network outperforms some state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

Time series forecasting is emerging as one of the most important branches of big data analysis. However, traditional time series forecasting models can not effectively extract good enough sequence data features and often result in poor forecasting accuracy. In this paper, a novel time series forecasting model, named SeriesNet, which can fully learn features of time series data in different interval lengths. The SeriesNet consists of two networks. The LSTM network aims to learn holistic features and to reduce dimensionality of multi-conditional data, and the dilated causal convolution network aims to learn different time interval. This model can learn multi-range and multi-level features from time series data, and has higher predictive accuracy compared those models using fixed time intervals. Moreover, this model adopts residual learning and batch normalization to improve generalization. Experimental results show our model has higher forecasting accuracy and has greater stableness on several typical time series datasets. [ABSTRACT FROM AUTHOR]

It is well known that any kind of time series algorithm requires past information to model the inherent temporal relationship between past and future. This temporal dependency (i.e. number of past samples required for a good prediction) is generally addressed by feeding a number of past instances to the model in an empirical manner. Conventional approaches mostly rely on offline model, making them impractical to be adopted in the online or streaming context. Hence, a novel method of online temporality analysis is proposed in this paper. The estimated temporality is then employed to form an Adaptive Temporal Neural Network (ATNN) with an elastic memory capable of automatically selecting number of past samples to be used. Temporality change or drift can be a common occurrence in data streams. Hence a drift detection mechanism is also proposed. Once such drift is detected, a drift handling mechanism kicks in which utilizes the rate of drift, making our solution truly autonomous. The entire mechanism is termed as LEMON: Learning Elastic Memory Online. LEMON although not a time series model in itself, can work with any predictive models to improve their performance. Synthetic datasets are used here as proof of correct temporality estimation and drift detection whereas real world datasets are employed to demonstrate how LEMON improves the predictive performance and speed of an existing model with the knowledge of temporality and drift. [ABSTRACT FROM AUTHOR]

Neural attention has become a key component in many deep learning applications, ranging from machine translation to time series forecasting. While many variations of attention have been developed over recent years, all share a common component in the application of a softmax function to normalize the attention weights, in order to transform them into valid mixing coefficients. In this paper, we aim to improve the modeling flexibility of a generic attention module by innovatively replacing this softmax operation with a learnable softmax, in which the normalizing functions are also adapted from the data. Specifically, our generalized softmax builds upon recent work in learning activation functions for deep networks, in particular the kernel activation function and its extensions. We describe the application of the proposed technique for the challenging case of time series forecasting with the dual-stage attention-based recurrent neural network (DA-RNN), an innovative model for predicting time series that employs two different attention modules for handling exogenous factors and long-term dependencies. A series of real-world benchmarks are used to show that simply plugging-in our generalized attention model can improve results on all datasets, even when keeping the number of trainable parameters in the model constant. To further evaluate the algorithm, we collect a novel dataset for predicting the Bitcoin closing exchange rate, a problem of high practical significance lately. Finally, to foster research in the topic, we also release both the dataset and our model as an open source extensible library. Over a baseline DA-RNN, our proposed model delivers an improvement of MAR ranging from 6% to 15% using our newly-released dataset. [ABSTRACT FROM AUTHOR]

• A new hybrid ARIMA-ANN method is proposed for time series forecasting. • Our new hybrid method avoids making strong assumptions like existing methods. • The method achieved better forecasting accuracy than many existing ARIMA-ANN models. • The performance of the proposed hybrid method is improved by using EMD. Many applications in different domains produce large amount of time series data. Making accurate forecasting is critical for many decision makers. Various time series forecasting methods exist that use linear and nonlinear models separately or combination of both. Studies show that combining of linear and nonlinear models can be effective to improve forecasting performance. However, some assumptions that those existing methods make, might restrict their performance in certain situations. We provide a new Autoregressive Integrated Moving Average (ARIMA)-Artificial Neural Network (ANN) hybrid method that work in a more general framework. Experimental results show that strategies for decomposing the original data and for combining linear and nonlinear models throughout the hybridization process are key factors in the forecasting performance of the methods. By using these findings, the proposed hybrid method is combined with Empirical Mode Decomposition (EMD) technique which generates more predictable components. We show that our hybrid method with EMD can be an effective way to improve forecasting accuracy obtained by traditional hybrid methods and also any of the individual methods that we used separately. [ABSTRACT FROM AUTHOR]

The hybrid methodology for orbit propagation is a technique that allows improving the accuracy of any propagator for predicting the future trajectory of a satellite or space-debris object in orbit around the Earth. It is based on modeling the error of the base propagator to be enhanced. Both statistical time-series forecasting methods and machine-learning techniques can be used for that purpose. The standard procedure for developing a hybrid orbit propagator requires some initial control data, that is, a set of precise ephemerides corresponding to either real observations or accurately computed pseudo-observations, from which to model the base-propagator error dynamics. It also needs tuning the time-series forecaster from those control data. We propose an improvement to the hybrid methodology for orbit propagation, based on fitting new hybrid propagators from others previously developed for nearby initial conditions, which avoids the need for both the control data and the tuning process, and achieves comparable results. [ABSTRACT FROM AUTHOR]

Abstract In many regression and time series forecasting problems, the input data is not fully available at the beginning of the training phase. Conventional machine learning methods for batch data are not able to handle this problem. The sequential version of ELM, called Online Sequential Extreme Learning Machine (OS-ELM), addresses this problem through the least squares recursive solution for updating the network output weights. However, the implementation of OS-ELM and its extensions suffer from the problem of multicollinearity and its side effect on the variance of the weight estimates. This paper introduces a new method of sequential learning for handling the effects of multicollinearity. The proposed method, called Kalman Learning Machine (KLM), uses the Kalman filter to sequentially update the output weights of a Single Layer Feedforward Network (SLFN) based on OS-ELM. An extension of the proposed method, called Extended Kalman Learning Machine (EKLM), is presented in order to address the problem of nonlinear data. The proposed method was compared with some of the most recent and effective methods for handling the effects of multicollinearity in sequential learning problems. The experiments performed showed that the proposed method performs better than most state-of-the-art methods considering both the prediction error and training time. [ABSTRACT FROM AUTHOR]

Abstract Time series forecasting (TSF) is the task of predicting future values of a given sequence using historical data. Recently, this task has attracted the attention of researchers in the area of machine learning to address the limitations of traditional forecasting methods, which are time-consuming and full of complexity. With the increasing availability of extensive amounts of historical data along with the need of performing accurate production forecasting, particularly a powerful forecasting technique infers the stochastic dependency between past and future values is highly needed. In this paper, we propose a deep learning approach capable to address the limitations of traditional forecasting approaches and show accurate predictions. The proposed approach is a deep long-short term memory (DLSTM) architecture, as an extension of the traditional recurrent neural network. Genetic algorithm is applied in order to optimally configure DLSTM's optimum architecture. For evaluation purpose, two case studies from the petroleum industry domain are carried out using the production data of two actual oilfields. Toward a fair evaluation, the performance of the proposed approach is compared with several standard methods, either statistical or soft computing. Using different measurement criteria, the empirical results show that the proposed DLSTM model outperforms other standard approaches. [ABSTRACT FROM AUTHOR]

The use of computational intelligence models for multi-step time series forecasting tasks has presented satisfactory results in such a way that they are considered models with an excellent future for this type of problem. From the point of view of computational cost, the current alternatives combined with classical models are generating hybrid models that present even better results. Within the AutoML category, the optimization of hyperparameters and the selection of network topologies has become a challenge. Reservoir Computing, which is within the area of ​​Recurrent Neural Networks (RNN), proposes a particular model called Echo State Networks. which has been tested in different applications with excellent results; however, the difficulty in specifying the hyperparameters has been the subject of continuous study given the random nature of the set of neurons called Reservoir. Based on the Separation Ratio Graph (SRG) model for performance analysis, this paper proposes a new model, called Echo State Network - Genetic Algorithm - Separation Ratio Graph (ESN-GA-SRG), which optimizes network hyperparameters and at the same time selects the best topology for the Reservoir using the SRG coefficient, to find the reservoir that offers the most suitable dynamic behavior. The performance of this new model is evaluated on forecasting two sets of time series benchmarks with different characteristics of sampling periodicity, skewness, and stationarity. The results obtained show that the ESN-GA-SRG model was superior in predicting these time series in most cases, with statistical significance, when compared to other models that have been presented for this type of problem in the literature. [ABSTRACT FROM AUTHOR]

Linear autoregressive models serve as basic representations of discrete time stochastic processes. Different attempts have been made to provide non-linear versions of the basic autoregressive process, including different versions based on kernel methods. Motivated by the powerful framework of Hilbert space embeddings of distributions, in this paper we apply this methodology for the kernel embedding of an autoregressive process of order p . By doing so, we provide a non-linear version of an autoregressive process, that shows increased performance over the linear model in highly complex time series. We use the method proposed for one-step ahead forecasting of different time-series, and compare its performance against other non-linear methods. [ABSTRACT FROM AUTHOR]

Time series forecasting is of fundamental importance in big data analysis. The prediction of noisy, non-stationary, and chaotic time series demands good generalization from small amounts of data. Vapnik showed that the total risk is dependent on both, empirical error as well as model complexity, where the latter may be measured in terms of the Vapnik-Chervonenkis (VC) dimension. In other words, good generalization requires minimizing model complexity. The recently proposed Minimal Complexity Machine (MCM) has been shown to minimize a tight bound on the VC dimension, and has further been extended to Minimal Complexity Machine Regression (MCMR). In this paper, we present an original approach based on the MCM regressor, which builds sparse and accurate models for short-term time series forecasting. Results on a number of datasets establish that the proposed approach is superior to a number of state-of-the-art methods, and yields sparse models. These sparse models are able to extract only the most important information present in the data sets, thereby achieving high accuracy. Sparsity in time series forecasting models is also important in reducing the evaluation time. This assumes importance when the models need to be evaluated in real time, such as when they are used as part of trading flows. [ABSTRACT FROM AUTHOR]

Time series forecasting is an important research field in machine learning. Since the literature shows several techniques for the solution of this problem, combining outputs of different models is a simple and robust strategy. However, even when using combiners, the experimenter may face the following dilemma: which technique should one use to combine the individual predictors? Inspired by classification and pattern recognition algorithms, this work presents a dynamic selection method of forecast combiners. In the dynamic selection, each test pattern is submitted to a certain combiner according to a nearest neighbor rule. The proposed method was used to forecast eight time series with chaotic behavior in short and long term. In general, the dynamic selection presented satisfactory results for all datasets. [ABSTRACT FROM AUTHOR]

Thanks to social media platforms, topics are diffused online. The ones which diffuse rapidly and widely are known as viral topics. Predicting the popularity of viral topics is important to online recommendation systems and marketing services. However, the task is still challenging due to the fact that the popularity of viral topics is highly dynamic and little research has been focused on how to predict the popularity accurately for viral topics. This paper first uses data collected from the largest BBS in China to find high correlations for the short-term popularity of viral topics. Based on this finding, this paper then utilizes a time series feature space to capture the behaviors of popularity of viral topics and presents a method for predicting the short-term popularity of a given viral topic by using only data of historical popularity of the topic. This paper demonstrates that our method outperforms a baseline model and one of the most sophisticated methods in the terms of MAPE (mean absolute percentage error) and RAE (relative absolute error). Furthermore, this paper gives out how long the minimum observation period is in order to predict promptly. Finally, this paper uses two cases to show the effectiveness and simplicity of our method. [ABSTRACT FROM AUTHOR]

In the field of time series forecasting, the combination of linear and nonlinear models has been explored to improve the accuracy of predictions when compared with the performance of individual models. Traditional forecasting methods such as the autoregressive integrated moving average (ARIMA) and exponential smoothing (ETS) are employed to map linear patterns in a time series while neural networks (ANNs) and support vector machines (SVMs) map nonlinear patterns. A traditional hybrid ARIMA–ANN system works by performing linear relationships in the data, considering the residual error a nonlinear component which is mapped by the ANN. The nature of a time series can be taken into consideration before applying a model. This work employs both linear and nonlinear patterns of a time series based on its volatility. This is explored using a hybrid evolutionary system composed by a simple exponential smoothing filter, ARIMA and autoregressive (AR) linear models and a SVR model. Particle swarm optimization is employed to optimize the order of the AR model, SVR parameters and number of lags of the time series. Experimental results show that the evolutionary hybrid system presented promising results in the forecasting domain. [ABSTRACT FROM AUTHOR]

In this paper, we discuss the long-term time series forecasting using a Multilayer Neural Network with Multi-Valued Neurons (MLMVN). This is a complex-valued neural network with a derivative-free backpropagation learning algorithm. We evaluate the proposed approach using a real-world data set describing the dynamic behavior of an oilfield asset located in the coastal swamps of the Gulf of Mexico. We show that MLMVN can be efficiently applied to univariate and multivariate one-step- and multi-step ahead prediction of reservoir dynamics. This paper is not only intended for proposing to use a complex-valued neural network for forecasting, but to deeper study some important aspects of the application of ANN models to time series forecasting that could be of the particular interest for pattern recognition community. [ABSTRACT FROM AUTHOR]

Combining time series forecasts from several models is a fruitful alternative to using only a single individual model. In the literature, it has been widely documented that a combined forecast improves the overall accuracy to a great extent and is often better than the forecast of each component model. The accuracy of a linear combination of forecasts primarily depends on the associated combining weights. Despite extensive research in this direction, finding out the most appropriate weights is still very challenging. This paper proposes a linear combination method for time series forecasting that determines the combining weights through a novel neural network structure. The designed neural network successively recognizes the weight patterns of the constituent models from their past forecasting records and then predicts the desired set of the combining weights. Empirical results from eight real-world time series show that our approach provides significantly better forecasting accuracies than the component models and other well recognized linear combination schemes. These findings are also verified through ranking methods and a non-parametric statistical test. [ABSTRACT FROM AUTHOR]

Abstract: Multi-layer perceptron (MLP) and other artificial neural networks (ANNs) have been widely applied to time series forecasting since 1980s. However, for some problems such as initialization and local optima existing in applications, the improvement of ANNs is, and still will be the most interesting study for not only time series forecasting but also other intelligent computing fields. In this study, we propose a method for time series prediction using Hinton and Salakhutdinov׳s deep belief nets (DBN) which are probabilistic generative neural network composed by multiple layers of restricted Boltzmann machine (RBM). We use a 3-layer deep network of RBMs to capture the feature of input space of time series data, and after pretraining of RBMs using their energy functions, gradient descent training, i.e., back-propagation learning algorithm is used for fine-tuning connection weights between “visible layers” and “hidden layers” of RBMs. To decide the sizes of neural networks and the learning rates, Kennedy and Eberhart׳s particle swarm optimization (PSO) is adopted during the training processes. Furthermore, “trend removal”, a preprocessing to the original data, is also approached in the forecasting experiment using CATS benchmark data. Additionally, approximating and short-term prediction of chaotic time series such as Lorenz chaos and logistic map were also applied by the proposed method. [Copyright &y& Elsevier]

Abstract: The Box–Cox transformation is a technique mostly utilized to turn the probabilistic distribution of a time series data into approximately normal. And this helps statistical and neural models to perform more accurate forecastings. However, it introduces a bias when the reversion of the transformation is conducted with the predicted data. The statistical methods to perform a bias-free reversion require, necessarily, the assumption of Gaussianity of the transformed data distribution, which is a rare event in real-world time series. So, the aim of this study was to provide an effective method of removing the bias when the reversion of the Box–Cox transformation is executed. Thus, the developed method is based on a focused time lagged feedforward neural network, which does not require any assumption about the transformed data distribution. Therefore, to evaluate the performance of the proposed method, numerical simulations were conducted and the Mean Absolute Percentage Error, the Theil Inequality Index and the Signal-to-Noise ratio of 20-step-ahead forecasts of 40 time series were compared, and the results obtained indicate that the proposed reversion method is valid and justifies new studies. [Copyright &y& Elsevier]

Abstract: This paper describes the coevolutionary algorithm L-Co-R (Lags COevolving with Radial Basis Function Neural Networks – RBFNs), and analyzes its performance in the forecasting of time series in the short, medium and long terms. The method allows the coevolution, in a single process, of the RBFNs as the time series models, as well as the set of lags to be used for predictions, integrating two genetic algorithms with real and binary codification, respectively. The individuals of one population are radial basis neural networks (used as models), while sets of candidate lags are individuals of the second population. In order to test the behavior of the algorithm in a new context of a variable horizon, 5 different measures have been analyzed, for more than 30 different databases, comparing this algorithm against six existing algorithms and for seven different prediction horizons. Statistical analysis of the results shows that L-Co-R outperforms other methods, regardless of the horizon, and is capable of predicting short, medium or long horizons using real known values. [Copyright &y& Elsevier]

Abstract: A new algebraic forecasting method with internal smoothing is proposed for short-term time series prediction. The concept of the H-rank of a sequence is exploited for the detection of a base algebraic fragment of the time series. Evolutionary algorithms are exploited for the identification of the set of corrections which are used to perturb the original time series. The proposed forecasting method is constructed to find a near-optimal balance between the variability of algebraic predictors and the smoothness of averaging methods. Numerical experiments with an artificially generated and real-world time series are used to illustrate the potential of the proposed method. [Copyright &y& Elsevier]

Abstract: In this paper, we address a regression problem for economic data forecasting by using multiple-kernel learning (MKL) and propose a novel two-step multiple-kernel regression (MKR) method. The proposed MKR method firstly reformulates learning from linear convex combination of the basis kernels as a maximum eigenvalue problem. The optimal weights of basis kernels in the combination can be conveniently derived from solving the maximum eigenvalue problem by eigenvalue decomposition instead of solving complicated optimization like most existing MKR algorithms. By means of SVR optimization routine, finally, we can learn from basis kernels which have different predictive ability so as to improve prediction performance. More significantly, the way to address MKR problem can make sense of the weights and the correspondingly optimal kernel in terms of interpretability. To evaluate performance, the proposed MKR method is compared with the state-of-the-art methods on three real sets of economics data. The experimental results prove that the proposed two-step MKR method outperforms the other methods in terms of prediction performance and model selection, and demonstrates satisfied efficiency. [Copyright &y& Elsevier]

Abstract: The ability to forecast the future based on past data is a key tool to support individual and organizational decision making. In particular, the goal of Time Series Forecasting (TSF) is to predict the behavior of complex systems by looking only at past patterns of the same phenomenon. In recent years, several works in the literature have adopted Evolutionary Artificial Neural Networks (EANNs) for TSF. In this work, we propose a novel EANN approach, where a weighted n-fold validation fitness scheme is used to build an ensemble of neural networks, under four different combination methods: mean, median, softmax and rank-based. Several experiments were held, using six real-world time series with different characteristics and from distinct domains. Overall, the proposed approach achieved competitive results when compared with a non-weighted n-fold EANN ensemble, the simpler 0-fold EANN and also the popular Holt–Winters statistical method. [Copyright &y& Elsevier]

Abstract: A self-organizing complex neuro-fuzzy intelligent approach using complex fuzzy sets (CFSs) is presented in this paper for the problem of time series forecasting. CFS is an advanced fuzzy set whose membership function is characterized within a unit disc of the complex plane. With CFSs, the proposed complex neuro-fuzzy system (CNFS) that acts as a predictor has excellent adaptive ability. The design for the proposed predictor comprises the structure and parameter learning stages. For structure learning, the FCM-Based Splitting Algorithm for clustering was used to determine an appropriate number of fuzzy rules for the predictor. For parameter learning, we devised a learning method that integrates the method of particle swarm optimization and the recursive least squares estimator in a hybrid and cooperative way to optimize the predictor for accurate forecasting. Four examples were used to test the proposed approach whose performance was then compared to other approaches. The experimental results indicate that the proposed approach has shown very good performance and accurate forecasting. [Copyright &y& Elsevier]

Abstract: We propose a novel homogeneous neural network ensemble approach called Generalized Regression Neural Network (GEFTS–GRNN) Ensemble for Forecasting Time Series, which is a concatenation of existing machine learning algorithms. GEFTS uses a dynamic nonlinear weighting system wherein the outputs from several base-level GRNNs are combined using a combiner GRNN to produce the final output. We compare GEFTS with the 11 most used algorithms on 30 real datasets. The proposed algorithm appears to be more powerful than existing ones. Unlike conventional algorithms, GEFTS is effective in forecasting time series with seasonal patterns. [Copyright &y& Elsevier]

Abstract: A new short-term time series forecasting method based on the identification of skeleton algebraic sequences is proposed in this paper. The concept of the rank of the Hankel matrix is exploited to detect a base fragment of the time series. Particle swarm optimization and evolutionary algorithms are then used to remove the noise and identify the skeleton algebraic sequence. Numerical experiments with an artificially generated and a real-world time series are used to illustrate the functionality of the proposed method. [Copyright &y& Elsevier]

Abstract: Time series forecasting by fuzzy inference systems based on optimal non-uniform attractor embedding in the multidimensional delay phase space is analyzed in this paper. A near-optimal set of time lags is identified by evolutionary algorithms after the optimal dimension of the reconstructed phase space is determined by the FNN (false nearest neighbors) algorithm. The fitness function is constructed in such a way that it represents the spreading of the attractor in the delay coordinate space but does not contain any information on prediction error metrics. The weighted one-point crossover rule enables an effective identification of near-optimal sets of non-uniform time lags which are better than the globally optimal set of uniform time lags. Thus the reconstructed information on the properties of the underlying dynamical system is directly elaborated in the fuzzy prediction system. A number of numerical experiments are used to test the functionality of this method. [Copyright &y& Elsevier]

Abstract: A new method for identification of an optimal set of time lags based on non-uniform attractor embedding from the observed non-linear time series is proposed in this paper. Simple deterministic method for the determination of non-uniform time lags comprises the pre-processing stage of the time series forecasting algorithm which is implemented in the form of a fuzzy inference system. Identification of embedding parameters of the underlying dynamical system includes not only optimization of time lags but also determination of optimal dimension of the reconstructed phase space. Experiments done with benchmark chaotic time series show that the proposed method can considerably improve the forecasting accuracy. The proposed method seems to be an efficient candidate for prediction of time series with multiple time scales and noise. [Copyright &y& Elsevier]

Abstract: Time series forecasting is an active research area that has drawn considerable attention for applications in a variety of areas. Auto-Regressive Integrated Moving Average (ARIMA) models are one of the most important time series models used in financial market forecasting over the past three decades. Recent research activities in time series forecasting indicate that two basic limitations detract from their popularity for financial time series forecasting: (a) ARIMA models assume that future values of a time series have a linear relationship with current and past values as well as with white noise, so approximations by ARIMA models may not be adequate for complex nonlinear problems; and (b) ARIMA models require a large amount of historical data in order to produce accurate results. Both theoretical and empirical findings have suggested that integration of different models can be an effective method of improving upon their predictive performance, especially when the models in the ensemble are quite different. In this paper, ARIMA models are integrated with Artificial Neural Networks (ANNs) and Fuzzy logic in order to overcome the linear and data limitations of ARIMA models, thus obtaining more accurate results. Empirical results of financial markets forecasting indicate that the hybrid models exhibit effectively improved forecasting accuracy so that the model proposed can be used as an alternative to financial market forecasting tools. [Copyright &y& Elsevier]