Search

Long short-term memory (LSTM) network is extensively researched as an effective tool for time series prediction, the addition of spatial attention can portray the spatial relationship between inputs and outputs. However, the input features are not extracted adequately and are weakly representative, resulting in models with inadequate learning ability, and the cumulative errors lead to inaccurate output results. To surmount the above matters, a novel improved LSTM-based model, namely residual-driven and spatial-attentive LSTM networks with implicit features, is proposed to forecast time series. At first, abundant implicit features are extracted by means of the convolutional layer to enhance the learning ability of the model. Then, the preliminary predictive results are derived by utilizing the spatial–temporal attention-based LSTM module. Subsequently, a new kernel ridge regression (KRR)-based residual-driven module is designed to correct the above results to further develop the forecasting performance in a low time-consuming. Finally, the experiments on public time series datasets from different fields depict the effective performance of the designed method in contrast to other tested models. [ABSTRACT FROM AUTHOR]

Time series data prediction is crucial in system control, social management, and economic production. For the complex features of time series data and the massive amount of arithmetic in deep learning, a novel network model is proposed based on the broad learning architecture to handle time series prediction tasks. The model utilizes the restricted Boltzmann machine (RBM) in the mapping layer to learn feature information from the input data. Simultaneously, it employs the echo state network (ESN) as the fundamental unit in the enhancement layer to fit the learned feature information from the mapping layer. The proposed model's predictive performance has been validated on air quality index, PM2.5, and electric power datasets. Compared with the benchmark model, it has the advantage of training speed and can reduce prediction error by up to 36%, proving its effectiveness in time series prediction tasks. [ABSTRACT FROM AUTHOR]

The rock drilling process is a critical component of underground mining, and its operation time is a crucial factor in mine planning and production scheduling optimization; consequently, it is essential to make an accurate prediction of rock drilling operation time using historical time series data. This study proposes a combination prediction model for underground mine rock drilling time based on Seasonal and Trend decomposition using Loess (STL). The STL model decomposes the historical time series data into the trend, seasonal, and random components, uses the Deep Belief Network - Extreme Learning Machine (DBN-ELM) model to predict the trend component, the Support Vector Regression (SVR) model to predict the seasonal component, and the historical mean value to predict the random component, and then overlays and reconstructs the prediction results of the three components to obtain the predicted values of the final working hours of rock drilling operations. The experimental results indicate that the Mean Absolute Percentage Error (MAPE) of the prediction result of the trend component with the highest weight proportion is 0.0159%, whereas the MAPE of the prediction result of the three components superposition reconstruction model is 0.2135%, representing a significant improvement in prediction accuracy compared to each comparison model and a broad range of practical applications. • The issue of predicting the rock drilling time is studied. • An ensemble model for time series prediction of rock drilling time is established. • Combined use of deep learning and machine learning. • The effect of the dataset on the prediction results is also considered. [ABSTRACT FROM AUTHOR]

Although adaptive network fuzzy inference system and fuzzy functions approach can be utilized as a prediction tool, they have been not designed for prediction problem and they ignore the dependency structure of time series observations. From this viewpoint, making a design of the method that considers the dependency structure of observations will provide to get more accurate prediction. In this study, an intuitionistic time series fuzzy inference system (I-TSFIS) has been proposed. In the I-TSFIS, in just the same way as in the intuitionistic fuzzy inference systems, not only the membership values and crisp observations but also the non-membership values are used as inputs. Moreover, due to the use of crisp values as targets and outputs, the output does not need to be deffuzzified. Non-linear relationships between inputs and outputs of the proposed I-TSFIS are determined by Sigma-Pi neural network (SP-NN). The obtaining of optimal weights of SP-NN is performed by modified particle swarm optimization. And also I-TSFIS uses intuitionistic fuzzy C-means to obtain fuzzy clusters, membership and non-membership values of observations for these clusters. To evaluate the prediction performance of the proposed I-TSFIS, various real-life time series data sets have been analyzed and the results demonstrate the superior prediction ability of the proposed I-TSFIS. • An IS with dependency structure of observations can ensure successful prediction. • We have proposed an intuitionistic time series fuzzy inference system (I-TSFIS). • I-TSFIS, as inputs, uses the membership, the non-membership and the crisp values. • I-TSFIS forms non-linear relationships between inputs and outputs via SP-NN. • The training of SP-NN is performed by modified particle swarm optimization. [ABSTRACT FROM AUTHOR]

Complicated road conditions make organic Rankine cycle (ORC) operation characteristics show hysteresis and uncertainty. Under the strong coupling correlation of many operating parameters, how to realize the dynamic optimization of ORC comprehensive performance is the key to obtain practical application potential. Based on ensemble learning mechanism, neural network modeling, ensemble system, unsupervised learning, partial mutual information and optimization algorithm are integrated. This paper presents a nonlinear time series prediction and dynamic multi-objective optimization scheme. The average accuracy increased by at least 59.6%. Taking the thermodynamic performance and environmental impact as optimization objectives, dynamic multi-objective optimization is carried out under road conditions. The optimization scheme can effectively trade off the nonlinear correlation between thermal efficiency and emissions of CO 2 equivalent. • A nonlinear modeling scheme is proposed for ORC multivariate time series prediction. • An integrated dynamic multi-objective optimization for ORC under driving cycle. • Timing characteristics of parameters during driving cycle are investigated. • The proposed scheme shows better accuracy and robustness. [ABSTRACT FROM AUTHOR]

Although research on time series prediction based on deep learning is being actively carried out in various industries, deep learning technology still has a high entry barrier for researchers who have not majored in computer science. This paper presents a tutorial on time series prediction using a deep learning-based model. The entire process of time series data prediction is presented—from data collection to evaluation of prediction results. The details of each step are shown through a case example of predicting peak electricity demand and system marginal price of Jeju Island in Korea using the 1D-CNN and BiLSTM model. In Jeju Island, the proportion of renewable energy in the total power generation capacity is increased to 67% in 2021, requiring more accurate electricity demand forecasts. Therefore, using 808 days of training data from February 2018, electricity demand and SMP for the next 21 days were predicted. To make it easier for readers to follow, the example uses only open public data, and the entire Python source code is shared via a GitHub repository. The prediction error calculated by WRMSSE showed 0.42 in electricity demand and 0.63 in SMP max. A WRMSSE value less than one means that the forecast was relatively good, that is, better than naïve forecasting. This tutorial is not limited to the energy industry but can be utilized for any application requiring time-series data prediction. This article is expected to be of great help to researchers who need to understand the process of time series prediction using deep learning and use it for application in their industry. [ABSTRACT FROM AUTHOR]

In this paper, a dynamic model for online time series prediction is proposed, namely kernel general loss algorithm based on evolving participatory learning (EPL-KGLA). The algorithm can develop its structure and parameters autonomously in response to complex environments, capturing the dynamic changes of time series and achieving accurate prediction. Specifically, EPL based on evolving fuzzy systems is employed in recursive clustering to fully utilize useful information in data streams and generate/prune structures to ensure compactness and reduce computational burden. Then, the general loss function is combined with online kernel learning to propose KGLA for updating consequent parameters in real-time, capturing the dynamic features of data streams and avoiding the negative effects of large anomalies or complex noise on model performance, thereby improving prediction accuracy. Finally, simulation experiments on a benchmark dataset and two real-world datasets are verified the robustness of EPL-KGLA. [ABSTRACT FROM AUTHOR]

A broad variety of real-world problems have been solved using multilayer perceptron (MLP) artificial neural networks (ANNs). Optimization techniques aid ANNs to select suitable weights and achieve correct results. Recently, the kidney-inspired algorithm (KA) has been proposed for optimization problems. This algorithm is based on the filtration, reabsorption, secretion, and excretion processes that take place in the kidneys of the human body. In the KA, the value of α in the filtration rate formula is a constant value in the range of [0, 1] that is set in the initialization stage of the algorithm. In this paper, an improved KA for optimization of the ANN model is presented in which the filtration rate is controlled by changing the value of α from minimum to maximum during the search process, which helps in achieving a better balance between exploration and exploitation in the algorithm. In this algorithm if more solutes are filtered and moved to filtered blood it means that the algorithm has more exploration. In contrast, if more solutes move to waste it means that more exploitation is performed by the algorithm . In addition, the separate use of three chaotic maps instead of a random number in the movement formula of the modified KA is investigated in order to assess the ability of each map to help to achieve superior results. The proposed method is tested on benchmark classification and time series prediction problems. The method is also applied to a real-world rainfall forecasting problem. The results of a statistical analysis prove the ability of the method. [ABSTRACT FROM AUTHOR]

The leading causes of earthquakes are crustal movements, plate movements, and collisions. In recent years, many researchers in earthquake prediction have been predicting earthquakes from historical seismic data in local areas. This approach ignores the underlying internal patterns of crustal motion, plate movement, and collisions. This paper proposes a purely data-driven deep learning model called EPT. The model uses gated feature extraction blocks (GFEB) to mine potential crustal motion and plate movement patterns from global historical seismic catalog data. It uses them to aid mainshock prediction in each local, provincial region. Experiments show that this approach improves model prediction accuracy by up to 50 percent. We also use multi-headed self-attention for the first time to capture long-term dependencies within regional time series, highlighting links between focal features and compensating for the difficulty of focusing on longer-term information in long-term time series with long short-term memory networks (LSTM). In addition, we also use the gradient harmonization mechanism classification (GHMC) loss function for the first time in earthquake prediction, effectively addressing the problem of uneven data distribution across different earthquake magnitude ranges. Finally, we validated the effectiveness of the EPT model in five provincial datasets in mainland China, and the experimental results all achieved an accuracy of over 90 percent. [ABSTRACT FROM AUTHOR]

Echo state network (ESN), a novel type of recurrent neural network, possesses high nonlinear mapping capability, which is particularly appropriate for time series prediction. However, the huge reservoir may lead to ill-conditioned solutions in the output weight matrix, reducing the generalization ability and prediction performance of the network. To address this issue, a t-distributed stochastic neighbor embedding ESN (TESN) is proposed in this paper to replace the initial large-scale reservoir state matrix with a low-dimensional manifold. By maintaining the local neighbor relationship of the data in the original high-dimensional space, the ill-conditioned dilemma of the output weight matrix is successfully solved. Moreover, the proposed TESN has a strong ability to preserve the global features of the data, which effectively improves the prediction performance of the network. The superiority of the TESN model is demonstrated through two benchmark prediction tasks and a practical application. [ABSTRACT FROM AUTHOR]

Accurate prediction of soil seismic response is necessary for geotechnical engineering. The conventional physics-based models such as the finite element method (FEM) usually fail to obtain accurate predictions due to the model assumption and parameter uncertainties. And the physics-based models are computationally expensive. This study proposes deep learning models to develop data-driven surrogate models for the prediction of soil seismic response based on the recorded ground motions from KiK-net downhole array sites. Two kinds of advanced neural networks, convolution neural network (CNN) and long short-term memory (LSTM) neural network, are applied in this framework respectively. These models do not rely on any prior knowledge about the soil site. The performance of the deep learning models is demonstrated through both numerical and recorded examples. Compared with the state-of-art FEM models, the proposed models could achieve better prediction performance with higher efficiency. The average prediction error is reduced by more than 40% in time domain and 30% in frequency domain. Even though great variability exists during the propagation of seismic in the reality, the models can still get satisfactory predictions. • CNN and LSTM models are proposed to develop data-driven surrogate models for the prediction of soil seismic response. • Recorded acceleration time histories from downhole array sites are applied to train the deep learning models. • The prediction performance of the deep learning models is compared with the state-of-art FEM models. [ABSTRACT FROM AUTHOR]

The task of time series forecasting is to predict the future trend of data based on the collected historical data, providing theoretical and data support for human judgment and decision making. Randomization-based echo state networks (ESNs) are widely used in the research and application field of time series analysis for their simple structure and fast training speed. The core of the ESN is its dynamic reservoir, which the original reservoir are randomly generated and controlled only by parameter sparsity, often performing poorly on complex tasks and affecting the performance of networks. Manual design of the topology of reservoir is difficult, time-consuming and inconvenient to operate, which is not conducive to the development of ESNs. The construction of a suitable reservoir topology for practical application problems to enrich reservoir dynamics is a hot research point for researchers. In this paper, an automatic optimization method is introduced into the topology optimization of ESN (TP-ESN), and the particle swarm optimization algorithm is used to optimize the topological construction of the ESN. The connection structure between the reservoir neurons is first encoded and then iteratively optimized. The optimized structure is decoded and then the reservoir is initialized for ESN training. Prediction results on Mackey–Glass benchmark time series and two electroencephalogram (EEG) datasets demonstrate that TP-ESN method can have better adaptability, stronger prediction ability and stability than several other manually designed ESN reservoir topologies in the case of relatively complex tasks. [ABSTRACT FROM AUTHOR]

Grey theory is one of the most common methods for solving uncertain problems using limited data and poor information, due to its high performance in time series prediction. However, the inappropriate background value and initial value are the main factors affecting prediction accuracy of the Grey Model GM(1,1). An improved grey model based on particle swarm optimization algorithm named PGM(1,1) is proposed for time series prediction in this paper. The development coefficient of the grey model is calculated by PGM(1,1) based on particle swarm optimization, targeting at minimizing the average relative errors between the restored value and real value of the model to avoid the problem caused by background value optimization. In addition, the initial value of the Grey Model GM(1,1) is optimized and a sliding window is introduced to improve both precision and adaptability. Finally, three data sets, featuring increasing trend, decreasing trend, and wide fluctuations, are used in the experiments, showing that the proposed method achieves better prediction accuracy. [ABSTRACT FROM AUTHOR]

In this work we are presenting an approach for fuzzy aggregation in ensembles of neural networks for forecasting. The aggregator is used in an ensemble to combine the outputs of the networks forming the ensemble. This is done in such a way that the total output of the ensemble is better than the outputs of the individual modules. In our approach a fuzzy system is used to estimate the weights that will be assigned to the outputs in the process of combining them in a weighted average calculation. The uncertainty in the process of aggregation is modeled with interval type-3 fuzzy, which in theory can outperform type-2 and type-1. Publicly available data sets of COVID-19 cases for several countries in the world were utilized to test the proposed approach. Simulation results of the COVID-19 data show the potential of the approach to outperform other aggregators in the literature. [ABSTRACT FROM AUTHOR]

Representation learning impacts the performance of Machine Learning (ML) models. Feature extraction-based methods such as Auto-Encoders (AEs) are used to find new, more accurate data representations from original ones. They perform efficiently on a specific task, in terms of: (1) high accuracy, (2) large short-term memory and (3) low execution time. The Echo-State Network (ESN) is a recent specific kind of a Recurrent Neural Networks (RNN), that presents very rich dynamics on account of its reservoir-based hidden layer. It is widely used in dealing with complex non-linear problems and has been shown to outperform classical approaches in a number of benchmark tasks. In this paper, the powerful dynamism and large memory provided by the ESN and complementary strengths of AEs in feature extraction are integrated, to develop a novel Echo-State Recurrent Autoencoder (ES-RA). In order to devise more robust alternatives to conventional reservoir-based networks, both single- (SL-ES-RA) and multi-layer (ML-ES-RA) models are formulated. The new features, once extracted from ESN's hidden layer, are applied to various benchmark ML tasks including classification, time series prediction and regression. A range of evaluation metrics are shown to improve considerably compared to those obtained when applying original data features. An accuracy-based comparison is performed between our proposed recurrent AEs and two variants of ELM feed-forward AEs (Single and ML), for both noise free and noisy data. In summary, a comparative empirical study reveals the key contribution of exploiting recurrent connections in improving benchmark performance results. [ABSTRACT FROM AUTHOR]

Improving time series forecasting accuracy has received considerable attention in recent years. This paper presents a new hybrid evolutionary algorithm for determining both architecture (input variables and neurons of hidden layer) and network parameters (centers, width and weights) of radial basis function neural networks (RBFNNs) simultaneously. Our proposed algorithm generates new architecture applying genetic algorithm (GA). Modified adaptive particle swarm optimization (APSO) is used to determine the training parameters efficiently. Inertia weight and acceleration coefficients in APSO are adapted by swarm status. Since PSO algorithms suffer premature convergence, especially when global best is found, mutation operator is applied to overcome the drawback. Comparing the performance of the proposed approach with several benchmark time series modeling and algorithms shows that the proposed method is able to predict time series more accurately than others. Finally, proposed GA–APSO based RBFNNs method is applied to predict the demand of emergency supplies after earthquake in the East Azerbayjan in 2012 in Iran. The results show that the proposed evolving RBF based method can be applied to forecast the emergency supply demand time series successfully with the automatically selected nodes and inputs. [ABSTRACT FROM AUTHOR]

Abstract: Construction projects frequently face cost overruns during the construction phase. Thus, a proactive approach is essential for monitoring project costs and detection of potential problems. In construction management, Estimate at Completion (EAC) is an indicator for assisting project managers in identifying potential problems and developing appropriate responses. This study utilizes weighted Support Vector Machine (wSVM), fuzzy logic, and fast messy Genetic Algorithm (fmGA) to handle distinct characteristics in EAC prediction. The wSVM is employed as a supervised learning technique that can address the features of time series data. The fuzzy logic is aimed to enhance the model capability of approximate reasoning and to deal with uncertainty in EAC prediction. Moreover, fmGA is utilized to optimize model''s tuning parameters. Simulation results show that the new developed model has achieved a significant improvement in EAC forecasting. [Copyright &y& Elsevier]

Abstract: An improved novel non-linear time series prediction method is presented based on optimizing the combination of non-linear signal analysis and deterministic chaos techniques with Artificial Neural Networks of the Multilayer Perceptron (MLP) type. The proposed methodology has been applied to the non-linear time series produced by a diode resonator chaotic circuit. Multisim is used to simulate the circuit and show the presence of chaos. The first stage of the proposed approach employs a non-linear time series analysis module applying the method proposed by Grasberger and Procaccia, involving estimation of the correlation and minimum embedding dimension as well as of the corresponding largest Lyapunov exponent in combination with a nearest neighbour-based non-linear signal predictor. The two previously mentioned modules are used to construct the first stage of a one-step/multistep predictor while a back-propagation MLP is involved in the second stage to enhance prediction results. The novelty of the proposed two-stage predictor lies on that the back-propagation MLP is employed as an error predictor of the nearest neighbour-based first-stage non-linear signal forecasting application following an efficient strategy for optimizing the combination of nearest neighbour prediction based on deterministic chaos techniques and MLP neural networks. This novel two-stage predictor is evaluated through an extensive experimental study and is favourably compared with rival approaches. [Copyright &y& Elsevier]

This paper proposes a Recurrent Radial Basis Function network (RRBFN) that can be applied to dynamic monitoring and prognosis. Based on the architecture of the conventional Radial Basis Function networks, the RRBFN have input looped neurons with sigmoid activation functions. These looped-neurons represent the dynamic memory of the RRBF, and the Gaussian neurons represent the static one. The dynamic memory enables the networks to learn temporal patterns without an input buffer to hold the recent elements of an input sequence. To test the dynamic memory of the network, we have applied the RRBFN in two time series prediction benchmarks (MacKey-Glass and Logistic Map). The third application concerns an industrial prognosis problem. The nonlinear system identification using the Box and Jenkins gas furnace data was used. A two-steps training algorithm is used: the RCE training algorithm for the prototype''s parameters, and the multivariate linear regression for the output connection weights. The network is able to predict the two temporal series and gives good results for the nonlinear system identification. The advantage of the proposed RRBF network is to combine the learning flexibility of the RBF network with the dynamic performances of the local recurrence given by the looped-neurons. [Copyright &y& Elsevier]

The aim is to provide a framework for robust time series prediction in the presence of anomalies. The framework is developed based on a data set from motorsport but is not limited to this specific area. In motorsport, the usage of sensors during races is generally restricted. Estimating the outputs of these missing sensors therefore provides an advantage over the competition. Deep learning approaches such as long short-term memory (LSTM) neural networks have proven to be useful for that task, however, their accuracy decreases significantly if anomalies occur in the input signals. To overcome this problem, we propose the variational autoencoder based selective prediction (VASP) framework which combines the tasks of anomaly detection and time series prediction. VASP consists of a variational autoencoder (VAE), an anomaly detector and LSTM predictors. Depending on the anomaly detector, a subset of the inputs may be replaced by the VAE, allowing a more robust prediction. To the best of our knowledge the approach of using a VAE to only selectively replace anomalous input data before prediction has not yet been published. Our contributions are clear implementation guidelines and a comparison to other VAE-based methods and a LSTM approach as baseline. We simulate anomalies with three approaches and show that VASP outperforms other methods by having no trade-off between accuracy and robustness. VASP is as accurate as the baseline for regular data, but for anomalous inputs the error is reduced by 13% to 33% on average and up to 70% in special cases. [ABSTRACT FROM AUTHOR]

This paper proposes a multi-step prediction model for time series prediction, i.e. Meta-cognitive Recurrent Kernel Online Sequential Extreme Learning Machine with Drift Detector Mechanism (Meta-RKOS-ELM ALD). Recurrent multi-step algorithm is applied to release the limitation in the number of prediction steps, and Drift Detector Mechanism (DDM) is used to overcome the problem of concept drift in the prediction model. The new meta-cognitive strategy decides the way of the incoming data during training, which decreases the training computation of prediction model and solves the parameter dependency. In our evaluation, we use a total of six artificial data sets and three real-world data sets (Standard & Poor's 500 Index, Shanghai Stock Exchange Composite Index, and Ozone Concentration in Toronto) to prove the ability of kernel filters, the detecting ability of concept drift detector, and situation of applying meta-cognitive strategy in our proposed model. Experiments results indicate that the Meta-KOS-ELM ALD with DDM has better forecasting ability in various predicting periods with the shortest learning time, as compared with other algorithms. [ABSTRACT FROM AUTHOR]

Market prediction has been an important research problem for decades. Having better predictive models that are both more accurate and faster has been attractive for both researchers and traders. Among many approaches, semi-supervised graph-based prediction has been used as a solution in recent researches. Based on this approach, we present two prediction models. In the first model, a new network structure is introduced that can capture more information about markets' direction of movements compared to the previous state of the art methods. Based on this novel network, a new algorithm for semi-supervised label propagation is designed that is able to prediction the direction of movement faster and more accurately. The second model is a mixture of experts system that decides between supervised or semi-supervised approaches. Besides this, the model gives us the ability to identify the markets that their data are helpful in constructing the network. Our models are shown to be both faster regarding computational complexity and running time and more accurate in prediction comparing to best rival models in literature of graph-based semi-supervised prediction. The results are also tested to be statistically significant. • A new semi-supervised graph-based classification method is presented. • A new way of interaction graph construction in time series datasets is proposed. • A novel graph edge-weighting method in semi-supervised learning is presented. • A new mixture of experts system for model selection and graph-design is presented. [ABSTRACT FROM AUTHOR]