Search

The Vector AutoRegressive Moving Average (VARMA) model is fundamental to the theory of multivariate time series; however, identifiability issues have led practitioners to abandon it in favor of the simpler but more restrictive Vector AutoRegressive (VAR) model. We narrow this gap with a new optimization-based approach to VARMA identification built upon the principle of parsimony. Among all equivalent data-generating models, we use convex optimization to seek the parameterization that is "simplest" in a certain sense. A user-specified strongly convex penalty is used to measure model simplicity, and that same penalty is then used to define an estimator that can be efficiently computed. We establish consistency of our estimators in a double-asymptotic regime. Our non-asymptotic error bound analysis accommodates both model specification and parameter estimation steps, a feature that is crucial for studying large-scale VARMA algorithms. Our analysis also provides new results on penalized estimation of infinite-order VAR, and elastic net regression under a singular covariance structure of regressors, which may be of independent interest. We illustrate the advantage of our method over VAR alternatives on three real data examples.

Video stabilization is the process of improving the video quality by removing annoying fluctuant motion caused by camera jittering. A key issue of a successful solution is the temporal adaptability to motion and the overall robustness with respect to different motion types. However, most previous methods usually produce non-motion adaptive stabilized videos. In other words, under-smoothing in slow motion segments and over-smoothing in rapid motion segments will be produced for complex shaky videos. To overcome these drawbacks, we propose a novel video stabilization approach using a motion morphological component (MMC decomposition. Specifically, the observed motion is decomposed into three MMCs: low-frequency smoothed (LFS motion, high-frequency compensatory (HFC motion, and shaky motion. LFS motion helps to largely stabilize videos, and HFC motion helps to recover missing motion to deal with over-smoothing. Subsequently, we present an MMC-based model to retrieve the desired smoothed motion, in which weighted nuclear norm and autoregression priors are used for LFS motion, while a sparsity prior is adopted for HFC motion. In addition, we design an adaptive weight setting scheme to detect rapid motions and to calculate the optimal weights. Finally, we develop a stabilization algorithm under the Alternating Direction Method of Multipliers (ADMM framework. Experimental results demonstrate that our method can achieve high-quality results compared with that of other state-of-the-art stabilization methods in terms of robustness and efficiency, both quantitatively and qualitatively.

We consider a new class of spatio-temporal models with sparse autoregressive coefficient matrices and exogenous variable. To estimate the model, we first profile the exogenous variable out of the response. This leads to a profiled model structure. Next, to overcome endogeneity issue, we propose a class of generalized methods of moment (GMM) estimators to estimate the autoregressive coefficient matrices. A novel bagging-based estimator is further developed to conquer the over-determined issue which also occurs in Chang et al. (2015) and Dou et al. (2016). An adaptive forward–backward greedy algorithm is proposed to learn the sparse structure of the autoregressive coefficient matrices. A new BIC-type selection criteria is further developed to conduct variable selection for GMM estimators. Asymptotic properties are further studied. The proposed methodology is illustrated with extensive simulation studies. A social network dataset is analyzed for illustration purpose.

Monitoring economic conditions in real time, or nowcasting, and Big Data analytics share some challenges, sometimes called the three “Vs”. Indeed, nowcasting is characterized by the use of a large number of time series (Volume), the complexity of the data covering various sectors of the economy, with different frequencies and precision and asynchronous release dates (Variety), and the need to incorporate new information continuously and in a timely manner (Velocity). In this paper, we explore three alternative routes to nowcasting with Bayesian Vector Autoregressive (BVAR) models and find that they can effectively handle the three Vs by producing, in real time, accurate probabilistic predictions of US economic activity and a meaningful narrative by means of scenario analysis.

This work introduces a methodology to estimate coffee prices based on the use of Extreme Learning Machines. The process is initiated by identifying the presence of nonstationary components, like seasonality and trend. These components are withdrawn if they are found. Next, the temporal lags are selected based on the response of the Partial Autocorrelation Function filter. As predictors, we address the following models: Exponential Smoothing (ES), Autoregressive (AR) and Autoregressive Integrated and Moving Average (ARIMA) models, Multilayer Perceptron (MLP) and Extreme Learning Machines (ELMs) neural networks. The computational results based on three error metrics and two coffee types (Arabica and Robusta) showed that the neural networks, especially the ELM, can reach higher performance levels than the other models. The methodology, which presents preprocessing stages, lag selection, and use of ELM, is a novelty that contributes to the coffee prices forecasting field.

One of the major challenges in developing powered lower limb prostheses is emulating the behavior of an intact lower limb with different walking speeds over diverse terrains. Numerous studies have been conducted on control algorithms in the field of rehabilitation robotics to achieve this overarching goal. Recent studies on powered prostheses have frequently used a hierarchical control scheme consisting of three control levels. Most control structures have at least one element of discrete transition properties that requires numerous sensors to improve classification accuracy, consequently increasing computational load and costs. In this study, we proposed a user-independent and free-mode method for eliminating the need to switch among different controllers. We constructed a database by using four OPAL wearable devices (Mobility Lab, APDM Inc., USA) for seven able-bodied subjects. We recorded the gait of each subject at three ambulation speeds during ground-level walking to train a nonlinear autoregressive network with an exogenous input recurrent neural network (NARX RNN) to estimate foot orientation (angular position) in the sagittal plane using shank angular velocity as external input. The trained NARX RNN estimated the foot orientation of all the subjects at different walking speeds over flat terrain with an average root-mean-square error (RMSE) of 2.1° ± 1.7°. The minimum correlation between the estimated and measured values was 86%. Moreover, a t-test showed that the error was normally distributed with a high certainty level (0.88 minimum p -value).

This paper proposes a Bayesian multifactor spatio-temporal model with a dynamic spatial autoregressive coefficient to estimate time-varying network interdependence with time-series cross-sectional data. To correct biases caused by unobserved time-varying homophily and common shocks with heterogeneous impact, we adopt the multifactor method to control for latent confounders and tackle the classic ``reflection problem." We use Bayesian shrinkage techniques to select latent factors. Monte Carlo studies and two empirical examples demonstrate that the multifactor specification reduces biases and improves the identification of network interdependence, and that falsely assuming a time-invariant network effect not only misses the variation of interdependence but also mis-estimates its central tendency. The proposed model is robust to misspecifcation and can accommodate various types of social networks.

This paper considers a problem of estimating brain effective connectivity from EEG signals using a Granger causality (GC) concept characterized on state-space models. We propose a state-space model for explaining coupled dynamics of the source and EEG signals where EEG is a linear combination of sources according to the characteristics of volume conduction. Our formulation has a sparsity prior on the source output matrix that can further classify active and inactive sources. The scheme is comprised of two main steps: model estimation and model inference to estimate brain connectivity. The model estimation consists of performing a subspace identification and the active source selection based on a group-norm regularized least-squares. The model inference relies on the concept of state-space GC that requires solving a discrete-time Riccati equation for the covariance of estimation error. We verify the performance on simulated data sets that represent realistic human brain activities under several conditions including percentages of active sources, a number of EEG electrodes and the location of active sources. The performance of estimating brain networks is compared with a two-stage approach using source reconstruction algorithms and VAR-based Granger analysis. Our method achieved better performances than the two-stage approach under the assumptions that the true source dynamics are sparse and generated from state-space models. The method is applied to a real EEG SSVEP data set and we found that the temporal lobe played a role of a mediator of connections between temporal and occipital areas, which agreed with findings in previous studies.

In this paper, we investigate brain directed connectivity (BDC) networks for emotion recognition using electroencephalogram (EEG) source signals that were estimated from high-density sensor EEG signals, for the first time. Currently, a variety of features extracted from sensor EEG signals are used for emotion recognition. However, they cannot unambiguously describe the location of emotions associated with neural activities and information propagation or the interaction between brain regions. In addition, most current studies use low-density sensor EEG signals. Moreover, source signals estimated from high-density sensor EEG signal have not been employed for emotion recognition to date. We designed a BDC network-based framework using EEG source signals to investigate emotion recognition. The global cortex factor-based multivariate autoregressive (GCF-MVAR) method was utilized to extract emotion-related BDC features. Our study revealed that the combined BDC and DE features facilitated a recognition accuracy of up to 89.58 %, which is higher than the rate obtained from BDC features and DE features alone. The sensor features derived from high-density EEG signals also exhibited higher recognition accuracy compared to low-density EEG signals. These findings suggest that BDC features derived from EEG source signals can better characterize human emotional states and are meaningful for emotion recognition.

Long horizon state of health (SOH) monitoring and remaining useful life (RUL) prediction are of industrial value in prognostic and health management (PHM) of lithium-ion batteries (LIBs) to ensure their reliable functionality by early detection. Machine Learning, as a data-driven health diagnostic technique, has been widely utilized in solitary and hybrid structures. However, an accurate SOH estimation and RUL prediction method with less computational burden are highly desirable for the online state prediction in an electric vehicle application. This paper evaluates nonlinear autoregressive with external input (NARX) recurrent neural network (RNN) and time delay neural network (TDNN) in their prediction precision using the NASA dataset. The superior method, NARXRNN, is employed for two different datasets to estimate the battery's SOH and predict its RUL on a broad horizon. The results reveal the outstanding performance by presenting the root mean square error within 3% and mean absolute error within 2% for unseen data. Therefore, this method is capable to accurately predict the SOH of LIBS from historical data at low computational complexity. It is a promising model for long horizon SOH and RUL prediction and practical for online applications.

This article develops a robust global strategy for identifying the linear parameter varying (LPV) errors-in-variables (EIVs) systems subjected to randomly missing observations and outliers. The parameter interpolated LPV autoregressive exogenous model with an uncertain/noisy input is investigated and a nonlinear state-space model is considered for the input generation model (IGM). The parameters estimation of the LPV EIV systems with nonideal observations is realized using the expectation-maximization algorithm which is particular effective for the incomplete data issue. To ensure the robustness in the identification, the Student's t-distribution which is characterized by its adjustable degree of freedom, is used to handle the measurement non-normality. Since the posterior distributions of the latent states in the IGM are also involved in the identification process and they are difficult to calculate directly, the particle filter is introduced to recursively approximate them instead. Finally, the verification examples are given to demonstrate the effectiveness of the developed strategy.

Gas field production forecast is an important basis for decision-making in the gas industry, how to accurately predict the dynamic production during gas field development is an important content of reservoir engineering research. Reservoir numerical simulation is the most common method for predicting oil and gas production. However, it requires a lot of data to build an accurate geological model and the models are tedious and time-consuming. At present, many scholars have used machine learning and data mining methods to predict oil and gas production, but they have not considered whether the use of increasing production measures will affect the predicted results. Thus, ARIMA-RTS optimal smooth algorithm is the first applied to establish prediction model of gas well production. According to the historical production data, the model is processed, the production differential autoregressive integral moving average (ARIMA) model in time series is established, then ARIMA model is combined with RTS (Rauch Tung Striebel) smoothing, the production prediction model is constructed. RTS smoothing algorithm is an enhanced version of Kalman filter, the measurements are the first processed by the forward filter, and then, a separate backward smoothing pass is used for obtaining the smoothing solution. The correctness of ARIMA-RTS model was verified with the actual production data. The results show that prediction based on ARIMA-RTS model can accurately reflect the production performance of gas well. This method can effectively reduce the error caused by stimulation when prediction. Use the ARIMA-RTS model and the ARIMA-Kalman model to predict production of the same gas well, and the prediction accuracy of ARIMA-RTS model is higher than that of ARIMA-Kalman model in production wells with stimulation, compared with the ARIMA-Kalman model, the mean relative error fitted by the ARIMA-RTS model is reduced by 46.3%, and the relative mean square error is reduced by 56.48%. ARIMA-RTS optimal smooth algorithm improves the prediction accuracy of gas well that use stimulation, we therefore conclude that the ARIMA-RTS optimal smooth algorithm can service us better forecasting gas well production used stimulation, as well as other fuels output.

The brain must extract behaviorally relevant latent variables from the signals streamed by the sensory organs. Such latent variables are often encoded in the dynamics that generated the signal rather than in the specific realization of the waveform. Therefore, one problem faced by the brain is to segment time series based on underlying dynamics. We present two algorithms for performing this segmentation task that are biologically plausible, which we define as acting in a streaming setting and all learning rules being local. One algorithm is model-based and can be derived from an optimization problem involving a mixture of autoregressive processes. This algorithm relies on feedback in the form of a prediction error, and can also be used for forecasting future samples. In some brain regions, such as the retina, the feedback connections necessary to use the prediction error for learning are absent. For this case, we propose a second, model-free algorithm that uses a running estimate of the autocorrelation structure of the signal to perform the segmentation. We show that both algorithms do well when tasked with segmenting signals drawn from autoregressive models with piecewise-constant parameters. In particular, the segmentation accuracy is similar to that obtained from oracle-like methods in which the ground-truth parameters of the autoregressive models are known. We also test our methods on datasets generated by alternating snippets of voice recordings. We provide implementations of our algorithms at https://github.com/ttesileanu/bio-time-series., Comment: v2.1; 34 pages, 14 figures

This paper aims to explore a modified method of high-dimensional dynamic variance–covariance matrix estimation via risk factor mapping, which can yield a dependence estimation of asset returns within a large portfolio with high computational efficiency. The essence of our methodology is to express the time-varying dependence of high-dimensional return variables using the co-movement concept of returns with respect to risk factors. A novelty of the proposed methodology is to allow mapping matrices, which govern the co-movement of returns, to be time-varying. We also consider the flexible modeling of risk factors by a copula multivariate generalized autoregressive conditional heteroscedasticity (MGARCH) model. Through the proposed risk factor mapping model, the number of parameters and the time complexity are functions of a small number of risk factors instead of the number of stocks in the portfolio, making our proposed methodology highly scalable. We adopt Bayesian methods to estimate unknown parameters and various risk measures in the proposed model. The proposed risk mapping method and financial applications are demonstrated by an empirical study of the Hong Kong stock market. The assessment of the effectiveness of the mapping via risk measure estimation is also discussed.

Introduction Multivariate time series prediction of infectious diseases is significant to public health, and the deep learning method has attracted increasing attention in this research field. Material and methods An adaptively temporal graph convolution (ATGCN) model, which learns the contact patterns of multiple age groups in a graph-based approach, was proposed for COVID- 19 and influenza prediction. We compared ATGCN with autoregressive models, deep sequence learning models, and experience- based ATGCN models in short-term and long-term prediction tasks. Results Results showed that the ATGCN model performed better than the autoregressive models and the deep sequence learning models on two datasets in both short-term (12.5% and 10% improvements on RMSE) and long-term (12.4% and 5% improvements on RMSE) prediction tasks. And the RMSE of ATGCN predictions fluctuated least in different age groups of COVID- 19 (0.029 ± 0.003) and influenza (0.059±0.008). Compared with the Ones-ATGCN model or the Pre-ATGCN model, the ATGCN model was more robust in performance, with RMSE of 0.0293 and 0.06 on two datasets when horizon is one. Discussion Our research indicates a broad application prospect of deep learning in the field of infectious disease prediction. Transmission characteristics and domain knowledge of infectious diseases should be further applied to the design of deep learning models and feature selection. Conclusions The ATGCN model addressed the multivariate time series forecasting in a graph-based deep learning approach and achieved robust prediction on the confirmed cases of multiple age groups, indicating its great potentials for exploring the implicit interactions of multivariate variables.

Stationarity is fundamental for time-series modeling and prediction. In this article, we focus on the radial basis function network-based autoregressive (RBF-AR) models which have been widely used in practical applications. Compared to previous work, we give a less-restrictive sufficient condition for the asymptotic stationarity of the RBF-AR model. The parameter estimation of the RBF-AR model is converted to the optimization of a variable projection functional with constraints of stationarity to always derive a stationary model. The constrained evolutionary algorithm is used to solve the optimization problem. Numerical results demonstrate the effectiveness of the proposed method.

The paper presents a prediction of non-linear exogenous signal by optimized intelligent auto-regressive neural network model (ARNN). A signal comprises of two sets of data called deterministic and error. The former type of data represents the degradation index of a signal, while the error is the uncertainties associated with the signal. To understand and predict signals, a intelligent approach is taken through the use of ARNN model. In this approach, the rst step is to diagnose whether a time series signal is normally distributed or not by utilizing the Jarque-Bera test. The high and low volatility data ele- ments can be separated via kurtosis hypothesis. The deterministic component of the signal is also predicted by developing a neural network based non-linear autoregressive model (NN-NARX) and the error component by using a linear model. The nal forecast is formed by combining the results determined from each of the models and evaluated using the mean square error results. Vali- dation of the prediction is obtained through a comparison of the results with other models such as ARNN, traditional ARMX, and NARX models. The re- sults show that the proposed model provides improved predictions, minimize high dependence on design parameters with low computational cost.

This article proposes a speed sensor fault diagnosis methodology based on a learning-based data-driven principle in induction motor drive systems. The proposed method is derived from signal estimation and residual evaluation. First, a speed estimator is designed with a nonlinear autoregressive exogenous (NARX) learning model and a randomized learning technique called random vector functional link (RVFL) network. A data preprocessing method by discrete wavelet transform (DWT) is applied to better trace the signal trends, in order to further improve the speed estimation accuracy. After the estimation, the residual between the measured and estimated signals can be obtained, and a decision-making mechanism is developed for fault diagnosis based on an outlier test. The offline test results show that the proposed method can accurately estimate the speed signal with a 1.554e -4 root-mean-square error (RMSE) and outperforms the state-of-the-art methods. Moreover, real-time tests are also carried out to verify the feasibility and stability during the online stage. Moreover, the proposed approach does not require any motor parameters and other additional hardware, which makes it quite suitable for online practical applications. This work was supported by the National Natural Science Foundation of China (NSFC) under Grant 51907163 and Grant U1934204.

Partially observable Markov decision processes have been widely adopted in the automatic planning literature since it elegantly captures both execution and observation uncertainties. In our previous paper, we proposed a model called vector autoregressive partially observable Markov decision process (VAR-POMDP) which extends the traditional POMDP by considering the temporal correlation among continuous observations. However, it is a non-trivial problem to develop a tractable planning algorithm for the VAR-POMDP model with performance guarantees as most existing algorithms need to explicitly enumerate all possible observation histories, which is in an unbounded continuous space. In this letter, we extend the famous point-based value iteration algorithm to a double point-based value iteration and show that the VAR-POMDP model can be solved by dynamic programming through approximating the exact value function by a class of piece-wise linear functions. Meanwhile, we prove that the approximation error is bounded. The effectiveness of the proposed planning algorithm is illustrated by an example.

Tensor-valued data are becoming increasingly available in economics and this calls for suitable econometric tools. We propose a new dynamic linear model for tensor-valued response variables and covariates that encompasses some well-known econometric models as special cases. Our contribution is manifold. First, we define a tensor autoregressive process (ART), study its properties and derive the associated impulse response function. Second, we exploit the PARAFAC low-rank decomposition for providing a parsimonious parametrization and to incorporate sparsity effects. We also contribute to inference methods for tensors by developing a Bayesian framework which allows for including extra-sample information and for introducing shrinking effects. We apply the ART model to time-varying multilayer networks of international trade and capital stock and study the propagation of shocks across countries, over time and between layers.

Recently, many efforts have been devoted to speeding up neural machine translation models. Among them, the non-autoregressive translation (NAT) model is promising because it removes the sequential dependence on the previously generated tokens and parallelizes the generation process of the entire sequence. On the other hand, the autoregressive translation (AT) model in general achieves a higher translation accuracy than the NAT counterpart. Therefore, a natural idea is to fuse the AT and NAT models to seek a trade-off between inference speed and translation quality. This paper proposes an ARF-NAT model (NAT with auxiliary representation fusion) to introduce the merit of a shallow AT model to an NAT model. Three functions are designed to fuse the auxiliary representation into the decoder of the NAT model. Experimental results show that ARF-NAT outperforms the NAT baseline by 5.26 BLEU scores on the WMT’14 German-English task with a significant speedup (7.58 times) over several strong AT baselines.

Forecasting is one of the fundamental inputs to support planning decisions in retail chains. Frequently, forecasting systems in retail are based on Gaussian models, which may be highly unrealistic when considering daily retail data. In addition, the majority of these systems rely on point forecasts, limiting their practical use in retailing decisions, which often requires the full predictive density for decision making. The main contribution of this paper is the modeling of daily distribution centers (DCs) level aggregate demand forecasting using a recently proposed framework for non-Gaussian time series called score-driven models or Generalized Autoregressive Score (GAS) models. An experimental study was carried out using real data from a large retail chain in Brazil. A log-normal GAS model is compared to usual benchmarks, namely neural networks, linear regression, and exponential smoothing. The results show the GAS model is a competitive alternative to retail demand forecasting in daily frequency, with the advantage of producing a closed form predictive density by construction.

In this paper we present a thorough description of the digital twin development for a continuous pharmaceutical powder blending process in accordance with the Process Analytical Technologies (PAT) and Quality by Design (QbD) guidelines. A low-dosage system of caffeine active pharmaceutical ingredient (API) and dextrose excipient was examined via continuous blending experiments. Near infrared (NIR) spectroscopy-based process analytics were applied; quantitative evaluation of spectra was achieved using multivariate data analysis. The blending system was represented with mechanistic residence time distribution (RTD) models and two types of recurrent artificial neural networks (ANN), experimental datasets were used for model training or fitting and validation. Detailed comparison of the two modelling approaches, the optimization of the model-based digital twin, and efficiency of the soft sensor-based process monitoring is presented through several validating simulations. Both RTD models and nonlinear autoregressive neural networks demonstrated excellent predictive power for the low dosage blending process. RTD models can prove to be more advantageous in industrial development as they are less resource-intensive to develop and prediction accuracy on low concentration levels lacks dependency from the precision of chemometric calibration. Reduced material costs and limited development timeframe render the digital twin an efficient tool in technological development.

Traffic data collected from sensor networks often exhibit strong spatial correlations and recurrent temporal patterns. Learning these patterns and diagnosing anomalies in such spatiotemporal traffic data is critical to improving transportation systems and services. This paper proposes a dynamic framework to model spatiotemporal traffic data, with a particular application on diagnosing anomalies. Within the framework, we focus on characterizing the variation in system dynamics with a time-varying vector autoregressive model. We impose a low-rank tensor structure to model the collection of time-varying system matrices. As the temporal factor matrix captures the principal patterns/signatures across all time-varying system matrices, it is a useful tool to diagnose abnormal generative mechanisms and unexpected temporal patterns. We demonstrate the proposed tensor learning framework's effectiveness by experimenting with a synthetic data set and real-world spatiotemporal traffic speed data set. The results show the superiority of the proposed model in uncovering anomalous traffic network dynamics.

With Industry 4.0, temporal relations between process and quality variables become increasingly complex. Some dynamic supervised learning algorithms are designed to extract their dynamic cross-relations, but auto-correlations among quality variables are rarely considered, which, however, can provide additional valuable information. This article proposes a novel dynamic auto-regressive latent variable model (DALVM) to capture both auto and cross correlations from high-dimensional time series data. DALVM is designed to maximize the covariance between current quality score and the weighted sum of past quality and process scores, and an auto-regressive exogenous inner model is developed for consistency purpose. Further, a concurrent anomaly detection system is developed based on DALVM, referred to as ConDALVM, which conducts subsequent decompositions in the extracted latent spaces. ConDALVM realizes a comprehensive monitoring for both static and dynamic anomalies in process and quality spaces. The superiority of the methods is demonstrated through a numerical simulation and two industrial processes.

Wind speed predictions are essential for wind power management and wind farm operation. However, due to the high volatility and nonstationarity of measured wind data, it is often difficult to achieve an accurate prediction. This study proposes a hybrid approach that consists of two stages, i.e., data preprocessing and wind speed predicting, to improve the accuracy of short-term wind speed prediction. A preprocessing algorithm for the transformation and standardization of hourly mean wind speed is utilized to remove the non-Gaussian distribution of wind data and diurnal nonstationarity. Several statistical models and artificial intelligence models are then adopted in the second stage of the prediction process, including a persistence model, autoregressive model, autoregressive moving average model and backpropagation neural network. The proposed approach is developed based on the weighted averaging of these models and error optimization theory. Finally, wind speed data for 12 months from two meteorological towers located in Yanan, China, are investigated to demonstrate the effectiveness and accuracy of the proposed approach for multistep wind speed predictions, and its performance is then compared with several existing prediction models. The results indicate that the prediction accuracy improves significantly after preprocessing with the proposed approach, outperforming all the existing aforementioned models.

Non-autoregressive machine translation aims to speed up the decoding procedure by discarding the autoregressive model and generating the target words independently. Because non-autoregressive machine translation fails to exploit target-side information, the ability to accurately model source representations is critical. In this paper, we propose an approach to enhance the encoder’s modeling ability by using a pre-trained BERT model as an extra encoder. With a different tokenization method, the BERT encoder and the Raw encoder can model the source input from different aspects. Furthermore, having a gate mechanism, the decoder can dynamically determine which representations contribute to the decoding process. Experimental results on three translation tasks show that our method can significantly improve the performance of non-autoregressive MT, and surpass the baseline non-autoregressive models. On the WMT14 EN $$\rightarrow$$ DE translation task, our method achieves 27.87 BLEU with a single decoding step. This is a comparable result with the baseline autoregressive Transformer model which obtains a score of 27.8 BLEU.

Dissolved oxygen content prediction plays an important role in the intelligence management of aquaculture systems. Compared with traditional point prediction, interval prediction can quantify the uncertainties effectively and more closes to the fact. Unfortunately, there is rarely study about it. In this paper, a novel interval prediction method based on a deep auto-regression recurrent neural network (DeepAR) is proposed to construct prediction intervals (PIs) directly. Besides, a variational mode decomposition (VMD) has been used to extract the frequency feature of the original data. Moreover, a multi-objective weighted optimization framework based on the sparrow swarm algorithm (SSA) was proposed to improve PIs accuracy. Finally, simulations with water quality datasets were conducted to show the effectiveness of the proposed model. The results demonstrated that the proposed model significantly improved PI quality and performance compared to the state-of-the-art method.

Many previous studies have developed decomposition and ensemble models to improve runoff forecasting performance. However, these decomposition-based models usually introduce large decomposition errors into the modeling process. Since the variation in runoff time series is greatly driven by climate change, many previous studies considering climate change focused on only rainfall-runoff modeling, with few meteorological factors as input. Therefore, a climate-driven streamflow forecasting (CDSF) framework was proposed to improve the runoff forecasting accuracy. This framework is realized by using principal component analysis (PCA), long short-term memory (LSTM) and Bayesian optimization (BO), referred to as PCA-LSTM-BO. To validate the effectiveness and superiority of the PCA-LSTM-BO method along with one autoregressive LSTM model and two other CDSF models based on PCA, BO, and either support vector regression (SVR) or gradient boosting regression trees (GBRT), namely, PCA-SVR-BO and PCA-GBRT-BO, respectively, were compared. A generalization performance index based on the Nash-Sutcliffe efficiency (NSE), called the GI(NSE) value, is proposed to evaluate the generalizability of the model. The results show that (1) the proposed model is significantly better than the other benchmark models in terms of the mean square error (MSE =0.819, and GI(NSE)

Improving time series forecasting accuracy is an important work for decision makers. A single model applied on a data series may not obtain satisfactory prediction accuracy. Both theoretical and empirical findings have indicated that integration of linear model and nonlinear model may provide more accurate prediction than an individual linear or nonlinear model. This paper presents a hybrid modeling approach that combines a linear autoregressive (AR) model and a nonlinear deep belief network-based autoregressive (DBN-AR) model for time series forecasting. The proposed modeling approach first applies an AR model to fit time series data, and the error between the original date and the AR model forecast data as a nonlinear component is considered, and then the error is modeled by a DBN-AR model. DBaN-AR model is a modeling method for nonlinear time series, which uses a set of deep belief networks to approximate the state-dependent functional coefficients of state dependent auto-regressive model. The proposed hybrid model can overcome limitation of an individual model and obtain more general and more accurate forecasting result than some existing hybrid models. The experiment results demonstrate that the MSE of modeling residuals using the proposed hybrid model is largely reduced compared with the results of some single prediction models and some hybrid models for one-step-ahead and multistep-ahead forecast.

This study validated a more exact automated method of determining cardiovascular resonance frequency (RF) against the “stepped” protocol described by Lehrer et al. (Appl Psychophysiol Biofeedback 25(3):177–191, https://doi.org/10.1023/a:1009554825745 , 2000; in Foundations of heart rate variability biofeedback: A book of readings, The Association for Applied Psychophysiology and Biofeedback, pp 9–19, 2016). Thirteen participants completed a 15-min RF determination session by each method. The “stepped” protocol assesses HRV in five 3-min stationary windows from 4.5 to 6.5 breaths per minute (bpm), decreasing in 0.5 bpm steps. Multiple criteria, subjectively weighted by the clinician, determines RF. For this study, the proposed method used a sliding window with a fixed rate of change (67.04 ms per breath) at each of 78 breath cycles ranging from 4.25 to 6.75 bpm. Its algorithm analyzes IBI to locate the midpoint of the 1-min region of stable maximum peak-trough variability. RF is quantified from breath duration at that point. The software generates a visual display of superimposed HR and breathing data. Thus, the new method fully automates RF determination. Eleven of the 13 matched pairs fell within the 0.5 bpm resolution of the stepped method. Comparisons of LF power generated by the autoregressive (AR) spectral method showed a strong correlation in LF power production by the stepped and sliding methods (R = 0.751, p = 0.000). The “sliding” pacing protocol was favored by 69% of participants (p

This editorial summarizes the papers published in issue 16(4) (2021). The first paper adopts a higher order spatial autoregressive model with endogenous spatial weight matrices. The second paper in...

The autoregressive model is a tool used in time series analysis to describe and model time series data. Its main structure is a linear equation using the previous values to compute the next time step, i.e., the short time relationship is the core component of the autoregressive model. Therefore, short-term effects can be modeled in an easy way, but the global structure of the model is not obvious. However, this global structure is a crucial aid in the model selection process in data analysis. If the global properties are not reflected in the data, a corresponding model is not compatible. This helpful knowledge avoids unsuccessful modeling attempts. This article analyzes the global structure of the autoregressive model through the derivation of a closed form. In detail, the closed form of an autoregressive model consists of the basis functions of a fundamental system of an ordinary differential equation with constant coefficients, i.e., it consists of a combination of polynomial factors with sinusoidal, cosinusoidal, and exponential functions. This new insight supports the model selection process.

A new method for clustering nonlinear time series data is proposed. It is based on the forecast distributions, which are estimated by using a feed-forward neural network and the pair bootstrap. The procedure is shown to deliver consistent results for pure autoregressive dependent structures. It is model-free within a general class of nonlinear autoregression processes, and it avoids the specification of a finite dimensional model for the data generating process. The results of a Monte Carlo study are reported in order to investigate the finite sample performances of the proposed time series clustering approach. An application to a real dataset of economic time series is also discussed.

Recently, surface electromyography (sEMG) emerged as a novel biometric authentication method. Since EMG system parameters, such as the feature extraction methods and the number of channels, have been known to affect system performances, it is important to investigate these effects on the performance of the sEMG-based biometric system to determine optimal system parameters. In this study, three robust feature extraction methods, Time-domain (TD) feature, Frequency Division Technique (FDT), and Autoregressive (AR) feature, and their combinations were investigated while the number of channels varying from one to eight. For these system parameters, the performance of sixteen static wrist and hand gestures was systematically investigated in two authentication modes: verification and identification. The results from 24 participants showed that the TD features significantly ( ${p} ) and consistently outperformed FDT and AR features for all channel numbers. The results also showed that the performance of a four-channel setup was not significantly different from those with higher number of channels. The average equal error rate (EER) for a four-channel sEMG verification system was 4% for TD features, 5.3% for FDT features, and 10% for AR features. For an identification system, the average Rank-1 error (R1E) for a four-channel configuration was 3% for TD features, 12.4% for FDT features, and 36.3% for AR features. The electrode position on the flexor carpi ulnaris (FCU) muscle had a critical contribution to the authentication performance. Thus, the combination of the TD feature set and a four-channel sEMG system with one of the electrodes positioned on the FCU are recommended for optimal authentication performance.

In this paper, a nonlinear predictive generalized minimum variance (NPGMV) controller is proposed and explicitly formulated for a class of nonlinear systems modeled by autoregressive second-order Volterra series, applying the polynomial approach. Hence, a new benchmark controller for performance assessment is introduced to improve the achievable control performance. Furthermore, to have an efficient control assessment, a data-driven algorithm based on the NPGMV control is presented that uses only the closed-loop operating data. In the design procedure, a multi-step cost function is defined to incorporate predictive action. Exploiting the predictive control concept enables the control scheme to handle constrained problems. Also, the proposed control algorithm utilizes an inherent integrating effect, which is essential for practical purposes. Volterra series are employed for modeling and identification of the nonlinear processes, using conventional least-squares methods. To show the effectiveness of the proposed methodology, simulation results and comparison studies are provided on a cascade Wiener model and a continuous stirred tank reactor (CSTR) chemical pilot plant. Finally, an experimental study on a pressure pilot plant is used to demonstrate the applicability of the proposed control scheme. The simulation and experimental results indicate satisfactory performance of the proposed controller.

This study aims to determine an automatic forecasting method of univariate time series, using the nonlinear autoregressive neural network model with exogenous input (NARX). In this automatic setting, users only need to supply the input of time series. Then, an automatic forecasting algorithm sets up the appropriate features, estimate the parameters in the model, and calculate forecasts, without the users’ intervention. The algorithm method used include preprocessing, tests for trends, and the application of first differences. The time series were tested for seasonality, and seasonal differences were obtained from a successful analysis. These series were also linearly scaled to [−1, +1]. The autoregressive lags and hidden neurons were further selected through the stepwise and optimization algorithms, respectively. The 20 NARX models were fitted with different random starting weights, and the forecasts were combined using the ensemble operator, in order to obtain the final product. This proposed method was applied to real data, and its performance was compared with several available automatic models in the literature. The forecasting accuracy was also measured by mean squared error (MSE) and mean absolute percent error (MAPE), and the results showed that the proposed method outperformed the other automatic models.

Dynamic inference problems in autoregressive (AR/ARMA/ARIMA), exponential smoothing, and navigation are often formulated and solved using state-space models (SSM), which allow a range of statistical distributions to inform innovations and errors. In many applications the main goal is to identify not only the hidden state, but also additional unknown model parameters (e.g. AR coefficients or unknown dynamics). We show how to efficiently optimize over model parameters in SSM that use smooth process and measurement losses. Our approach is to project out state variables, obtaining a value function that only depends on the parameters of interest, and derive analytical formulas for first and second derivatives that can be used by many types of optimization methods. The approach can be used with smooth robust penalties such as Hybrid and the Student's t, in addition to classic least squares. We use the approach to estimate robust AR models and long-run unemployment rates with sudden changes., 7 pages, 3 figures

This article presents the results of the development of a mathematical model for predicting the levelized cost of energy (LCOE) for solar concentrating thermal power systems (CSP systems) based on a nonlinear autoregressive neural network with exogenous inputs (NARX). A two-layer NARX network with sigmoid hidden neurons and linear output neurons has been developed. The input layer is made up of the following variables: the volume of input power of CSP systems in the world, the total world energy consumption, domestic energy consumption, domestic gas consumption, domestic consumption of coal and lignite, domestic energy consumption, the share of renewable energy in electricity generation, the share of wind and solar energy in the production of electricity, carbon dioxide emissions from fuel combustion, the price of Brent oil against the US dollar, and the average price for natural gas auctions. The output layer specifies LCOE values for CSP systems.

A motion cueing algorithm (MCA) is employed to transform the linear and angular motion signals generated from a motion simulator without violating the physical and dynamical boundaries of the motion platform. In this respect, the accurate prediction of the motion scenarios is essential to enhance the efficiency of the MCA using prepositioning or time-varying reference model predictive control. While a recent approach that utilizes a feedforward neural network (NN) to forecast the motion scenarios is useful, the feedforward NN model has only forward dynamics relating to the signals without any feedback loop. In this article, a time-delay feedforward NN, a recurrent NN, and a nonlinear autoregressive (NAR) models with three different training procedures, i.e., Levenberg–Marquardt, Bayesian regularization, and scaled conjugate gradient, are exploited to predict the motion scenarios. As the NAR model employs the historical signals as the inputs, it can predict the motion scenarios with higher accuracy rates. Based on the series of empirical evaluations, NAR trained with Levenberg–Marquardt is able to outperform the other two counterparts in producing more accurate predictions of the motion signals. The NAR method has a lower computational load as compared with that of the recurrent NN, facilitating its real-time application. In addition to the MCA, the NAR method can be employed in other areas, including autonomous vehicles and motion sickness studies. It can also be easily implemented for air, sea, and/or land vehicle simulators for training purposes in virtual reality environments.

In the field of dynamic system identification and prediction, linear models (e.g., autoregressive models), nonlinear models (namely, neural networks models), and hybrid predictors (HPs) that are a hybridization of linear and nonlinear models have been proposed in the past. However, they are not completely free from limitations: they exhibit difficulties to describe high-order nonlinear relations between input and output variables. In this study, we propose fuzzy reinforced polynomial neural networks (FRPNNs), which are polynomial neural network architecture-based on fuzzy reinforced polynomial neurons (FRPNs) to overcome this limitation. The proposed FRPNs that consist of approximation part (AP) and compensation part (CP) arise as novel HPs. Here the CP for modeling nonlinear patterns can be regarded as forming the reinforced part for the AP that aims at capturing linear patterns, while AP is the linear polynomial neuron used in the conventional polynomial neural networks. In some sense, the overall FRPNNs are essentially generalized polynomial neural network architecture with novel HPs. The parameters considered in the design of the proposed fuzzy reinforced polynomial neural networks are optimized with the aid of the particle swarm optimization (PSO). The performance of FRPNNs is discussed involving time series and system identification datasets. Experimental results demonstrate that the proposed FRPNNs achieve at most the accuracy of 43.6% higher in comparison with the accuracy produced by some classical models reported in the literature.

BACKGROUND Long-term projections of mobility are key inputs to sub-national population projections. These long-term projections are based on extrapolations of long-term trends. In cases of strong, potentially temporal, fluctuations it is informative to analyse the short-to medium -term dynamics of mobility, using data of monthly frequency. OBJECTIVE We develop two univariate models to forecast short-to medium-term mobility in the Netherlands. We apply a recent turning point in the time series of mobility to demonstrate how short-to medium-term forecasts can provide early warning signals about possible changes in the annual trend. METHODS The models we apply are Dynamic Linear Models (DLMs) which belong to the state space family of models. The two models developed in the paper incorporate trend, sea-sonal and autoregressive components but differ in the representation of the long-term trend. Posterior sampling allows for calculation of consistent prediction intervals for both monthly and annual data. CONCLUSION Forecast accuracy is evaluated using time series cross-validation. Point forecast errors and calibration of prediction intervals are compared to those of several other popular univariate forecasting models. One of our DLM models is more accurate than the models included as comparison. CONTRIBUTION The paper shows how short-to medium-term forecasts of mobility can be used to inform long-term projections based on annual data. This will be a challenging task for statistical offices generating post-COVID-19 demographic projections.

Due to the interference of complex transmission path and noise, the weak characteristic signal of a fault in complex equipment is difficult to extract, and then it is hard to establish an index that can timely and effectively reflect the degradation state of the bearing fault. A bearing degradation state index was constructed by using moving average coarsegrained, fast iterative filtering decomposition, permutation entropy, Wasserstein distance and cumulative sum method, which realize the recognition and evaluation of bearing degradation state. On this basis, a hybrid model based on autoregressive integral moving average and nonlinear autoregressive neural network was constructed. Experimental study showed that the model could achieve accurately the residual life prediction of bearing for complex equipment.

A bootstrap procedure for constructing prediction bands for a stationary functional time series is proposed. The procedure exploits a general vector autoregressive representation of the time-reversed series of Fourier coefficients appearing in the Karhunen-Loève representation of the functional process. It generates backward-in-time functional replicates that adequately mimic the dependence structure of the underlying process in a model-free way and have the same conditionally fixed curves at the end of each functional pseudo-time series. The bootstrap prediction error distribution is then calculated as the difference between the model-free, bootstrap-generated future functional observations and the functional forecasts obtained from the model used for prediction. This allows the estimated prediction error distribution to account for the innovation and estimation errors associated with prediction and the possible errors due to model misspecification. We establish the asymptotic validity of the bootstrap procedure in estimating the conditional prediction error distribution of interest, and we also show that the procedure enables the construction of prediction bands that achieve (asymptotically) the desired coverage. Prediction bands based on a consistent estimation of the conditional distribution of the studentized prediction error process also are introduced. Such bands allow for taking more appropriately into account the local uncertainty of the prediction. Through a simulation study and the analysis of two data sets, we demonstrate the capabilities and the good finite-sample performance of the proposed method.

Identification of spatial outlier is essential in revealing hidden useful knowledge in different fields of statistical applications for big data. The Score Statistic (SCi) has been used as a diagnostic tool for the identification of spatial outliers in big data. Nonetheless, the SCi method suffers from masking and swamping effects. In order to reduce the swamping effect, we propose two methods denoted as tMDR and tEW. The tMDR and tEW methods adopt location adjacency to construct spatial weights, namely metric distance reciprocal (MDR) weight and exponential weight (EW), respectively, to detect outliers in spatial autoregressiveregressive model (SAR), spatial autoregressive error model (SEM) and general spatial autoregressive-regressive model (GSM). Difference between spatial residuals are calibrated to incorporate adjacency effect into spatial outlier residual. The results of the simulation study and real example show that the performances of the three methods are equally good for SAR model. The tMDR and tEW are comparable and both outperform the SCi for SEM and GSM models with less swamping effects and less computational running times.