Search

As a multivariate time series, the prediction of curling trajectories is crucial for athletes to devise game strategies. However, the wide prediction range and complex data correlations present significant challenges to this task. This paper puts forward an innovative deep learning approach, CasLSTM, by introducing integrated inter-layer memory, and establishes an encoder-predictor curling trajectory forecasting model accordingly. Additionally, tailored training techniques involving non-teacher-forcing, ExMSE loss and incremental multi-trajectory learning are devised to enhance model performance. Notably, the model demonstrates astounding accuracy, achieving sub-1cm average errors over 30m trajectories, outperforming vanilla LSTM by 41.8%. It also showcases robustness across various curling settings, with strict validation metrics on a static test set further verifying precision. Field test results reveal promising predictive capabilities for real-world scenarios as well, exhibiting applicability. The proposed technique liberates data-driven curling stone trajectory prediction from sole reliance on analytical models and tackles key challenges of long sequence forecasting. The presented technologies and insights could also generalize to prediction tasks in other remote trajectories and multivariate time series domains., Competing Interests: Declarations. Competing interests: The authors declare no competing interests., (© 2025. The Author(s).)

Multivariate time series anomaly detection (MTSAD) can effectively identify and analyze anomalous behavior in complex systems, which is particularly important in fields such as financial monitoring, industrial equipment fault detection, and cybersecurity. MTSAD requires simultaneously analyze temporal dependencies and inter-variable relationships have prompted researchers to develop specialized deep learning models to detect anomalous patterns. In this paper, we conducted a structured and comprehensive overview of the latest techniques in deep learning for multivariate time series anomaly detection methods. Firstly, we proposed a taxonomy for the anomaly detection strategies from the perspectives of learning paradigms and deep learning models, and then provide a systematic review that emphasizes their advantages and drawbacks. We also organized the public datasets for time series anomaly detection along with their respective application domains. Finally, open issues for future research on MTSAD were identified.

Transformer-based models demonstrate tremendous potential for Multivariate Time Series (MTS) forecasting due to their ability to capture long-term temporal dependencies by using the self-attention mechanism. However, effectively modeling the spatial correlation cross series for MTS is a challenge for Transformer. Although Graph Neural Networks (GNN) are competent for modeling spatial dependencies across series, existing methods are based on the assumption of static relationships between variables, which do not align with the time-varying spatial dependencies in real-world series. Therefore, we propose DyGraphformer, which integrates graph convolution into Transformer to assist Transformer in effectively modeling spatial dependencies, while also dynamically inferring time-varying spatial dependencies by combining historical spatial information. In DyGraphformer, decoder module involving complex recursion is abandoned to accelerate model execution. First, the input is embedded using DSW (Dimension Segment Wise) through integrating its position and node level embedding to preserve temporal and spatial information. Then, the time self-attention layer and dynamic graph convolutional layer are constructed to capture temporal dependencies and spatial dependencies of multivariate time series, respectively. The dynamic graph convolutional layer utilizes Gated Recurrent Unit (GRU) to obtain historical spatial dependencies, and integrates the series features of the current time to perform graph structure inference in multiple subspaces. Specifically, to fully utilize the spatio-temporal information at different scales, DyGraphformer performs hierarchical encoder learning for the final forecasting. Extensive experimental results on seven real-world datasets demonstrate DyGraphformer outperforms state-of-the-art baseline methods, with comparisons including Transformer-based and GNN-based methods., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024 Elsevier Ltd. All rights reserved.)

In data analysis and forecasting, particularly for multivariate long-term time series, challenges persist. The Transformer model in deep learning methods has shown significant potential in time series forecasting. The Transformer model's dot-product attention mechanism, however, due to its quadratic computational complexity, impairs training and forecasting efficiency. In addition, the Transformer architecture has limitations in modeling local features and dealing with multivariate cross-dimensional dependency relationship. In this article, a Multi-Scale Convolution Enhanced Transformer model (MSCformer) is proposed for multivariate long-term time series forecasting. As an alternative to modeling the time series in its entirety, a segmentation strategy is designed to convert the input original series into segmented forms with different lengths, then process time series segments using a new constructed multi-Dependency Aggregation module. This multi-Scale segmentation approach reduces the computational complexity of the attention mechanism part in subsequent models, and for each segment of length corresponds to a specific time scale, it also ensures that each segment retains the semantic information of the data sequence level, thereby comprehensively utilizing the multi-scale information of the data while more accurately capturing the real dependency of the time series. The Multi-Dependence Aggregate module captures both cross-temporal and cross-dimensional dependencies of multivariate long-term time series and compensates for local dependencies within the segments thereby captures local series features comprehensively and addressing the issue of insufficient information utilization. MSCformer synthesizes dependency information extracted from various temporal segments at different scales and reconstructs future series using linear layers. MSCformer exhibits higher forecasting accuracy, outperforming existing methods in multiple domains including energy, transportation, weather, electricity, disease and finance., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024. Published by Elsevier Ltd.)

Vector AutoRegressive Moving Average (VARMA) models form a powerful and general model class for analyzing dynamics among multiple time series. While VARMA models encompass the Vector AutoRegressive (VAR) models, their popularity in empirical applications is dominated by the latter. Can this phenomenon be explained fully by the simplicity of VAR models? Perhaps many users of VAR models have not fully appreciated what VARMA models can provide. The goal of this review is to provide a comprehensive resource for researchers and practitioners seeking insights into the advantages and capabilities of VARMA models. We start by reviewing the identification challenges inherent to VARMA models thereby encompassing classical and modern identification schemes and we continue along the same lines regarding estimation, specification, and diagnosis of VARMA models. We then highlight the practical utility of VARMA models in terms of Granger Causality analysis, forecasting and structural analysis as well as recent advances and extensions of VARMA models to further facilitate their adoption in practice. Finally, we discuss some interesting future research directions where VARMA models can fulfill their potentials in applications as compared to their subclass of VAR models., Competing Interests: The authors declare no conflicts of interest., (© 2025 The Author(s). WIREs Computational Statistics published by Wiley Periodicals LLC.)

Background: Type 1 diabetes (T1D) is a chronic condition in which the body produces too little insulin, a hormone needed to regulate blood glucose. Various factors such as carbohydrates, exercise, and hormones impact insulin needs. Beyond carbohydrates, most factors remain underexplored. Regulating insulin is a complex control task that can go wrong and cause blood glucose levels to fall outside a range that protects people from adverse health effects. Automated insulin delivery (AID) has been shown to maintain blood glucose levels within a narrow range. Beyond clinical outcomes, data from AID systems are little researched; such systems can provide data-driven insights to improve the understanding and treatment of T1D., Objective: The aim is to discover unexpected temporal patterns in insulin needs and to analyze how frequently these occur. Unexpected patterns are situations where increased insulin does not result in lower glucose or where increased carbohydrate intake does not raise glucose levels. Such situations suggest that factors beyond carbohydrates influence insulin needs., Methods: We analyzed time series data on insulin on board (IOB), carbohydrates on board (COB), and interstitial glucose (IG) from 29 participants using the OpenAPS AID system. Pattern frequency in hours, days (grouped via k-means clustering), weekdays, and months were determined by comparing the 95% CI of the mean differences between temporal units. Associations between pattern frequency and demographic variables were examined. Significant differences in IOB, COB, and IG across temporal dichotomies were assessed using Mann-Whitney U tests. Effect sizes and Euclidean distances between variables were calculated. Finally, the forecastability of IOB, COB, and IG for the clustered days was analyzed using Granger causality., Results: On average, 13.5 participants had unexpected patterns and 9.9 had expected patterns. The patterns were more pronounced (d>0.94) when comparing hours of the day and similar days than when comparing days of the week or months (0.3

Accurate blood glucose (BG) prediction is greatly benefit for the treatment of diabetes. Generally, clinical physicians are required to comprehensively analyze various factors, such as patient's body temperature, meal, sleep, insulin injection, continuous glucose monitoring (CGM), and other information, to evaluate the fluctuation trend of blood glucose. To address this problem, this paper proposes a multivariate blood glucose prediction method based on mixed feature clustering. It clusters time series data with diverse or mixed features related to blood glucose, effectively leveraging correlations and distribution characteristics. By combining incremental clustering of multivariate time series with transfer learning, this method achieves online prediction of blood glucose levels. The experimental results indicate that the proposed method can decrease the prediction error RMSE by 4.2% (PH=30min) and 5.9% (PH=60min). Compared with other prediction methods, the training time of the multivariate prediction method is reduced by 5.2% (PH=30min) and 4.7% (PH=60min). It was also validated and compared with other methods in a real dataset. The proposed method in this study has lower prediction error and better prediction performance in the prediction horizon (PH) of PH=30, 45, 60, 75, and 90 min, respectively. Compared with the traditional unitary and multivariate time series prediction method, the approach proposed in this paper significantly improves the accuracy and robustness of blood glucose prediction. According to the evaluation results on the data set from OhioT1DM and the Sixth People's Hospital of Shanghai, the proposed method has better generalization performance and clinical acceptability., Competing Interests: Conflict of interestThe authors have no Conflict of interest to declare that are relevant to the content of this article., (© The Author(s), under exclusive licence to Springer Nature Switzerland AG 2024. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.)

A novel coronavirus discovered in late 2019 (COVID-19) quickly spread into a global epidemic and, thankfully, was brought under control by 2022. Because of the virus's unknown mutations and the vaccine's waning potency, forecasting is still essential for resurgence prevention and medical resource management. Computational efficiency and long-term accuracy are two bottlenecks for national-level forecasting. This study develops a novel multivariate time series forecasting model, the densely connected highly flexible dendritic neuron model (DFDNM) to predict daily and weekly positive COVID-19 cases. DFDNM's high flexibility mechanism improves its capacity to deal with nonlinear challenges. The dense introduction of shortcut connections alleviates the vanishing and exploding gradient problems, encourages feature reuse, and improves feature extraction. To deal with the rapidly growing parameters, an improved variation of the adaptive moment estimation (AdamW) algorithm is employed as the learning algorithm for the DFDNM because of its strong optimization ability. The experimental results and statistical analysis conducted across three Japanese prefectures confirm the efficacy and feasibility of the DFDNM while outperforming various state-of-the-art machine learning models. To the best of our knowledge, the proposed DFDNM is the first to restructure the dendritic neuron model's neural architecture, demonstrating promising use in multivariate time series prediction. Because of its optimal performance, the DFDNM may serve as an important reference for national and regional government decision-makers aiming to optimize pandemic prevention and medical resource management. We also verify that DFDMN is efficiently applicable not only to COVID-19 transmission prediction, but also to more general multivariate prediction tasks. It leads us to believe that it might be applied as a promising prediction model in other fields., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024 The Author(s). Published by Elsevier Ltd.. All rights reserved.)

Effective reflection of the spatio-temporal characteristics of time series is crucial in development of time-series-based surrogate models for hydrologic systems, especially in coastal areas. In this study, a deep learning-based surrogate modeling framework, named STA-GRU, is proposed to predict groundwater levels accurately and efficiently through incorporation of spatio-temporal attention mechanism of multivariate time series and gated recurrent neural network. Firstly, a three-dimensional groundwater flow model is developed based on GMS-MODFLOW and used to generate groundwater levels as input datasets for the STA-GRU framework. The spatio-temporal sequence window is then reconstructed, and the spatio-temporal attention mechanism is employed to assign different weights to the time series of each groundwater well and the time step of a single time series. The gated recurrent unit (GRU) is finally introduced to address the spatial and temporal characteristics of groundwater levels. The comparison between the ablation experiment and the baseline model demonstrates that the framework is efficient in reducing the conflict of non-target variables by capturing the spatiotemporal dependence of variables. The STA-GRU modeling framework developed in this study can effectively extract the spatio-temporal characteristics of the groundwater table and improve model performance. In addition, compared with the finite difference method, the STA-GRU surrogate model saves a lot of calculation and time costs to achieve accurate prediction of complex hydrological sequences. The proposed STA-GRU framework has provided an effective method for predicting groundwater levels in coastal areas., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024 Elsevier Ltd. All rights reserved.)

In recent years, the uncertainty of weather conditions and the impact of future climate change on building energy assessment has received increasing attention. As an important part of these studies, several types of methods for generating stochastic meteorological data have also been developed. Since solar radiation drops to zero at night, unlike the continuous 24-hour data for elements such as temperature and humidity, this has posed challenges for previous research to fully account for the simultaneity among multiple elements. Therefore, this study proposes a framework for meteorological data generation: First, perform multivariate time series modeling of meteorological data of air temperature, solar radiation and absolute humidity at 12:00 of each day of a typical year based on the S-vine copula method and simulating daily series data at 12:00 for 365 days. Then, based on the probability of change of each element evaluated from the historical meteorological observation data, the daily series data at 12:00 were expanded to 24 h, after which the yearly stochastic weather data were obtained. The analysis of 30 years of stochastic data generated by this method, compared with the original data, reveals that air temperature and solar radiation closely match the original distribution characteristics, except for a minor deviation in the absolute humidity's kurtosis. Furthermore, the comparison of thermal load distributions for office buildings shows that the original data curve falls within the range of the generated data. This suggests that the generated data includes more information about uncertainty but still keeps the original data's characteristics., (© 2024. The Author(s).)

We propose a multivariate GARCH model for non-stationary health time series by modifying the observation-level variance of the standard state space model. The proposed model provides an intuitive and novel way of dealing with heteroskedastic data using the conditional nature of state-space models. We follow the Bayesian paradigm to perform the inference procedure. In particular, we use Markov chain Monte Carlo methods to obtain samples from the resultant posterior distribution. We use the forward filtering backward sampling algorithm to efficiently obtain samples from the posterior distribution of the latent state. The proposed model also handles missing data in a fully Bayesian fashion. We validate our model on synthetic data and analyze a data set obtained from an intensive care unit in a Montreal hospital and the MIMIC dataset. We further show that our proposed models offer better performance, in terms of WAIC than standard state space models. The proposed model provides a new way to model multivariate heteroskedastic non-stationary time series data. Model comparison can then be easily performed using the WAIC., (© 2024 The Author(s). Statistics in Medicine published by John Wiley & Sons Ltd.)

To guarantee the safety and reliability of equipment operation, such as liquid rocket engine (LRE), carrying out system-level anomaly detection (AD) is crucial. However, current methods ignore the prior knowledge of mechanical system itself, and seldom unite the observations with the inherent relation in data tightly. Meanwhile, they neglect the weakness and nonindependence of system-level anomaly which is different from component fault. To overcome above limitations, we propose a separate reconstruction framework using worsened tendency for system-level AD. To prevent anomalous feature being attenuated, we first propose to divide single sample into two equal-length parts along the temporal dimension. And we maximize the mean maximum discrepancy (MMD) between feature segments to force encoders to learn normal features with different distributions. Then, to fully explore the multivariate time series, we model temporal-spatial dependence by temporal convolution and graph attention. Besides, a joint graph learning strategy is proposed to handle prior knowledge and data characteristics simultaneously. Finally, the proposed method is evaluated on two real multi-sensor datasets from LRE and the results demonstrate the effectiveness and potential of the proposed method on system-level AD., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024 ISA. Published by Elsevier Ltd. All rights reserved.)

Although existing reconstruction-based multivariate time series anomaly detection (MTSAD) methods have shown advanced performance, most assume the training data is clean. When faced with noise or contamination in training data, they can also reconstruct the anomaly well, weakening the distinction between normal and anomaly. Some probabilistic generation-based methods have been used to address this issue because of their implicit robust structure to noise, but the training process and suppression of anomalous generalization are not stable. The recently proposed explicit method based on the memory module would also sacrifice the reconstruction effect of normal patterns, resulting in limited performance improvement. Moreover, most existing MTSAD methods use a single fixed-length window for input, which weakens their ability to extract long-term dependency. This paper proposes a robust multi-scale feature extraction framework with the dual memory module to comprehensively extract features fusing different levels of semantic information and lengths of temporal dependency. First, this paper designs consecutive neighboring windows as inputs to allow the model to extract local and long-term dependency information. Secondly, a dual memory-augmented encoder is proposed to extract global typical patterns and local common features. It ensures the reconstruction ability of normal data while suppressing the generalization of the anomaly. Finally, this paper proposes a multi-scale fusion module to fuse latent variables representing different levels of semantic information and uses the reconstructed latent variables to reconstruct samples for anomaly detection. Experimental results on five datasets from diverse domains show that the proposed method outperforms 16 typical baseline methods., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024 Elsevier Ltd. All rights reserved.)

The Industrial Internet of Things has enabled the integration and analysis of vast volumes of data across various industries, with the maritime sector being no exception. Advances in cloud computing and deep learning (DL) are continuously reshaping the industry, particularly in optimizing maritime operations such as Predictive Maintenance (PdM). In this study, we propose a novel DL-based framework focusing on the fault detection task of PdM in marine operations, leveraging time-series data from sensors installed on shipboard machinery. The framework is designed as a scalable and cost-efficient software solution, encompassing all stages from data collection and pre-processing at the edge to the deployment and lifecycle management of DL models. The proposed DL architecture utilizes Graph Attention Networks (GATs) to extract spatio-temporal information from the time-series data and provides explainable predictions through a feature-wise scoring mechanism. Additionally, a custom evaluation metric with real-world applicability is employed, prioritizing both prediction accuracy and the timeliness of fault identification. To demonstrate the effectiveness of our framework, we conduct experiments on three types of open-source datasets relevant to PdM: electrical data, bearing datasets, and data from water circulation experiments.

Background: Describing the transmission dynamics of infectious diseases across different regions is crucial for effective disease surveillance. The multivariate time series (MTS) model has been widely adopted for constructing cross-regional infectious disease transmission networks due to its strengths in interpretability and predictive performance. Nevertheless, the assumption of constant parameters frequently disregards the dynamic shifts in disease transmission rates, thereby compromising the accuracy of early warnings. This study investigated the applicability of time-varying MTS models in multi-regional infectious disease monitoring and explored strategies for model selection., Methods: This study focused on two prominent time-varying MTS models: the time-varying parameter-stochastic volatility-vector autoregression (TVP-SV-VAR) model and the time-varying VAR model using the generalized additive framework (tvvarGAM), and intended to explore and verify their applicable conditions for the surveillance of infectious diseases. For the first time, this study proposed the time delay coefficient and spatial sparsity indicators for model selection. These indicators quantify the temporal lags and spatial distribution of infectious disease data, respectively. Simulation study adopted from real-world infectious disease surveillance was carried out to compare model performances under various scenarios of spatio-temporal variation as well as random volatility. Meanwhile, we illustrated how the modelling process could help the surveillance of infectious diseases with an application to the influenza-like case in Sichuan Province, China., Results: When the spatio-temporal variation was small (time delay coefficient: 0.1-0.2, spatial sparsity:0.1-0.3), the TVP-SV-VAR model was superior with smaller fitting residuals and standard errors of parameter estimation than those of the tvvarGAM model. In contrast, the tvvarGAM model was preferable when the spatio-temporal variation increased (time delay coefficient: 0.2-0.3, spatial sparsity: 0.6-0.9)., Conclusion: This study emphasized the importance of considering spatio-temporal variations when selecting appropriate models for infectious disease surveillance. By incorporating our novel indicators-the time delay coefficient and spatial sparsity-into the model selection process, the study could enhance the accuracy and effectiveness of infectious disease monitoring efforts. This approach was not only valuable in the context of this study, but also has broader implications for improving time-varying MTS analyses in various applications., (© 2024. The Author(s).)

The accurate effluent prediction plays a crucial role in providing early warning for abnormal effluent and achieving the adjustment of feedforward control parameters during wastewater treatment. This study applied a dual-staged attention mechanism based on long short-term memory network (DA-LSTM) to improve the accuracy of effluent quality prediction. The results showed that input attention (IA) and temporal attention (TA) significantly enhanced the prediction performance of LSTM. Specially, IA could adaptively adjust feature weights to enhance the robustness against input noise, with R 2 increased by 13.18%. To promote its long-term memory ability, TA was used to increase the memory span from 96 h to 168 h. Compared to a single LSTM model, the DA-LSTM model showed an improvement in prediction accuracy by 5.10%, 2.11%, 14.47% for COD, TP, and TN. Additionally, DA-LSTM demonstrated excellent generalization performance in new scenarios, with the R 2 values for COD, TP, and TN increasing by 22.67%, 20.06%, and 17.14% respectively, while the MAPE values decreased by 56.46%, 63.08%, and 42.79%. In conclusion, the DA-LSTM model demonstrated excellent prediction performance and generalization ability due to its advantages of feature-adaptive weighting and long-term memory focusing. This has forward-looking significance for achieving efficient early warning of abnormal operating conditions and timely management of control parameters., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024 Elsevier Ltd. All rights reserved.)

Quadratic Phase Coupling (QPC) serves as an essential statistical instrument for evaluating nonlinear synchronization within multivariate time series data, especially in signal processing and neuroscience fields. This study explores the precision of QPC detection using numerical estimates derived from cross-bicoherence and bivariate Granger causality within a straightforward, yet noisy, instantaneous multiplier model. It further assesses the impact of accidental statistically significant bifrequency interactions, introducing new metrics such as the ratio of bispectral quadratic phase coupling and the ratio of bivariate Granger causality quadratic phase coupling. Ratios nearing 1 signify a high degree of accuracy in detecting QPC. The coupling strength between interacting channels is identified as a key element that introduces nonlinearities, influencing the signal-to-noise ratio in the output channel. The model is tested across 59 experimental conditions of simulated recordings, with each condition evaluated against six coupling strength values, covering a wide range of carrier frequencies to examine a broad spectrum of scenarios. The findings demonstrate that the bispectral method outperforms bivariate Granger causality, particularly in identifying specific QPC under conditions of very weak couplings and in the presence of noise. The detection of specific QPC is crucial for neuroscience applications aimed at better understanding the temporal and spatial coordination between different brain regions., (© 2024. The Author(s).)

Background: Water is the lifeblood of river networks, and its quality plays a crucial role in sustaining both aquatic ecosystems and human societies. Real-time monitoring of water quality is increasingly reliant on in-situ sensor technology.Anomaly detection is crucial for identifying erroneous patterns in sensor data, but can be a challenging task due to the complexity and variability of the data, even under typical conditions. This paper presents a solution to the challenging task of anomaly detection for river network sensor data, which is essential for accurate and continuous monitoring., Methods: We use a graph neural network model, the recently proposed Graph Deviation Network (GDN), which employs graph attention-based forecasting to capture the complex spatio-temporal relationships between sensors. We propose an alternate anomaly threshold criteria for the model, GDN+, based on the learned graph. To evaluate the model's efficacy, we introduce new benchmarking simulation experiments with highly-sophisticated dependency structures and subsequence anomalies of various types. We also introduce software called gnnad., Results: We further examine the strengths and weaknesses of this baseline approach, GDN, in comparison to other benchmarking methods on complex real-world river network data., Conclusions: Findings suggest that GDN+ outperforms the baseline approach in high-dimensional data, while also providing improved interpretability., Competing Interests: No competing interests were disclosed., (Copyright: © 2024 Buchhorn K et al.)

Deep generative models have advantages in modeling complex time series and are widely used in anomaly detection. Nevertheless, the existing deep generative approaches mainly concentrate on the investigation of models' reconstruction capability rather than customizing a model suitable for anomaly detection. Meanwhile, VAE-based models suffer from posterior collapse, which can lead to a series of undesirable consequences, such as high false positive rate etc. Based on these considerations, in this paper, we propose a novel self-adversarial variational auto-encoder combined with contrast learning, short for ACVAE, to address these challenges. ACVAE consist of three parts 〈T,E,G〉, wherein the transformation network T is employed to generate abnormal latent representations similar to those normal latent representations encoded by the encoder E, and the decoder G is used to distinguish the two representations. In the framework of this model, the normal reconstructions are considered as positive samples and abnormal reconstructions as negative samples, and the contrast learning is executed on the part E to measure the similarities between inputs and positive samples, dissimilarities between inputs and negative samples. Thus, an improved objective function is proposed by integrating two novel regularizers, one refers to adversarial mechanism and the other involves contrast learning, in which the encoder E and decoder G hold the capability to distinguish, and decoder G is constrained to mitigate the posterior collapse. We perform several experiments on five datasets, whose results show ACVAE outperforms state-of-the-art methods., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2023 Elsevier Ltd. All rights reserved.)

Anomaly detection in multivariate time series is of critical importance in many real-world applications, such as system maintenance and Internet monitoring. In this article, we propose a novel unsupervised framework called SVD-AE to conduct anomaly detection in multivariate time series. The core idea is to fuse the strengths of both SVD and autoencoder to fully capture complex normal patterns in multivariate time series. An asymmetric autoencoder architecture is proposed, where two encoders are used to capture features in time and variable dimensions and a shared decoder is used to generate reconstructions based on latent representations from both dimensions. A new regularization based on singular value decomposition theory is designed to force each encoder to learn features in the corresponding axis with mathematical supports delivered. A specific loss component is further proposed to align Fourier coefficients of inputs and reconstructions. It can preserve details of original inputs, leading to enhanced feature learning capability of the model. Extensive experiments on three real world datasets demonstrate the proposed algorithm can achieve better performance on multivariate time series anomaly detection tasks under highly unbalanced scenarios compared with baseline algorithms., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2023. Published by Elsevier Ltd.)

While existing reconstruction-based multivariate time series (MTS) anomaly detection methods demonstrate advanced performance on many challenging real-world datasets, they generally assume the data only consists of normal samples when training models. However, real-world MTS data may contain significant noise and even be contaminated by anomalies. As a result, most existing approaches easily capture the pattern of the contaminated data, making identifying anomalies more difficult. Although a few studies have aimed to mitigate the interference of the noise and anomalies by introducing various regularizations, they still employ the objective of fully reconstructing the input data, impeding the model from learning an accurate profile of the MTS's normal pattern. Moreover, it is difficult for existing methods to apply the most appropriate normalization schemes for each dataset in various complex scenarios, particularly for mixed-feature MTS. This paper proposes a filter-augmented auto-encoder with learnable normalization (NormFAAE) for robust MTS anomaly detection. Firstly, NormFAAE designs a deep hybrid normalization module. It is trained with the backbone end-to-end in the current training task to perform the optimal normalization scheme. Meanwhile, it integrates two learnable normalization sub-modules to deal with the mixed-feature MTS effectively. Secondly, NormFAAE proposes a filter-augmented auto-encoder with a dual-phase task. It separates the noise and anomalies from the input data by a deep filter module, which facilitates the model to only reconstruct the normal data, achieving a more robust latent representation of MTS. Experimental results demonstrate that NormFAAE outperforms 17 typical baselines on five real-world industrial datasets from diverse fields., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2023 Elsevier Ltd. All rights reserved.)

Count data with an excess of zeros are often encountered when modeling infectious disease occurrence. The degree of zero inflation can vary over time due to nonepidemic periods as well as by age group or region. A well-established approach to analyze multivariate incidence time series is the endemic-epidemic modeling framework, also known as the HHH approach. However, it assumes Poisson or negative binomial distributions and is thus not tailored to surveillance data with excess zeros. Here, we propose a multivariate zero-inflated endemic-epidemic model with random effects that extends HHH. Parameters of both the zero-inflation probability and the HHH part of this mixture model can be estimated jointly and efficiently via (penalized) maximum likelihood inference using analytical derivatives. We found proper convergence and good coverage of confidence intervals in simulation studies. An application to measles counts in the 16 German states, 2005-2018, showed that zero inflation is more pronounced in the Eastern states characterized by a higher vaccination coverage. Probabilistic forecasts of measles cases improved when accounting for zero inflation. We anticipate zero-inflated HHH models to be a useful extension also for other applications and provide an implementation in an R package., (© 2023 The Authors. Biometrical Journal published by Wiley-VCH GmbH.)

Multivariate time series data is becoming increasingly ubiquitous in various fields such as servers, industrial applications, and healthcare. However, detecting anomalies in such data is challenging due to its complex time-dependent, high-dimensional, and label scarcity. Aiming at this problem, this paper proposes an Attention Factorization Normalizing Flow (AFNF) algorithm for unsupervised multivariate time series anomaly detection. Our hypothesis is that anomalies are in a low-density region of the distribution. To transform the complex density of high-dimensional time series into a simple evaluable conditional density, we propose a time series factorization strategy and parameterize the conditional information generated by factorization in the time and attribute dimensions using an attention mechanism. Moreover, to compensate for the lack of temporal information due to the permutation invariance attention mechanism, a adjacency contrasting approach is proposed to model the local invariance of the time series. To provide long-term location information, a learnable global location encoding is introduced. Conditional normalizing flows are applied to evaluate the conditional probability of the observations. Finally, through extensive experiments on three real data sets, our method yielded the best results and its effectiveness in density estimation and anomaly detection is demonstrated., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2023 ISA. Published by Elsevier Ltd. All rights reserved.)

Amid the ongoing emphasis on reducing manufacturing costs and enhancing productivity, one of the crucial objectives when manufacturing is to maintain process tools in optimal operating conditions. With advancements in sensing technologies, large amounts of data are collected during manufacturing processes, and the challenge today is to utilize these massive data efficiently. Some of these data are used for fault detection and classification (FDC) to evaluate the general condition of production machinery. The distinctive characteristics of semiconductor manufacturing, such as interdependent parameters, fluctuating behaviors over time, and frequently changing operating conditions, pose a major challenge in identifying defective wafers during the manufacturing process. To address this challenge, a multivariate fault detection method based on a 1D ResNet algorithm is introduced in this study. The aim is to identify anomalous wafers by analyzing the raw time-series data collected from multiple sensors throughout the semiconductor manufacturing process. To achieve this objective, a set of features is chosen from specified tools in the process chain to characterize the status of the wafers. Tests on the available data confirm that the gradient vanishing problem faced by very deep networks starts to occur with the plain 1D Convolutional Neural Network (CNN)-based method when the size of the network is deeper than 11 layers. To address this, a 1D Residual Network (ResNet)-based method is used. The experimental results show that the proposed method works more effectively and accurately compared to techniques using a plain 1D CNN and can thus be used for detecting abnormal wafers in the semiconductor manufacturing industry.

The prevalence of multivariate time series data across several disciplines fosters a demand and, subsequently, significant growth in the research and advancement of multivariate time series analysis. Drawing inspiration from a popular natural language processing model, the Transformer, we propose the Spatio-Temporal Transformer with Relative Embeddings (STTRE) to address multivariate time series forecasting. This work primarily focuses on developing a Transformer-based framework that can fully exploit the spatio-temporal nature of a multivariate time series by incorporating several of the Transformer's key components, but with augmentations that allow them to excel in multivariate time series forecasting. Current Transformer-based models for multivariate time series often neglect the data's spatial component(s) and utilize absolute position embeddings as their only means to detect the data's temporal component(s), which we show is flawed for time series applications. The lack of emphasis on fully exploiting the spatio-temporality of the data can incur subpar results in terms of accuracy. We redesign relative position representations, which we rename to relative embeddings, to unveil a new method for detecting latent spatial, temporal, and spatio-temporal dependencies more effectively than previous Transformer-based models. We couple these relative embeddings with a restructuring of the Transformer's primary sequence learning mechanism, multi-head attention, in a way that allows for full utilization of relative embeddings, thus achieving up to a 24% improvement in accuracy over other state-of-the-art multivariate time series models on a comprehensive selection of publicly available multivariate time series forecasting datasets., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2023 The Author(s). Published by Elsevier Ltd.. All rights reserved.)

Time series data continuously collected by different sensors play an essential role in monitoring and predicting events in many real-world applications, and anomaly detection for time series has received increasing attention during the past decades. In this paper, we propose an anomaly detection method by densely contrasting the whole time series with its sub-sequences at different timestamps in a latent space. Our approach leverages the locality property of convolutional neural networks (CNN) and integrates position embedding to effectively capture local features for sub-sequences. Simultaneously, we employ an attention mechanism to extract global features from the entire time series. By combining these local and global features, our model is trained using both instance-level contrastive learning loss and distribution-level alignment loss. Furthermore, we introduce a reconstruction loss applied to the extracted global features to prevent the potential loss of information. To validate the efficacy of our proposed technique, we conduct experiments on publicly available time-series datasets for anomaly detection. Additionally, we evaluate our method on an in-house mobile phone dataset aimed at monitoring the status of Parkinson's disease, all within an unsupervised learning framework. Our results demonstrate the effectiveness and potential of the proposed approach in tackling anomaly detection in time series data, offering promising applications in real-world scenarios., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2023. Published by Elsevier Ltd.)

Accurate estimation of in-hospital mortality based on patients' physiological time series data improves the performance of the clinical decision support systems and assists hospital providers in allocating resources. In practice, the data quality issues of missing values are ubiquitous in electronic health records (EHRs). Since the vital signs are usually observed with irregular temporal intervals and different sampling rates, it is challenging to predict clinical outcomes with sparse and incomplete multivariate time series. We propose an auto-regressive recurrent neural network (RNN) based model, dubbed the bi-directional recursive encoder-decoder network (BiRED), to jointly perform data imputation and mortality prediction. To capture complex patterns of medical time sequences, a 2D cross-regression with an RNN unit (2DCR-RNN) and an imputation block with an RNN unit (IB-RNN) are designed as the recurrent component of the encoder and decoder, respectively. Furthermore, a state initialization method is proposed to alleviate errors accumulated in the generated sequence. The experimental results on two real EHR datasets show that our proposed method can predict hospital mortality with high AUC scores., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2023. Published by Elsevier Ltd.)

Anomaly detection has been widely used in grid operation and maintenance, machine fault detection, and so on. In these applications, the multivariate time-series data from multiple sensors with latent relationships are always high-dimensional, which makes multivariate time-series anomaly detection particularly challenging. In existing unsupervised anomaly detection methods for multivariate time series, it is difficult to capture the complex associations among multiple sensors. Graph neural networks (GNNs) can model complex relations in the form of a graph, but the observed time-series data from multiple sensors lack explicit graph structures. GNNs cannot automatically learn the complex correlations in the multivariate time-series data or make good use of the latent relationships among time-series data. In this paper, we propose a new method-masked graph neural networks for unsupervised anomaly detection (MGUAD). MGUAD can learn the structure of the unobserved causality among sensors to detect anomalies. To robustly learn the temporal context from adjacent time points of time-series data from the same sensor, MGUAD randomly masks some points of the time-series data from the sensor and reconstructs the masked time points. Similarly, to robustly learn the graph-level context from adjacent nodes or edges in the relation graph of multivariate time series, MGUAD masks some nodes or edges in the graph under the framework of a GNN. Comprehensive experiments are conducted on three public datasets. According to the experimental findings, MGUAD outperforms state-of-the-art anomaly detection methods.

In light of the ongoing COVID-19 pandemic, predicting its trend would significantly impact decision-making. However, this is not a straightforward task due to three main difficulties: temporal autocorrelation, spatial dependency, and concept drift caused by virus mutations and lockdown policies. Although machine learning has been extensively used in related work, no previous research has successfully addressed all three challenges simultaneously. To overcome this challenge, we developed a novel online multi-task regression algorithm that incorporates a chain structure to capture spatial dependency, the ADWIN drift detector to adapt to concept drift, and the lag time series feature to capture temporal autocorrelation. We conducted several comparative experiments based on the number of daily confirmed cases in 20 areas in California and affiliated cities. The results from our experiments demonstrate that our proposed model is superior in adapting to concept drift in COVID-19 data and capturing spatial dependencies across various regions. This leads to a significant improvement in prediction accuracy when compared to existing state-of-the-art batch machine learning methods, such as N-Beats, DeepAR, TCN, and LSTM., Competing Interests: The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (© 2023 Published by Elsevier Ltd.)

In this study, we present a thorough comparison of the performance of four different bootstrap methods for assessing the significance of causal analysis in time series data. For this purpose, multivariate simulated data are generated by a linear feedback system. The methods investigated are uncorrelated Phase Randomization Bootstrap (uPRB), which generates surrogate data with no cross-correlation between variables by randomizing the phase in the frequency domain; Time Shift Bootstrap (TSB), which generates surrogate data by randomizing the phase in the time domain; Stationary Bootstrap (SB), which calculates standard errors and constructs confidence regions for weakly dependent stationary observations; and AR-Sieve Bootstrap (ARSB), a resampling method based on AutoRegressive (AR) models that approximates the underlying data-generating process. The uPRB method accurately identifies variable interactions but fails to detect self-feedback in some variables. The TSB method, despite performing worse than uPRB, is unable to detect feedback between certain variables. The SB method gives consistent causality results, although its ability to detect self-feedback decreases, as the mean block width increases. The ARSB method shows superior performance, accurately detecting both self-feedback and causality across all variables. Regarding the analysis of the Impulse Response Function (IRF), only the ARSB method succeeds in detecting both self-feedback and causality in all variables, aligning well with the connectivity diagram. Other methods, however, show considerable variations in detection performance, with some detecting false positives and others only detecting self-feedback.

Significant heterogeneity in network structures reflecting individuals' dynamic processes can exist within subgroups of people (e.g., diagnostic category, gender). This makes it difficult to make inferences regarding these predefined subgroups. For this reason, researchers sometimes wish to identify subsets of individuals who have similarities in their dynamic processes regardless of any predefined category. This requires unsupervised classification of individuals based on similarities in their dynamic processes, or equivalently, in this case, similarities in their network structures of edges. The present paper tests a recently developed algorithm, S-GIMME, that takes into account heterogeneity across individuals with the aim of providing subgroup membership and precise information about the specific network structures that differentiate subgroups. The algorithm has previously provided robust and accurate classification when evaluated with large-scale simulation studies but has not yet been validated on empirical data. Here, we investigate S-GIMME's ability to differentiate, in a purely data-driven manner, between brain states explicitly induced through different tasks in a new fMRI dataset. The results provide new evidence that the algorithm was able to resolve, in an unsupervised data-driven manner, the differences between different active brain states in empirical fMRI data to segregate individuals and arrive at subgroup-specific network structures of edges. The ability to arrive at subgroups that correspond to empirically designed fMRI task conditions, with no biasing or priors, suggests this data-driven approach can be a powerful addition to existing methods for unsupervised classification of individuals based on their dynamic processes., (© 2023. The Author(s) under exclusive licence to The Psychometric Society.)

As military technology continues to evolve and the amount of situational information available on the battlefield continues to increase, data-driven deep learning methods are becoming the primary method for air target intention recognition. Deep learning is based on a large amount of high quality data; however, in the field of intention recognition, it often faces key problems such as low data volume and unbalanced datasets due to insufficient real-world scenarios. To address these problems, we propose a new method called time-series conditional generative adversarial network with improved Hausdorff distance (IH-TCGAN). The innovation of the method is mainly reflected in three aspects: (1) Use of a transverter to map real and synthetic data into the same manifold so that they have the same intrinsic dimension; (2) Addition of a restorer and a classifier in the network structure to ensure that the model can generate high-quality multiclass temporal data; (3) An improved Hausdorff distance is proposed that can measure the time order differences between multivariate time-series data and make the generated results more reasonable. We conduct experiments using two time-series datasets, evaluate the results using various performance metrics, and visualize the results using visualization techniques. The experimental results show that IH-TCGAN is able to generate synthetic data similar to the real data and has significant advantages in the generation of time series data.

Because of global lock-downs caused by the unexpected COVID-19, the interactions between emission trading and related markets have changed significantly compared to the pre-COVID-19 period. Considering the pandemic effect, this paper established an integrated system to identify the relationship trajectories between carbon trading market and impact factors. A noise-assisted multivariate empirical mode decomposition (N-A MEMD) method was utilized to simultaneously decompose the original multi-dimensional time series into intrinsic mode functions (IMFs), after which the Lempel-Ziv (LZ) complexity algorithm was applied to reconstruct the IMFs into high-frequency (HF), low-frequency (LF), and trend modules. Vector autoregression (VAR) and vector error correction (VEC) models were then used to systematically simulate the correlations. The time span was split into pre-COVID-19 and post-COVID-19 periods for comparison, and the mobility trends data during the outbreak period released by the Apple company was chosen to reflect the pandemic effects. The empirical analysis results revealed the energy prices, macroeconomic index, and exchange rate are the main external impact factors of carbon price in the short term. Summarizing from the cointegration models over the long term, the market stability reserve (MSR) mechanism was found to have ability on stabilizing the carbon price under the epidemic shock. Furthermore, the COVID-19 was found to complicate the relationships between carbon price and influence factors, which resulted in fluctuating markets., (© 2022. The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature.)

Bacterial resistance to antibiotics has been rapidly increasing, resulting in low antibiotic effectiveness even treating common infections. The presence of resistant pathogens in environments such as a hospital Intensive Care Unit (ICU) exacerbates the critical admission-acquired infections. This work focuses on the prediction of antibiotic resistance in Pseudomonas aeruginosa nosocomial infections at the ICU, using Long Short-Term Memory (LSTM) artificial neural networks as the predictive method. The analyzed data were extracted from the Electronic Health Records (EHR) of patients admitted to the University Hospital of Fuenlabrada from 2004 to 2019 and were modeled as Multivariate Time Series. A data-driven dimensionality reduction method is built by adapting three feature importance techniques from the literature to the considered data and proposing an algorithm for selecting the most appropriate number of features. This is done using LSTM sequential capabilities so that the temporal aspect of features is taken into account. Furthermore, an ensemble of LSTMs is used to reduce the variance in performance. Our results indicate that the patient's admission information, the antibiotics administered during the ICU stay, and the previous antimicrobial resistance are the most important risk factors. Compared to other conventional dimensionality reduction schemes, our approach is able to improve performance while reducing the number of features for most of the experiments. In essence, the proposed framework achieve, in a computationally cost-efficient manner, promising results for supporting decisions in this clinical task, characterized by high dimensionality, data scarcity, and concept drift., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2023 The Authors. Published by Elsevier B.V. All rights reserved.)

Purpose: To develop a deep learning network that treats the three-dimensional respiratory motion signals as a whole and considers the inter-dimensional correlation between signals of different directions for accurate respiratory tumor motion prediction., Methods: We propose a deep learning framework, named as LSTM-Global Temporal Convolution-External Attention Network (LGEANet). In LGEANet, we first feed each of the univariate time series into the Long Short-Term Memory (LSTM) module respectively and utilize the strength of the global temporal convolutional layer to discover the temporal pattern of the univariate signals from hidden states of the LSTM. Then, External attention is adopted to capture the dynamic dependence of the multiple time series. Also, a traditional autoregressive linear model in parallel to the non-linear neural network part was integrated to mitigate the scale insensitivity of the networks. A total of 304 motion traces for 31 patients are acquired from a public dataset in the experiments and four representative cases were selected for model evaluation. The respiratory signals were sampled at intervals of about 37.5 ms (26 frames per second) for an average duration of 71 min., Results: The proposed LGEANet achieved better performance with higher empirical correlation coefficient value (CORRs) and lower mean absolute error value (MAEs) and relative squared error value (RSEs) than other investigated models. For the four representative datasets, when the response time is less than 231 ms, the model can achieve CORRs more than 0.96. And the averaged position error reduction by using the proposed model was about 67% in the superior-inferior (SI) direction, 41% in the anterior-posterior (AP) direction and 38% in the right-left (RL) direction compared to that without prediction. The proposed network achieved the greatest error reduction in the SI direction, which is the main direction of tumor motion., Conclusions: The LGEANet achieves promising performance in minimizing the prediction error due to system latencies during real-time tumor motion tracking., (© 2023 American Association of Physicists in Medicine.)

Industrial data scarcity is one of the largest factors holding back the widespread use of machine learning in manufacturing. To overcome this problem, the concept of transfer learning was developed and has received much attention in recent industrial research. This paper focuses on the problem of time series segmentation and presents the first in-depth research on transfer learning for deep learning-based time series segmentation on the industrial use case of end-of-line pump testing. In particular, we investigate whether the performance of deep learning models can be increased by pretraining the network with data from other domains. Three different scenarios are analyzed: source and target data being closely related, source and target data being distantly related, and source and target data being non-related. The results demonstrate that transfer learning can enhance the performance of time series segmentation models with respect to accuracy and training speed. The benefit can be most clearly seen in scenarios where source and training data are closely related and the number of target training data samples is lowest. However, in the scenario of non-related datasets, cases of negative transfer learning were observed as well. Thus, the research emphasizes the potential, but also the challenges, of industrial transfer learning.

Modeling complex spatial and temporal dependencies in multivariate time series data is crucial for traffic forecasting. Graph convolutional networks have proved to be effective in predicting multivariate time series. Although a predefined graph structure can help the model converge to good results quickly, it also limits the further improvement of the model due to its stationary state. In addition, current methods may not converge on some datasets due to the graph structure of these datasets being difficult to learn. Motivated by this, we propose a novel model named Dynamic Correlation Graph Convolutional Network (DCGCN) in this paper. The model can construct adjacency matrices from input data using a correlation coefficient; thus, dynamic correlation graph convolution is used for capturing spatial dependencies. Meanwhile, gated temporal convolution is used for modeling temporal dependencies. Finally, we performed extensive experiments to evaluate the performance of our proposed method against ten existing well-recognized baseline methods using two original and four public datasets.

The recent wave of digitalization is characterized by the widespread deployment of sensors in many different environments, e.g., multi-sensor systems represent a critical enabling technology towards full autonomy in industrial scenarios. Sensors usually produce vast amounts of unlabeled data in the form of multivariate time series that may capture normal conditions or anomalies. Multivariate Time Series Anomaly Detection (MTSAD), i.e., the ability to identify normal or irregular operative conditions of a system through the analysis of data from multiple sensors, is crucial in many fields. However, MTSAD is challenging due to the need for simultaneous analysis of temporal (intra-sensor) patterns and spatial (inter-sensor) dependencies. Unfortunately, labeling massive amounts of data is practically impossible in many real-world situations of interest (e.g., the reference ground truth may not be available or the amount of data may exceed labeling capabilities); therefore, robust unsupervised MTSAD is desirable. Recently, advanced techniques in machine learning and signal processing, including deep learning methods, have been developed for unsupervised MTSAD. In this article, we provide an extensive review of the current state of the art with a theoretical background about multivariate time-series anomaly detection. A detailed numerical evaluation of 13 promising algorithms on two publicly available multivariate time-series datasets is presented, with advantages and shortcomings highlighted.

Given a cardiac-arrest patient being monitored in the ICU (intensive care unit) for brain activity, how can we predict their health outcomes as early as possible? Early decision-making is critical in many applications, e.g. monitoring patients may assist in early intervention and improved care. On the other hand, early prediction on EEG data poses several challenges: (i) earliness-accuracy trade-off; observing more data often increases accuracy but sacrifices earliness, (ii) large-scale (for training) and streaming (online decision-making) data processing, and (iii) multi-variate (due to multiple electrodes) and multi-length (due to varying length of stay of patients) time series. Motivated by this real-world application, we present BeneFitter that infuses the incurred savings from an early prediction as well as the cost from misclassification into a unified domain-specific target called benefit. Unifying these two quantities allows us to directly estimate a single target (i.e. benefit), and importantly, (a) is efficient and fast, with training time linear in the number of input sequences, and can operate in real-time for decision-making, (b) can handle multi-variate and variable-length time-series, suitable for patient data, and (c) is effective, providing up to 2× time-savings with equal or better accuracy as compared to competitors., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2023 Elsevier Inc. All rights reserved.)

Forecasting the stock market trend and movement is a challenging task due to multiple factors, including the stock's natural volatility and nonlinearity. It concerns discovering the market's hidden patterns with respect to time to enable proactive decision-making and better futuristic insights. Recurrent neural network-based methods have been a prime candidate for solving complex and nonlinear sequences, including the task of modeling multivariate time series forecasts. Due to the lack of comprehensive and reference work in short-term forecasts for the Saudi stock price and trends, this article introduces a comprehensive and accurate forecasting methodology tailored to the Saudi stock market. Two steps were configured to render effective short-term forecasts. First, a custom-built feature engineering streamline was constructed to preprocess the raw stock data and enable financial-related technical indicators, followed by a stride-based sliding window to produce multivariate time series data ready for the modeling phase. Second, a well-architected Gated Recurrent Unit (GRU) model was constructed and carefully calibrated to yield accurate multi-step forecasts, which was trained using the recently published historical multivariate time-series data from the primary Saudi stock market index (TASI index), in addition to being benchmarked against a suitable baseline model, namely Vector Autoregression Moving-Average with Exogenous Regressors (VARMAX). The output predictions from the proposed GRU model and the VARMAX model were evaluated using a set of regression-based metrics to assess and interpret the model precision. The empirical results demonstrate that the proposed methodology yields outstanding short-term forecasts of the Saudi stock price trends price compared to existing efforts related to this work., Competing Interests: The author declares that they have no competing interests., (© 2023 Aseeri.)

Clean air in cities improves our health and overall quality of life and helps fight climate change and preserve our environment. High-resolution measures of pollutants' concentrations can support the identification of urban areas with poor air quality and raise citizens' awareness while encouraging more sustainable behaviors. Recent advances in Internet of Things (IoT) technology have led to extensive use of low-cost air quality sensors for hyper-local air quality monitoring. As a result, public administrations and citizens increasingly rely on information obtained from sensors to make decisions in their daily lives and mitigate pollution effects. Unfortunately, in most sensing applications, sensors are known to be error-prone. Thanks to Artificial Intelligence (AI) technologies, it is possible to devise computationally efficient methods that can automatically pinpoint anomalies in those data streams in real time. In order to enhance the reliability of air quality sensing applications, we believe that it is highly important to set up a data-cleaning process. In this work, we propose AIrSense, a novel AI-based framework for obtaining reliable pollutant concentrations from raw data collected by a network of low-cost sensors. It enacts an anomaly detection and repairing procedure on raw measurements before applying the calibration model, which converts raw measurements to concentration measurements of gasses. There are very few studies of anomaly detection in raw air quality sensor data (millivolts). Our approach is the first that proposes to detect and repair anomalies in raw data before they are calibrated by considering the temporal sequence of the measurements and the correlations between different sensor features. If at least some previous measurements are available and not anomalous, it trains a model and uses the prediction to repair the observations; otherwise, it exploits the previous observation. Firstly, a majority voting system based on three different algorithms detects anomalies in raw data. Then, anomalies are repaired to avoid missing values in the measurement time series. In the end, the calibration model provides the pollutant concentrations. Experiments conducted on a real dataset of 12,000 observations produced by 12 low-cost sensors demonstrated the importance of the data-cleaning process in improving calibration algorithms' performances.

The Vector AutoRegressive Moving Average (VARMA) model is fundamental to the theory of multivariate time series; however, identifiability issues have led practitioners to abandon it in favor of the simpler but more restrictive Vector AutoRegressive (VAR) model. We narrow this gap with a new optimization-based approach to VARMA identification built upon the principle of parsimony. Among all equivalent data-generating models, we use convex optimization to seek the parameterization that is simplest in a certain sense. A user-specified strongly convex penalty is used to measure model simplicity, and that same penalty is then used to define an estimator that can be efficiently computed. We establish consistency of our estimators in a double-asymptotic regime. Our non-asymptotic error bound analysis accommodates both model specification and parameter estimation steps, a feature that is crucial for studying large-scale VARMA algorithms. Our analysis also provides new results on penalized estimation of infinite-order VAR, and elastic net regression under a singular covariance structure of regressors, which may be of independent interest. We illustrate the advantage of our method over VAR alternatives on three real data examples.

Multivariate time series long-term forecasting has always been the subject of research in various fields such as economics, finance, and traffic. In recent years, attention-based recurrent neural networks (RNNs) have received attention due to their ability of reducing error accumulation. However, the existing attention-based RNNs fail to eliminate the negative influence of irrelevant factors on prediction, and ignore the conflict between exogenous factors and target factor. To tackle these problems, we propose a novel Hierarchical Attention Network (HANet) for multivariate time series long-term forecasting. At first, HANet designs a factor-aware attention network (FAN) and uses it as the first component of the encoder. FAN weakens the negative impact of irrelevant exogenous factors on predictions by assigning small weights to them. Then HANet proposes a multi-modal fusion network (MFN) as the second component of the encoder. MFN employs a specially designed multi-modal fusion gate to adaptively select how much information about the expression of current time come from target and exogenous factors. Experiments on two real-world datasets reveal that HANet not only outperforms state-of-the-art methods, but also provides interpretability for prediction., (© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2022.)

Multivariate time series prediction models perform the required operation on a specific window length of a given input. However, capturing complex and nonlinear interdependencies in each temporal window remains challenging. The typical attention mechanisms assign a weight for a variable at the same time or the features of each previous time step to capture spatio-temporal correlations. However, it fails to directly extract each time step's relevant features that affect future values to learn the spatio-temporal pattern from a global perspective. To this end, a temporal window attention-based window-dependent long short-term memory network (TWA-WDLSTM) is proposed to enhance the temporal dependencies, which exploits the encoder-decoder framework. In the encoder, we design a temporal window attention mechanism to select relevant exogenous series in a temporal window. Furthermore, we introduce a window-dependent long short-term memory network (WDLSTM) to encode the input sequences in a temporal window into a feature representation and capture very long term dependencies. In the decoder, we use WDLSTM to generate the prediction values. We applied our model to four real-world datasets in comparison to a variety of state-of-the-art models. The experimental results suggest that TWA-WDLSTM can outperform comparison models. In addition, the temporal window attention mechanism has good interpretability. We can observe which variable contributes to the future value.

As a measure of complexity, information entropy is frequently used to categorize time series, such as machinery failure diagnostics, biological signal identification, etc., and is thought of as a characteristic of dynamic systems. Many entropies, however, are ineffective for multivariate scenarios due to correlations. In this paper, we propose a local structure entropy (LSE) based on the idea of a recurrence network. Given certain tolerance and scales, LSE values can distinguish multivariate chaotic sequences between stochastic signals. Three financial market indices are used to evaluate the proposed LSE. The results show that the LSEFSTE100 and LSES&P500 are higher than LSESZI, which indicates that the European and American stock markets are more sophisticated than the Chinese stock market. Additionally, using decision trees as the classifiers, LSE is employed to detect bearing faults. LSE performs higher on recognition accuracy when compared to permutation entropy.

An information theoretic reduction of auto-regressive modeling called the Reduced Auto-Regressive (RAR) modeling is applied to several multivariate time series as a method to detect the relationships among the components in the time series. The results are compared with the results of the transfer entropy, one of the common techniques for detecting causal relationships. These common techniques are pairwise by definition and could be inappropriate in detecting the relationships in highly complicated dynamical systems. When the relationships between the dynamics of the components are linear and the time scales in the fluctuations of each component are in the same order of magnitude, the results of the RAR model and the transfer entropy are consistent. When the time series contain components that have large differences in the amplitude and the time scales of fluctuation, however, the transfer entropy fails to detect the correct relationships between the components, while the results of the RAR modeling are still correct. For a highly complicated dynamics such as human brain activity observed by electroencephalography measurements, the results of the transfer entropy are drastically different from those of the RAR modeling., Competing Interests: TT is employed by Toyota Motor Corporation. The remaining author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2022 Tanizawa and Nakamura.)

Modern healthcare practice, especially in intensive care units, produces a vast amount of multivariate time series of health-related data, e.g., multi-lead electrocardiogram (ECG), pulse waveform, blood pressure waveform and so on. As a result, timely and accurate prediction of medical intervention (e.g., intravenous injection) becomes possible, by exploring such semantic-rich time series. Existing works mainly focused on onset prediction at the granularity of hours that was not suitable for medication intervention in emergency medicine. This research proposes a Multi-Variable Hybrid Attentive Model (MVHA) to predict the impending need of medical intervention, by jointly mining multiple time series. Specifically, a two-level attention mechanism is designed to capture the pattern of fluctuations and trends of different time series. This work applied MVHA to the prediction of the impending intravenous injection need of critical patients at the intensive care units. Experiments on the MIMIC Waveform Database demonstrated that the proposed model achieves a prediction accuracy of 0.8475 and an ROC-AUC of 0.8318, which significantly outperforms baseline models.

Objective. In this work, an analytical framework for the multiscale analysis of multivariate Gaussian processes is presented, whereby the computation of Partial Information Decomposition measures is achieved accounting for the simultaneous presence of short-term dynamics and long-range correlations. Approach. We consider physiological time series mapping the activity of the cardiac, vascular and respiratory systems in the field of Network Physiology. In this context, the multiscale representation of transfer entropy within the network of interactions among Systolic arterial pressure (S), respiration (R) and heart period (H), as well as the decomposition into unique, redundant and synergistic contributions, is obtained using a Vector AutoRegressive Fractionally Integrated (VARFI) framework for Gaussian processes. This novel approach allows to quantify the directed information flow accounting for the simultaneous presence of short-term dynamics and long-range correlations among the analyzed processes. Additionally, it provides analytical expressions for the computation of the information measures, by exploiting the theory of state space models. The approach is first illustrated in simulated VARFI processes and then applied to H, S and R time series measured in healthy subjects monitored at rest and during mental and postural stress. Main Results. We demonstrate the ability of the VARFI modeling approach to account for the coexistence of short-term and long-range correlations in the study of multivariate processes. Physiologically, we show that postural stress induces larger redundant and synergistic effects from S and R to H at short time scales, while mental stress induces larger information transfer from S to H at longer time scales, thus evidencing the different nature of the two stressors. Significance. The proposed methodology allows to extract useful information about the dependence of the information transfer on the balance between short-term and long-range correlations in coupled dynamical systems, which cannot be observed using standard methods that do not consider long-range correlations., (© 2022 Institute of Physics and Engineering in Medicine.)

Multivariate time series forecasting remains a challenging task because of its nonlinear, non-stationary, high-dimensional, and spatial-temporal characteristics, along with the dependence between variables. To address this limitation, we propose a novel method for multivariate time series forecasting based on nonlinear spiking neural P (NSNP) systems and non-subsampled shearlet transform (NSST). A multivariate time series is first converted into the NSST domain, and then NSNP systems are automatically constructed, trained, and predicted in the NSST domain. Because NSNP systems are used as nonlinear prediction models and work in the NSST domain, the proposed prediction method is essentially a multiscale transform (MST)-based prediction method. Therefore, the proposed prediction method can process nonlinear and non-stationary time series, and the dependence between variables can be characterized by the multiresolution features of the NSST transform. Five real-life multivariate time series were used to compare the proposed prediction method with five state-of-the-art and 28 baseline prediction methods. The comparison results demonstrate the effectiveness of the proposed method for multivariate time-series forecasting., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2022 Elsevier Ltd. All rights reserved.)

The problem addressed by dictionary learning (DL) is the representation of data as a sparse linear combination of columns of a matrix called dictionary. Both the dictionary and the sparse representations are learned from the data. We show how DL can be employed in the imputation of multivariate time series. We use a structured dictionary, which is comprised of one block for each time series and a common block for all the time series. The size of each block and the sparsity level of the representation are selected by using information theoretic criteria. The objective function used in learning is designed to minimize either the sum of the squared errors or the sum of the magnitudes of the errors. We propose dimensionality reduction techniques for the case of high-dimensional time series. For demonstrating how the new algorithms can be used in practical applications, we conduct a large set of experiments on five real-life data sets. The missing data (MD) are simulated according to various scenarios where both the percentage of MD and the length of the sequences of MD are considered. This allows us to identify the situations in which the novel DL-based methods are superior to the existing methods.