1. tsGT: Stochastic Time Series Modeling With Transformer
- Author
-
Kuciński, Łukasz, Drzewakowski, Witold, Olko, Mateusz, Kozakowski, Piotr, Maziarka, Łukasz, Nowakowska, Marta Emilia, Kaiser, Łukasz, and Miłoś, Piotr
- Subjects
Computer Science - Machine Learning - Abstract
Time series methods are of fundamental importance in virtually any field of science that deals with temporally structured data. Recently, there has been a surge of deterministic transformer models with time series-specific architectural biases. In this paper, we go in a different direction by introducing tsGT, a stochastic time series model built on a general-purpose transformer architecture. We focus on using a well-known and theoretically justified rolling window backtesting and evaluation protocol. We show that tsGT outperforms the state-of-the-art models on MAD and RMSE, and surpasses its stochastic peers on QL and CRPS, on four commonly used datasets. We complement these results with a detailed analysis of tsGT's ability to model the data distribution and predict marginal quantile values.
- Published
- 2024