1. Exploring deep learning architectures for spatiotemporal sequence forecasting
- Author
-
Shi, Xingjian CSE and Shi, Xingjian CSE
- Abstract
Spatiotemporal systems are common in the real world. Forecasting the multi-step future of these spatiotemporal systems based on past observations, or, Spatiotemporal Sequence Forecasting (STSF), is a significant and challenging problem. Due to the complex spatial and temporal relationships within the data and the potential long forecast horizon, it is challenging to design appropriate Deep Learning (DL) architectures for STSF. In this thesis, we explore DL architectures for STSF. We first define the STSF problem and classify it into three subcategories: Trajectory Forecasting of Moving Point Cloud (TF-MPC), STSF on Regular Grid (STSF-RG), and STSF on Irregular Grid (STSF-IG). We then propose architectures for STSF-RG and STSF-IG problems. For the STSF-RG problems, we proposed the Convolutional Long-Short Term Memory (ConvLSTM) and the Trajectory Gated Recurrent Unit (TrajGRU). ConvLSTM uses convolution in both the input-state and state-state transitions of LSTM and is better at capturing the spatiotemporal correlations than the Fully-connected LSTM (FC-LSTM). TrajGRU improves upon ConvLSTM by actively learning the recurrent connection structure, which achieves better prediction performance with less parameters. To better investigate the effectiveness of our proposed architectures and other DL models for STSF-RG, we chose to tackle the precipitation nowcasting problem, which is a representative STSF-RG problem with a huge real-world impact. By incorporating ConvLSTM into an Encoder-Forecaster (EF) structure, we proposed the first machine learning based solution for precipitation nowcasting that outperforms the operational algorithm. To facilitate future studies for this problem and gauge the state-of the-art methods, we proposed the first large-scale benchmark for precipitation nowcasting: HKO-7. HKO-7 has new evaluation metrics and has both the offline setting and the online settings in the evaluation protocol. We evaluated seven models in the offline and online set
- Published
- 2018