CSTformer: Cross Spatio-Temporal Transformer for Multivariate Time Series Forecasting

Tao Cai, Haixiang Wu, Dejiao Niu, Lei Li

Abstract: Transformer-based models have shown remarkable success in Multivariate Time Series Forecasting (MTSF). Previous methods apply Attention mechanisms to capture temporal and spatial (variable) dependencies separately. However, they struggle to model intricate local spatiotemporal correlations. To address this limitation and enhance performance, we propose CSTformer, a novel Transformer-based model that enables capturing Cross Spatio- Temporal (CST) dependency for MTSF. In CSTformer, through a Variate Compete Linear Attention (VCLA) mechanism, each variable efficiently achieves specific CST features, which compete against the backdrop of the local multivariate. Additionally, we develop a Mixture of Latent (MoL) module to provide adaptive predictions for variables with varying degrees of CST dependencies. Our experimental results on nine benchmarks indicate that, compared with the state-of-the-art method, CSTformer yields a 2.7% relative improvement.