MrCAN: Multi-relations aware convolutional attention network for multivariate time series forecasting

Jing Zhang, Qun Dai

Abstract: Multivariate time series forecasting (MTSF) has gathered extensive attention in various research areas. Many researchers leverage deep neural networks to explore spatial–temporal relationships for MTSF with great success. Nevertheless, plentifully available data required by deep neural networks often struggle to satisfy practical scenarios. To overcome this limitation, we exploit the internal structure of deep neural networks to automatically learn sample relationships, aiming to mitigate data scarcity by propagating information among different data samples. Consequently, in this paper, we propose a Multi-Relations aware Convolutional Attention Network, termed MrCAN, which integrates spatial–temporal relation learning and sample relation learning, ad hoc for addressing MTSF issues. In MrCAN, we propose a novel spatial–temporal attention module for spatial–temporal relationship representation learning. With the proposed batch attention module, we further explore relational modeling among samples in each mini-batch to implicitly enrich the training sample information. In particular, we introduce parameter-sharing regressors located before and after the batch attention module to alleviate the training and testing inconsistency in learning batch invariant representations. Extensive experiments on six practical datasets demonstrate that MrCAN compares favorably to nine baseline models. Larger performance gaps are exhibited, especially on small datasets like the SML2010 dataset. Code is publicly available at https://github.com/JZhangNA/MrCAN.