CAWformer: A cross variable attention with discrete wavelet denoising for multivariate time series forecasting

Shiming Fan, Hua Wang, Fan Zhang

Abstract: Multivariate time series forecasting plays an important role in finance, transportation, energy, and healthcare. However, existing Transformer-based models usually focus only on intra- or inter-series dependencies, which limits their ability to capture complex relationships among variables and dynamic patterns within a series, and thus limits their forecasting performance. To address this problem, we propose CAWformer, which achieves accurate prediction by capturing both inter- and intra-sequence features. First, to extract the within-sequence features, we perform multi-scale slicing of the time series and model the sequence fluctuation patterns using an autoregressive shift method, and then apply the autoregressive attention to the shifted data, which allows the model to efficiently identify the key relationships across the time steps and thus enhances its ability to capture the within-sequence features. In addition, to better capture complex relationships between variables, we compute cross-correlations in the frequency domain and use this information to measure interactions between features. This approach allows the model to accurately identify and exploit potential dependencies between variables. Inspired by the theory of decomposition of time series stochastic processes, we address the problem of random noise in residual signals. We decompose the residual terms into signals of different scales by means of the discrete wavelet transform (DWT), and then process the signals in such a way that the model is able to suppress random noise while preserving key details, thus reducing the risk of overfitting. Experiments on several real datasets show that CAWformer’s prediction accuracy is significantly better than existing state-of-the-art models.