Rethinking Attention Mechanism for Spatio-Temporal Modeling: A Decoupling Perspective in Traffic Flow Prediction

Qi Yu, Weilong Ding, Hao Zhang, Yang Yang, Tianpu Zhang

Abstract: The attention mechanism has the advantage of handling long-term correlations, and has been widely adopted in multivariate time series (MTS) prediction. As an important application of MTS, traffic flow prediction has the most popular solution using transformer-based prediction models nowadays. Just with attention mechanism, those models can learn the spatio-temporal correlations from traffic data. However, the up-to-date linear prediction models have questioned the effectiveness of current transformer-based models in certain conditions, which provides new possibilities for more efficient work. We rethink the role of the attention mechanism during spatio-temporal modeling from a decoupling perspective, and propose DEC-Former for traffic flow prediction. Specifically, the trend and seasonal parts of the time series data, the geographical adjacency of the nodes in the road network, and the traditional encoder-decoder architecture, are respectively decoupled. Such decoupling leverages the attention mechanism's advantage to capture long-term and long-range correlations.From extensive experiments on four real-world datasets, our work proves better predictive performance and efficiency than state-of-the-art attention-based models. Two case studies further show the distinct real effects.