CPAT: cross-patch aggregated transformer for time series forecasting

Bingyan Liu, Li Wu, Xiaoying Wang, Jianqiang Huang, Guojing Zhang

Abstract: Time series forecasting utilizes historical data to forecast future information over a specific period. It aims to predict forthcoming developmental trends through meticulous statistical analysis and modeling of historical data, addressing real-life challenges like power load prediction, traffic condition prognostication, and extreme weather warnings. Currently, Transformer-based models for time series prediction normally segment the original time series into multiple patches. While this modeling methodology has demonstrated superiority in improving performance, the approach of patch partition based on a fixed length constrains the model’s predictive accuracy when dealing with time series forecasting tasks of varying lengths. To overcome the limitation, this article proposes an innovative Cross-Patch Aggregated Transformer (CPAT), which introduces the Patch Reconstruction module to restructure patches between encoder layers, facilitating cross-patch connections and information interaction. This empowers the model to focus on the correlation among adjacent patches, acquiring effective representations of both global and local features. Consequently, the modeling of time dependency becomes more precise. Extensive experiments conducted on eight publicly available benchmark datasets in real-world scenarios showcase that the proposed CPAT model attains state-of-the-art (SOTA) accuracy overall compared to existing baseline models. Notably, it achieves relative improvement rates of 5.46% and 2.56% for Mean Square Error (MSE) and Mean Absolute Error (MAE), respectively, augmenting the predictive capabilities of Transformer family models in time series tasks.

External IDs:dblp:journals/ml/LiuWWHZ25