TwinsFormer: Revisiting Inherent Dependencies via Two Interactive Components for Time Series Forecasting

Yingbo Zhou, Yutong Ye, Pengyu Zhang, Xiao Du, Mingsong Chen

Keywords: Time Series Forecasting, Interactive Components

TL;DR: Empowered by a novel interactive design, we integrate decomposition into the Transformer architecture, enabling effective learning of inherent dependencies.

Abstract: Due to the remarkable ability to capture long-term dependencies, Transformer-based models have shown great potential in time series forecasting. However, real-world time series usually present intricate temporal patterns, making forecasting still challenging in many practical applications. To better grasp inherent dependencies, in this paper, we propose TwinsFormer, a novel Transformer-based framework utilizing two interactive components for time series forecasting. Unlike the mainstream paradigms of plain decomposition that train the model with two independent branches, we design an interactive strategy around the attention module and the feed-forward network to strengthen the dependencies via decomposed components. Specifically, we adopt dual streams to facilitate progressive and implicit information interactions for trend and seasonal components. For the seasonal stream, we feed the seasonal component to the attention module and feed-forward network with a subtraction mechanism. Meanwhile, we construct an auxiliary highway (without the attention module) for the trend stream under the supervision of seasonal signals. In this way, we can avoid the model overlooking inherent dependencies between different components for accurate forecasting. Our interactive strategy, albeit simple, can be adapted as a plug-and-play module to existing Transformer-based methods with negligible extra computational overhead. Extensive experiments on various real-world datasets show the superiority of TwinsFormer, which can outperform previous state-of-the-art methods in terms of both long-term and short-term forecasting performance.

Primary Area: learning on time series and dynamical systems

Submission Number: 9228