Is Bidirectionality Necessary in Mamba for Time Series Forecasting?

Seunghan Lee, Juri Hong, Kibok Lee, Taeyoung Park

Keywords: Time Series Forecasting, Mamba

TL;DR: We propose FSMamba, a time series forecasting method that uses Mamba with regularization strategy to address the sequential order bias in capturing channel dependencies.

Abstract:

Mamba is a sequential model that has recently emerged as a promising alternative to Transformers, offering near-linear complexity.
However, although channels in time series (TS) data generally lack a sequential order, recent studies have adopted Mamba to capture channel dependencies (CD) in TS, introducing a sequential order bias. To address this, prior works have adopted bidirectional Mamba to scan channels in both forward and reverse orders. In this paper, we show that unidirectional Mamba can effectively replace the bidirectional Mamba with simple strategies. To this end, we propose FSMamba, a TS forecasting method employing a unidirectional Mamba that incorporates a regularization strategy to minimize the discrepancy between two embedding vectors generated from data with reversed channel orders, thereby enhancing robustness to channel order. Furthermore, we introduce channel similarity modeling, a pretraining task to preserve similarities between channels from the data space to the latent space to enhance the ability to capture CD. Extensive experiments demonstrate the efficacy of our method, achieving state-of-the-art performance on diverse datasets.

Primary Area: learning on time series and dynamical systems

Submission Number: 23977