AltTS: Decoupling Autoregression and Cross-Variable Dependency via Alternating Optimization for Multivariate Time Series Forecasting

Zhihang Yuan, Zhiyuan Liu, Mahesh K. Marina

Keywords: multivariate time series, time series forecasting, alternating optimization, autoregression, cross-variable dependency

TL;DR: AltTS decouples autoregression and cross-variable dependency and trains them with alternating optimization, mitigating gradient entanglement and advancing multivariate time series forecasting.

Abstract:

Multivariate time series mix two qualitatively heterogeneous components: (i) consistent autoregressive dependencies within individual series, and (ii) intermittent cross-dimension interactions that are often spurious over long horizons.
Channel Dependent (CD) methods resolve the spatial complexity through sparse modeling or channel clustering, but the two components are modeled without distinction.
We show that training a single structure to capture both effects poses challenges for optimization, as the high-variance updates required to model cross-dimension relations contaminate the gradients needed for autoregression, leading to brittle learning and degraded long-horizon accuracy.
Motivated by this observation, we develop AltTS, a dual-path framework that explicitly decouples autoregression and cross-relation modeling.
In AltTS, the autoregression path is realized by a linear predictor and the cross-relation path by a Transformer with Cross-Relation Self-Attention (CRSA), while the two are coordinated through alternating optimization to isolate gradient noise and reduce cross-block interference.
Extensive experiments across multiple benchmarks demonstrate that AltTS consistently outperforms existing models, particularly on long-horizon forecasting tasks.
These results highlight that carefully designed optimization strategies, rather than increasingly complex architectures, can be the key to advancing multivariate time series forecasting.

Primary Area: learning on time series and dynamical systems

Submission Number: 20457