FATE: Focal-modulated Attention Encoder for Multivariate Time-series Forecasting

ICLR 2026 Conference Submission18382 Authors

Keywords: Time-Series Forecasting, Focal Modulation, Spatiotemporal Modeling

TL;DR: FATE introduces a tensorized focal attention mechanism that improves long-horizon multivariate time-series forecasting and outperforms state-of-the-art models on seven real-world datasets.

Abstract: Accurate multivariate time-series forecasting is crucial for understanding and mitigating the effects of climate change, as reliable long-horizon predictions support effective monitoring and informed decision-making. Existing neural approaches, ranging from CNNs and RNNs to attention-based Transformers, have achieved notable progress. Yet, they often suffer from two key limitations: difficulty in capturing hierarchical spatiotemporal dependencies and computational inefficiencies when scaling to high-dimensional meteorological data. We propose \fate (Focal-modulated Attention Encoder), a new Transformer architecture tailored for robust multivariate time-series forecasting. \fate introduces a tensorized focal modulation mechanism that enhances spatiotemporal dependency modeling while maintaining scalability. To improve interpretability, we further design dual modulation scores that identify critical environmental features driving the forecasts. Comprehensive experiments on seven diverse real-world datasets, including benchmark energy, traffic, and large-scale climate datasets, demonstrate that \fate consistently surpasses state-of-the-art methods, particularly on long-horizon and high-variability settings. Extensive ablations confirm the generalization ability of \fate across heterogeneous forecasting tasks. To foster reproducibility and future research, we will release the full implementation.

Primary Area: learning on time series and dynamical systems

Submission Number: 18382