TempoPFN: Synthetic Pre-training of Linear RNNs for Zero-shot Time Series Forecasting

ICLR 2026 Conference Submission16687 Authors

Keywords: time series forecasting, RNNs, synthetic data, mamba, linear rnn

Abstract: Foundation models for zero-shot time series forecasting face challenges in efficient long-horizon prediction and reproducibility, with existing synthetic-only approaches underperforming on challenging benchmarks. This paper presents TempoPFN, a univariate time series foundation model based on linear Recurrent Neural Networks (RNNs) pre-trained exclusively on synthetic data. The model uses a GatedDeltaProduct architecture with state-weaving for fully parallelizable training across sequence lengths, eliminating the need for windowing or summarization techniques while maintaining robust temporal state-tracking. Our comprehensive synthetic data pipeline unifies diverse generators including stochastic differential equations, Gaussian processes, and audio synthesis with novel augmentations such as time-varying TSMixup, differentiation, and integration. In zero-shot evaluations on the Gift-Eval benchmark, TempoPFN achieves state-of-the-art performance, matching models trained on real-world data while being significantly more efficient than existing baselines. We open-source our complete data generation pipeline and training code.

Primary Area: learning on time series and dynamical systems

Submission Number: 16687