The Arrow of Time: What Tabular Foundation Models Miss in Time Series Forecasting

Andres Potapczynski, Ravi Kiran Selvam, Tatiana Konstantinova, Malcolm Wolff, Kin G. Olivares, Ruijun Ma, Michael W. Mahoney, Andrew Gordon Wilson, Boris N. Oreshkin, Dmitry Efimov

Keywords: Neural Forecasting, Time Series, PFN, Zero-shot, Foundational Model

TL;DR: We identified why tabular foundation models fail at time series forecasting (they ignore temporal order) and proposed architectural fixes like positional encodings that improve performance.

Abstract: In many time series forecasting settings, the target time series is accompanied by exogenous covariates, such as promotions and prices in retail demand; temperature in energy load; calendar and holiday indicators for traffic or sales; and grid load or fuel costs in electricity pricing. Ignoring these exogenous signals can substantially degrade forecasting accuracy, particularly when they drive spikes, discontinuities, or regime changes in the target series. Most current time series foundation models ignore exogenous covariates and make forecasts solely from the numerical time series history. Recent attempts to apply tabular foundation models to time series forecasting show promise but exhibit fundamental failure modes due to the absence of temporal inductive biases. In this paper, we provide a detailed characterization of these failure modes and propose targeted modifications to address them with empirical results.

Track: Research Track (max 4 pages)

Submission Number: 57