Non-Stationarity in the Embedding Space of Time Series Foundation Models

Jinmyeong Choi, Brad Shook, Artur Dubrawski

Keywords: Non-stationary, Time Series Foundation Model

Abstract: Time series foundation models (TSFMs) are widely used as generic feature extractors, yet the notion of non‑stationarity in their embedding spaces remains poorly understood. Recent work often conflates non‑stationarity with distribution shift, blurring distinctions fundamental to classical time‑series analysis and long‑standing methodologies such as statistical process control (SPC). In SPC, non‑stationarity signals a process leaving a stable regime—via shifts in mean, variance, or emerging trends—and detecting such departures is central to quality monitoring and change‑point analysis. Motivated by this diagnostic tradition, we study how different forms of distributional non‑stationarity—mean shifts, variance changes, and linear trends—become linearly accessible in TSFM embedding spaces under controlled conditions. We further examine temporal non‑stationarity arising from persistence, which reflects violations of weak stationarity due to long‑memory or near‑unit‑root behavior rather than explicit distributional shifts. By sweeping shift strength and probing multiple TSFMs, we find that embedding‑space detectability of non‑stationarity degrades smoothly and that different models exhibit distinct, model‑specific failure modes.

Track: Research Track (max 4 pages)

Submission Number: 81