Are Time Series Foundation Models Ready for Zero-Shot Forecasting?

Yunkai Zhang, Qi Zeng, Yawen Zhang, Zhijie Xu, Ming Zheng, Chongyang Gao, Muyan Jiang, Zeyu Zheng

Keywords: time series foundation model, pca, out-of-distribution, zero-shot forecasting

TL;DR: We show PCA can also serve as a TSFM, achieving state-of-the-art forecasting results with reduced complexity.

Abstract: Recent successes of foundation models in various domains have spurred interest in time series foundation models (TSFMs), especially for zero-shot forecasting. We challenge the necessity of zero-shot forecasting, and demonstrate that a simpler model, PCA+Linear, can effectively serve as a TSFM. PCA+Linear uses Principal Component Analysis (PCA) as a universal feature extractor and a linear head trained specifically on each downstream dataset. Experiments show PCA+Linear achieves results competitive with state-of-the-art TSFMs. We further evaluate the robustness of transformer-encoder-based TSFMs on out-of-distribution data, highlighting the importance of the final linear layer in addition to the attention mechanism. Our findings also emphasize the effectiveness of diverse pretraining data over extensive datasets from limited sources.

Submission Number: 91