GIFT-Eval: A Benchmark for General Time Series Forecasting Model Evaluation
Keywords: benchmark, time series forecasting, foundation models, forecasting, univariate forecasting, multivariate forecasting, pretraining data, deep learning, statistical models, foundation models, dataset
TL;DR: We introduce a new benchmark designed specifically for time series foundation models, featuring diverse time series characteristics and structured with pretrain and train\test splits that avoids leakage.
Abstract: Time series foundation models excel in zero-shot forecasting, handling diverse tasks without explicit training. However, the advancement of these models has been hindered by the lack of comprehensive benchmarks. To address this gap, we introduce the **G**eneral T**I**me Series **F**orecas**T**ing Model **Eval**uation, **GIFT-EVAL**, a pioneering benchmark aimed at promoting evaluation across diverse datasets. GIFT-EVAL encompasses 28 datasets over 144,000 time series and 177 million data points, spanning seven domains, 10 frequencies, multivariate inputs, and prediction lengths ranging from short to long-term forecasts. To facilitate the effective pretraining and evaluation of foundation models, we also provide a non-leaking pretraining dataset containing approximately 230 billion data points. Additionally, we provide a comprehensive analysis of 20 baselines, which includes statistical models, deep learning models, and foundation models. We discuss each model in the context of various benchmark characteristics and offer a qualitative analysis that spans both deep learning and foundation models. We believe the insights from this analysis, along with access to this new standard zero-shot time series forecasting benchmark, will guide future developments in time series foundation models.
Primary Area: datasets and benchmarks
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 8531
Loading