TS-Arena: A Live Forecast Pre-Registration Platform for Leakage-Free Evaluation of Time Series Foundation Models

Marcel Meyer, Sascha Kaltenpoth, Henrik Albers, Kevin Zalipski, Oliver Mueller

Presentation Attendance: Yes, we will present in-person

Keywords: time series foundation models, time series benchmark, live benchmark

TL;DR: TS-Arena is a live benchmarking platform that ensures leakage-free evaluation of time series foundation models by requiring forecasts to be pre-registered before ground-truth data physically exists.

Abstract: Time Series Foundation Models (TSFMs) are transforming forecasting, yet evaluating them on historical data is increasingly compromised by train-test sample overlaps and temporal overlaps between correlated training and test series. This paper presents TS-Arena, an infrastructure designed to benchmark TSFMs by transitioning from retrospective historical testing to an environment of continuous, prospective evaluation. Our core contribution is a strict Forecast Pre-Registration Protocol (FPRP): models must submit predictions before the ground-truth data physically exists, making test-set contamination impossible by design. The platform relies on a modular microservice architecture that ingests real-time data streams and orchestrates containerized model submissions under enforced registration windows. By combining pre-registration with continuous evaluation rounds, TS-Arena aims to prevent both direct and indirect information leakage while enabling fast, ongoing model comparison. First results from simulating TS-Arena over one year of energy time series (e.g. renewable energy generation, electricity prices, district heating cogeneration) demonstrate the viability and discriminative power of leakage-free live evaluation. The live forecasting benchmark is available at https://ts-arena.live/.

Track: Research Track (max 4 pages)

Submission Number: 64