Accuracy Law for the Future of Deep Time Series Forecasting

Yuxuan Wang, Haixu Wu, Yuezhou Ma, Yuchen Fang, Ziyi Zhang, Yong Liu, Shiyu Wang, Zhou Ye, Yang Xiang, Jianmin Wang, Mingsheng Long

Keywords: deep learning, machine learning, time series

TL;DR: This paper discovers the accuracy law, which describes inherent relationship between time series complexity and best forecast performance achieved by SOTA models.

Abstract: Deep time series forecasting has emerged as a booming direction in recent years. Despite the exponential growth of community interests, researchers are sometimes confused about the direction of their efforts due to minor improvements on standard benchmarks. In this paper, we notice that, unlike image recognition, whose well-acknowledged and realizable goal is 100\% accuracy, time series forecasting inherently faces a non-zero error lower bound due to its partially observable and uncertain nature. To pinpoint the research objective and release researchers from saturated tasks, this paper focuses on a fundamental question: how to estimate the performance upper bound of deep time series forecasting? Going beyond classical series-wise predictability metrics, e.g., ADF test, we realize that the forecasting performance is highly related to window-wise properties because of the sequence-to-sequence forecasting paradigm of deep time series models. In this paper, we delve into univariate time series forecasting, which is a prevalent forecasting paradigm spanning traditional statistical models to advanced time series foundation models. Based on rigorous statistical tests of over 4700 newly trained deep forecasters, we discover a significant exponential relationship between the minimum forecasting error of deep models and the complexity of window-wise series patterns, which is termed the complexity law. The proposed complexity law successfully guides us to identify saturated tasks from widely used benchmarks and derives an effective training strategy for large time series models, offering valuable insights for future research.

Primary Area: learning on time series and dynamical systems

Submission Number: 2099