Dynamics is what you need for time-series forecasting!

Alexis-Raja Brachet, Pierre-Yves Richard, CELINE HUDELOT

Keywords: dynamics, time-series forecasting, benchmark, transformers

TL;DR: Systemic and empirical study of the dynamics in time-series forecasting models.

Abstract: While deep learning is facing a model homogenization across modalities, the usual successful deep models are still challenged by simple ones in the time-series forecasting task. Indeed, our hypothesis is that the nature of this task needs models able to learn the underlying dynamics, which is not often the case. We propose to validate this hypothesis through both systemic and empirical studies. We develop an original $\texttt{PRO-DYN}$ nomenclature to analyze existing models through the lens of dynamics. Two observations thus emerge: **1.** under-performing architectures learn dynamics at most partially, **2.** the location of the dynamics block at the model end is of prime importance. We conduct extensive experiments to confirm our observations on a set of performance-varying models with diverse backbones. Results support the need to incorporate a learnable dynamics block and its use as the final predictor.

Primary Area: learning on time series and dynamical systems

Submission Number: 2138