Bias-Variance Analysis of Multi-Step Loss Functions for Dynamical System Identification

Fabien Lionti, Nicolas Gutowski, Sébastien Aubin, Philippe Martinet

Abstract: System identification is a fundamental task in understanding and modeling dynamical systems, with extensive applications in engineering. Traditional statistical estimators for system identification rely on loss functions based on single-step predictions of model state variables. However, these approaches often lack robustness and reliability in real-world scenarios characterized by noisy and imperfect data. Recent advancements have introduced multi-step loss functions for autoregressive neural network predictions, leading to significant improvements in system identification performance. These loss functions are optimized via gradient descent, leveraging backpropagation through the numerically integrated neural network architecture. Despite their potential, the statistical and mathematical properties of these gradient estimators, such as bias, variance and robustness, remain underexplored. This paper examines the statistical and mathematical characteristics of multi-step loss function estimators in the context of dynamical system identification. We provide a theoretical foundation for the bias-variance decomposition of these loss functions, enabling the separation of error contributions from disturbances and deterministic model parameterization. Theoretical insights are validated and extended through empirical analysis, allowing an exploration of the bias-variance decomposition dynamics across the training phase. Our results demonstrate both theoretically and practically the influence of the contractive properties of the underlying dynamical system and the autoregressive prediction horizon on training stability. By bridging the theoretical and practical gap in the exploration of multi-step loss functions, this work contributes to the understanding and development of more robust and reliable methods for dynamical system identification involving gradient descent.

External IDs:dblp:conf/ijcnn/LiontiGAM25