Accelerating Neural Differential Equations for Irregularly-Sampled Dynamical Systems Using Variational Formulation

Hongjue Zhao, Yuchen Wang, Hairong Qi, Jiajia Li, Lui Sha, Han Zhao, Huajie Shao

Keywords: Neural differential equations, irregularly-sampled dynamical systems, variational formulation, acceleration

TL;DR: We propose a novel approach based on variational formulation (VF) to accelerate the training of Neural ODEs for dynamical systems.

Abstract: Neural ODEs have exhibited remarkable capabilities in continuously modeling dynamical systems from observational data. However, existing training methods, often based on adaptive-step-size numerical ODE solvers, are time-consuming and may introduce additional errors. Despite recent attempts to address these issues, existing methods rely heavily on numerical ODE solvers and lack efficient solutions. In this work, we propose the Fast-VF Neural ODE, a novel approach based on variational formulation (VF) to accelerate the training of Neural ODEs for dynamical systems. To further mitigate the influence of oscillatory terms in the VF loss, we incorporate the Filon's method into our design. Extensive experimental results show that our method can accelerate the training of Neural ODEs by 10 $\times$ to 100 $\times$ compared to the baselines while achieving comparable accuracy in irregularly-sampled dynamical systems.

Submission Number: 56