Learning Ordinary Differential Equations with the Line Integral Loss Function

Albert Johannessen

Keywords: deep learning, neural network, ode, optimization, time-series, line integral

TL;DR: A new loss function based on line integrals for training neural networks on ODEs

Abstract: A new training method for learning representations of dynamical systems with neural networks is derived using a loss function based on line integrals from vector calculus. The new training method is shown to learn the direction part of an ODE vector field with more accuracy and faster convergence compared to traditional methods. The learned direction can then be combined with another model that learns the magnitude explicitly to decouple the learning process of an ODE into two separate easier problems. It can also be used as a feature generator for time-series classification problems, performing well on motion classification of dynamical systems. The new method does however have multiple limitations that overall make the method less generalizable and only suited for some specific type of problems.