Constraining Gaussian Processes to Systems of Linear Ordinary Differential Equations

Andreas Besginow, Markus Lange-Hegermann

Keywords: Gaussian Process, Ordinary Differential Equations, Machine Learning, Probabilistic Model

Abstract: Data in many applications follows systems of Ordinary Differential Equations (ODEs).This paper presents a novel algorithmic and symbolic construction for covariance functions of Gaussian Processes (GPs) with realizations strictly following a system of linear homogeneous ODEs with constant coefficients, which we call LODE-GPs. Introducing this strong inductive bias into a GP improves modelling of such data. Using smith normal form algorithms, a symbolic technique, we overcome two current restrictions in the state of the art: (1) the need for certain uniqueness conditions in the set of solutions, typically assumed in classical ODE solvers and their probabilistic counterparts, and (2) the restriction to controllable systems, typically assumed when encoding differential equations in covariance functions. We show the effectiveness of LODE-GPs in a number of experiments, for example learning physically interpretable parameters by maximizing the likelihood.

TL;DR: Gaussian processes that are constrained such that they strictly satisfy a given system of linear homogenous ordinary differential equations with constant coefficients.