Variational multiple shooting for Bayesian ODEs with Gaussian processes

pashupati rajaram hegde, Cagatay Yildiz, Harri Lähdesmäki, Samuel Kaski, Markus Heinonen

Keywords: Gaussian processes, ordinary differential equations, dynamical systems, Bayesian inference

TL;DR: We introduce a method for efficiently performing Bayesian inference on unknown ODEs using Gaussian processes

Abstract: Recent machine learning advances have proposed black-box estimation of $\textit{unknown continuous-time system dynamics}$ directly from data. However, earlier works are based on approximative solutions or point estimates. We propose a novel Bayesian nonparametric model that uses Gaussian processes to infer posteriors of unknown ODE systems directly from data. We derive sparse variational inference with decoupled functional sampling to represent vector field posteriors. We also introduce a probabilistic shooting augmentation to enable efficient inference from arbitrarily long trajectories. The method demonstrates the benefit of computing vector field posteriors, with predictive uncertainty scores outperforming alternative methods on multiple ODE learning tasks.