Deep Kernel Learning of Nonlinear Latent Force Models

Published: 30 Oct 2024, Last Modified: 30 Oct 2024Accepted by TMLREveryoneRevisionsBibTeXCC BY 4.0
Abstract: Scientific processes are often modelled by sets of differential equations. As datasets grow, individually fitting these models and quantifying their uncertainties becomes a computationally challenging task. Latent force models offer a mathematically-grounded balance between data-driven and mechanistic inference in such dynamical systems, whilst accounting for stochasticity in observations and parameters. However, the required derivation and computation of the posterior kernel terms over a low-dimensional latent force is rarely tractable, requiring approximations for complex scenarios such as nonlinear dynamics. In this paper, we overcome this issue by posing the problem as learning the solution operator itself to a class of latent force models, thereby improving the scalability of these models. This is achieved by employing a deep kernel along with a meta-learned embedding of the output functions. Finally, we demonstrate the ability to extrapolate a solution operator trained on simulations to real experimental datasets, as well as scaling to large datasets.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: This revision is the camera-ready version. Previous changes: - Abstract updated to mention that our approach also leads to superior performance. - Table width has been fixed. - Font sizes on figures increased. - New ablation DKLFM-b - We have significantly reworded the section on the deep kernel and task representation to improve clarity following comments. - We have clarified how the periodic kernel is used in the LV tasks. Different LV tasks are, as you suggest, sampled by different initialisations, growth and decay rates. - The relationship between $h$ and $f$ is illustrated in Eq 4 of the revised manuscript. The function $h$ and its derivatives/partial derivatives are related according to the LFM’s differential equation, which involves another function, $f$, which is the latent force. - We appreciate that it is not made clear. We have updated Section 3.1 to make how we are observing latent force data clearer. Note that simulation of a LFM (the ``forward’’ direction) is typically easier than the inverse problem (i.e. fitting the LFM), since it just involves sampling and solving, whereas fitting requires gradients and optimising. - We have now defined $h$ properly in the text, with domain & image. $K_{ff}$, and $K_{hf}$ are defined in the definition of the joint in Section 3.1. We have extended the definition just after Equation 8 to define the domains. - We have introduced the first ODE from our experiments in Section 3.1 as an example to motivate our formulation. We hope that this makes our goals much clearer.
Assigned Action Editor: ~Alberto_Bietti1
Submission Number: 2538
Loading