Neural Context Flows for Learning Generalizable Dynamical Systems

Roussel Desmond Nzoyem, David A.W. Barton, Tom Deakin

Keywords: Neural ODE, Meta-Learning, Dynamical Systems, Scientific Machine Learning

TL;DR: We propose Neural Context Flow, a straightforward learning framework that allows dynamical systems to generalize to new parameter values.

Abstract: Neural Ordinary Differential Equations typically struggle to generalize to new dynamical behaviors created by parameter changes in the underlying system, even when the dynamics are close to previously seen behaviors. The issue gets worse when the changing parameters are unobserved, i.e., their value or influence is not directly measurable when collecting data. We introduce Neural Context Flow (NCF), a framework that encodes said unobserved parameters in a latent context vector as input to a vector field. NCFs leverage differentiability of the vector field with respect to the parameters, along with first-order Taylor expansion to allow any context vector to influence trajectories from other parameters. We validate our method and compare it to established Multi-Task and Meta-Learning alternatives, showing competitive performance in mean squared error for in-domain and out-of-distribution evaluation on the Lotka-Volterra, Glycolytic Oscillator, and Gray-Scott problems. This study holds practical implications for foundational models in science and related areas that benefit from conditional neural ODEs. Our code is openly available at \url{https://github.com/ddrous/ncflow}.

Submission Number: 13