Generalized One-Shot Transfer Learning of Linear Ordinary and Partial Differential Equations

Published: 31 Oct 2023, Last Modified: 10 Nov 2023DLDE III PosterEveryoneRevisionsBibTeX
Keywords: transfer learning, one-shot, multi-headed training, differential equations, physics-informed neural networks, closed-form formulas
TL;DR: This paper presents a method for one-shot transfer learning with physics-informed neural networks (PINNs) on linear differential equations, reducing computational costs and enabling rapid solutions transfer across different systems.
Abstract: We present a generalizable methodology to perform "one-shot" transfer learning on systems of linear ordinary and partial differential equations using physics informed neural networks (PINNs). PINNS have attracted researchers as an avenue through which both data and studied physical constraints can be leveraged in learning solutions to differential equations. Despite their benefits, PINNs are currently limited by the computational costs needed to train such networks on different but related tasks. Transfer learning addresses this drawback. In this work, we present a generalizable methodology to perform "one-shot" transfer learning on linear systems of equations. First, we describe a process to train PINNs on equations with varying conditions across multiple "heads". Second, we show how this multi-headed training process can be used to yield a latent space representation of a particular differential equation form. Third, we derive closed-form formulas, which represent generalized network weights that minimize the loss function. Finally, we demonstrate how the learned latent representation and derived network weights can be utilized to instantaneously transfer learn solutions to equations, demonstrating the ability to quickly solve many systems of equations in a variety of environments.
Submission Number: 9