Enforcing Physical Constraints in Neural Neural Networks through Differentiable PDE Layer

Chiyu "Max" Jiang, Karthik Kashinath, Prabhat, Philip Marcus

Abstract:

Recent studies at the intersection of physics and deep learning have illustrated successes in the application of deep neural networks to partially or fully replace costly physics simulations. Enforcing physical constraints to solutions generated
by neural networks remains a challenge, yet it is essential to the accuracy and trustworthiness of such model predictions. Many systems in the physical sciences are governed by Partial Differential Equations (PDEs). Enforcing these as hard
constraints, we show, are inefficient in conventional frameworks due to the high dimensionality of the generated fields. To this end, we propose the use of a novel differentiable spectral projection layer for neural networks that efficiently enforces
spatial PDE constraints using spectral methods, yet is fully differentiable, allowing for its use as a layer in neural networks that supports end-to-end training. We show that its computational cost is cheaper than a regular convolution layer. We apply it to
an important class of physical systems – incompressible turbulent flows, where the divergence-free PDE constraint is required. We train a 3D Conditional Generative Adversarial Network (CGAN) for turbulent flow super-resolution efficiently, whilst
guaranteeing the spatial PDE constraint of zero divergence. Furthermore, our empirical results show that the model produces realistic flow fields with more accurate flow statistics when trained with hard constraints imposed via the proposed
novel differentiable spectral projection layer, as compared to soft constrained and unconstrained counterparts.

Keywords: PDE, Hard Constraints, Turbulence, Super-Resolution, Spectral Methods

TL;DR: A novel way of enforcing hard linear constraints within a convolutional neural network using a differentiable PDE layer.