Integrating Kernel Methods and Deep Neural Networks for Solving PDEs

Carlos Mora, Amin Yousefpour, Shirin Hosseinmardi, Ramin Bostanabad

Keywords: Partial Differential Equations, Physics-informed Machine Learning, Neural Networks, Kernel Methods, Gaussian Processes

TL;DR: We propose a robust and efficient framework for integrating neural networks and Gaussian processes to solve PDEs

Abstract:

Physics-informed machine learning (PIML) has emerged as a promising alternative to conventional numerical methods for solving partial differential equations (PDEs).
PIML models are increasingly built via deep neural networks (NNs) whose performance is very sensitive to the NN's architecture, training settings, and loss function.
Motivated by this limitation, we introduce kernel-weighted Corrective Residuals (CoRes) to integrate the strengths of kernel methods and deep NNs for solving nonlinear PDE systems. To achieve this integration, we design a modular and robust framework which consistently outperforms competing methods in a broad range of benchmark problems. This performance improvement has a theoretical justification and is particularly attractive since we simplify the training process while negligibly increasing the inference costs.
Our studies also indicate that the proposed approach considerably decreases the sensitivity of NNs to factors such as random initialization, architecture type, and choice of optimizer.

Submission Number: 21