Regularizing Image Classification Neural Networks with Partial Differential Equations

Jungeun Kim, Seunghyun Hwang, Jeehyun Hwang, Kookjin Lee, Dongeun Lee, Noseong Park

Keywords: partial differential equations, image classification, differential equations

Abstract: Differential equations can be used to design neural networks. For instance, neural ordinary differential equations (neural ODEs) can be considered as a continuous generalization of residual networks. In this work, we present a novel partial differential equation (PDE)-based approach for image classification, where we learn both a PDE's governing equation for image classification and its solution approximated by our neural network. In other words, the knowledge contained by the learned governing equation can be injected into the neural network which approximates the PDE solution function. Owing to the recent advancement of learning PDEs, the presented novel concept, called PR-Net, can be implemented. Our method shows comparable (or better) accuracy and robustness for various datasets and tasks in comparison with neural ODEs and Isometric MobileNet V3. For the efficient nature of PR-Net, it is suitable to be deployed in resource-scarce environments, e.g., deploying instead of MobileNet.

One-sentence Summary: Use partial differential equations (PDEs) to regularize image classification neural networks