Interpolating Neural Network-Tensor Decomposition (INN-TD): a scalable and interpretable approach for large-scale physics-based problems

Jiachen Guo, Xiaoyu Xie, Chanwook Park, Hantao Zhang, Matthew J. Politis, Gino Domel, Wing Kam Liu

TL;DR: Novel neural networks based on interpolation theory from numerical analysis that has faster training/solving speed and better accuracy for modeling high-dimensional large-scale physics-based PDEs

Abstract: Deep learning has been extensively employed as a powerful function approximator for modeling physics-based problems described by partial differential equations (PDEs). Despite their popularity, standard deep learning models often demand prohibitively large computational resources and yield limited accuracy when scaling to large-scale, high-dimensional physical problems. Their black-box nature further hinders their application in industrial problems where interpretability and high precision are critical. To overcome these challenges, this paper introduces Interpolating Neural Network-Tensor Decomposition (INN-TD), a scalable and interpretable framework that has the merits of both machine learning and finite element methods for modeling large-scale physical systems. By integrating locally supported interpolation functions from finite element into the network architecture, INN-TD achieves a sparse learning structure with enhanced accuracy, faster training/solving speed, and reduced memory footprint. This makes it particularly effective for tackling large-scale high-dimensional parametric PDEs in training, solving, and inverse optimization tasks in physical problems where high precision is required.

Lay Summary: Standard deep learning models struggle with large-scale physical simulations described by partial differential equations (PDEs). They often demand prohibitive computational power, yield limited accuracy for complex, high-dimensional scenarios, and their "black-box" nature makes them difficult to trust for critical industrial problems requiring interpretability and high precision. We developed the Interpolating Neural Network-Tensor Decomposition (INN-TD) framework, inspired by traditional finite element methods. This approach achieves enhanced accuracy, significantly faster training and solving speeds, and a reduced memory footprint. Our framework empowers more effective and reliable modeling of large-scale, high-dimensional parametric PDEs, crucial for advancing training, solving, and inverse optimization tasks in physical systems where high precision and interpretability are paramount.

Primary Area: Applications->Chemistry, Physics, and Earth Sciences

Keywords: Scientific Machine Learning; Large-scale simulation; Partial Differential Equations (PDEs); Numerical Analysis; Tensor decomposition

Submission Number: 13975