Data-Efficient Inference of Neural Fluid Fields via SciML Foundation Model

Yuqiu Liu, Jingxuan Xu, Mauricio Soroco, Yunchao Wei, Wuyang Chen

Keywords: 3d reconstruction, neural scene representations, fluid dynamics, scientific machine learning, foundation model

TL;DR: This paper leverages scientific machine learning foundation models in 3D fluid reconstruction from sparse views, achieving data-efficient improvements in both quantitative and visual quality in real-world applications.

Abstract: Recent developments in 3D vision have enabled significant progress in inferring neural fluid fields and realistic rendering of fluid dynamics. However, these methods require dense captures of real-world flow, which demand specialized lab setups, making the process costly and challenging. Scientific machine learning (SciML) foundation models, pretrained on extensive simulations of partial differential equations (PDEs), encode rich multiphysics knowledge and thus provide promising sources of domain priors for inferring fluid fields. Nevertheless, the transferability of these foundation models to real-world vision problems remains largely underexplored. In this work, we demonstrate that SciML foundation models can significantly reduce the data costs of inferring real-world 3D fluid dynamics with improved generalization. Our method leverages strong forecasting capabilities and meaningful representations of SciML foundation models. We equip neural fluid fields with a novel collaborative training that utilizes augmented frames, and fluid features extracted by our foundation model. We demonstrate significant advancements in both quantitative metrics and visual quality over previous methods, improving 9-36% peak signal-to-noise ratio (PSNR) in future prediction with 25-50% reduction in the number of training frames, thereby showcasing the practical applicability of SciML foundation models in real-world fluid dynamics. We release our code at: https://github.com/delta-lab-ai/SciML-HY.

Submission Number: 345