How Temporal Unrolling Supports Neural Physics Simulators

Björn Malte List, Liwei Chen, Kartik Bali, Nils Thuerey

Primary Area: applications to physical sciences (physics, chemistry, biology, etc.)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: neural simulators, unrolling, differentiable simulator, partial differential equation, fluids benchmark

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: Unrolling reduces distribution shift and provides long-term gradients; we disentangle these effects in a large-scale empirical study.

Abstract:

Unrolling training trajectories over time strongly influences the inference accuracy of neural network-augmented physics simulators. We analyze these effects by studying three variants of training neural networks on discrete ground truth trajectories. In addition to commonly used one-step setups and fully differentiable unrolling, we include a third, less widely used variant: unrolling without temporal gradients. Comparing networks trained with these three modalities makes it possible to disentangle the two dominant effects of unrolling, training distribution shift and long-term gradients. We present a detailed benchmark across physical systems, network sizes, network architectures, training setups, and test scenarios. It provides an empirical basis for our main findings: Fully differentiable setups perform best across most tests, yielding an improvement of 38% on average. Nevertheless, the accuracy of unrolling without temporal gradients comes comparatively close with 23%. These results motivate integrating non-differentiable numerical simulators into training setups even if full differentiability is unavailable. Furthermore, we empirically show that these behaviors are invariant to changes in the underlying physical system, the network architecture and size, and the numerical
scheme.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 2391