An Extensible Benchmark Suite for Learning to Simulate Physical Systems

Karl Otness, Arvi Gjoka, Joan Bruna, Daniele Panozzo, Benjamin Peherstorfer, Teseo Schneider, Denis Zorin

Keywords: Scientific Computing, Physics and ML, Numerical Integration, Physical Simulation

TL;DR: We introduce an extensible benchmark to evaluate data-driven physical simulation

Abstract: Simulating physical systems is a core component of scientific computing, encompassing a wide range of physical domains and applications. Recently, there has been a surge in data-driven methods to complement traditional numerical simulation methods, motivated by the opportunity to reduce computational costs and/or learn new physical models leveraging access to large collections of data. However, the diversity of problem settings and applications has led to a plethora of approaches, each one evaluated on a different setup and with different evaluation metrics. We introduce a set of benchmark problems to take a step towards unified benchmarks and evaluation protocols. We propose four representative physical systems, as well as a collection of both widely used classical time integrators and representative data-driven methods (kernel-based, MLP, CNN, nearest neighbors). Our framework allows evaluating objectively and systematically the stability, accuracy, and computational efficiency of data-driven methods. Additionally, it is configurable to permit adjustments for accommodating other learning tasks and for establishing a foundation for future developments in machine learning for scientific computing.

URL: https://archive.nyu.edu/handle/2451/63285

Contribution Process Agreement: Yes

Dataset Url:

Updated source code and information on use are available at: https://github.com/karlotness/nn-benchmark

An archival copy of source code, experiment results, and generated datasets are available for download from the NYU Faculty Digital Archive: https://archive.nyu.edu/handle/2451/63285

The data loaders implemented in the project source code can be used to load the stored datasets. For specific information on their structure and contents, see the supplementary materials accompanying our paper.

The script download.sh in the GitHub repository linked above can help download, verify, and reassemble the archived data files.

License:

The source code produced in this project is available under the MIT license. The text of this license is included in the LICENSE.txt file which accompanies the code.

Archived datasets, generated files and experiment results are made available under a Creative Commons Attribution 4.0 license (CC BY 4.0).

Author Statement: Yes

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/arxiv:2108.07799/code)