LIPS - Learning Industrial Physical Simulation benchmark suiteDownload PDF

Published: 17 Sept 2022, Last Modified: 23 May 2023NeurIPS 2022 Datasets and Benchmarks Readers: Everyone
Keywords: benchmark suite, physical simulations, surrogate model, industrial use case
TL;DR: This paper introduces a new benchmark suite "Learning Industrial Physical Simulations" (LIPS), whose purpose is to assess the quality of surrogate models for emulation of a physical system following various evaluation criteria categories
Abstract: Physical simulations are at the core of many critical industrial systems. However, today's physical simulators have some limitations such as computation time, dealing with missing or uncertain data, or even non-convergence for some feasible cases. Recently, the use of data-driven approaches to learn complex physical simulations has been considered as a promising approach to address those issues. However, this comes often at the cost of some accuracy which may hinder the industrial use. To drive this new research topic towards a better real-world applicability, we propose a new benchmark suite "Learning Industrial Physical Simulations"(LIPS) to meet the need of developing efficient, industrial application-oriented, augmented simulators. To define how to assess such benchmark performance, we propose a set of four generic categories of criteria. The proposed benchmark suite is a modular and configurable framework that can deal with different physical problems. To demonstrate this ability, we propose in this paper to investigate two distinct use-cases with different physical simulations, namely: the power grid and the pneumatic. For each use case, several benchmarks are described and assessed with existing models. None of the models perform well under all expected criteria, inviting the community to develop new industry-applicable solutions and possibly showcase their performance publicly upon online LIPS instance on Codabench.
Author Statement: Yes
Dataset Url:
License: Mozilla Public License Version 2.0
Supplementary Material: pdf
Contribution Process Agreement: Yes
In Person Attendance: Yes
22 Replies