LIPS - Learning Industrial Physical Simulation benchmark suite

Milad Leyli-abadi, Antoine Marot, Jérôme Picault, David Danan, Mouadh Yagoubi, Benjamin Donnot, Seif-Eddine Attoui, Pavel Dimitrov, Asma Farjallah, Clement Etienam

Keywords: benchmark suite, physical simulations, surrogate model, industrial use case

TL;DR: This paper introduces a new benchmark suite "Learning Industrial Physical Simulations" (LIPS), whose purpose is to assess the quality of surrogate models for emulation of a physical system following various evaluation criteria categories

Abstract: Physical simulations are at the core of many critical industrial systems. However, today's physical simulators have some limitations such as computation time, dealing with missing or uncertain data, or even non-convergence for some feasible cases. Recently, the use of data-driven approaches to learn complex physical simulations has been considered as a promising approach to address those issues. However, this comes often at the cost of some accuracy which may hinder the industrial use. To drive this new research topic towards a better real-world applicability, we propose a new benchmark suite "Learning Industrial Physical Simulations"(LIPS) to meet the need of developing efficient, industrial application-oriented, augmented simulators. To define how to assess such benchmark performance, we propose a set of four generic categories of criteria. The proposed benchmark suite is a modular and configurable framework that can deal with different physical problems. To demonstrate this ability, we propose in this paper to investigate two distinct use-cases with different physical simulations, namely: the power grid and the pneumatic. For each use case, several benchmarks are described and assessed with existing models. None of the models perform well under all expected criteria, inviting the community to develop new industry-applicable solutions and possibly showcase their performance publicly upon online LIPS instance on Codabench.

Author Statement: Yes

URL: https://github.com/IRT-SystemX/LIPS

Dataset Url: https://github.com/IRT-SystemX/LIPS

License: Mozilla Public License Version 2.0

Contribution Process Agreement: Yes

In Person Attendance: Yes