Grounding Graph Network Simulators using Physical Sensor Observations

Jonas Linkerhägner, Niklas Freymuth, Paul Maria Scheikl, Franziska Mathis-Ullrich, Gerhard Neumann

Keywords: graph network simulators, deformable object simulation, point cloud, mesh

TL;DR: We ground Graph Network Simulators with physical sensor information to resolve uncertainties and improve long-term prediction quality.

Abstract: Physical simulations that accurately model reality are crucial for many engineering disciplines such as mechanical engineering and robotic motion planning. In recent years, learned Graph Network Simulators produced accurate mesh-based simulations while requiring only a fraction of the computational cost of traditional simulators. As these predictors have to simulate complex physical systems from only an initial state, they exhibit a high error accumulation for long-term predictions. In this work, we integrate sensory information to $\textit{ground}$ Graph Network Simulators on real world observations in the form of point clouds. The resulting model allows for accurate predictions over longer time horizons, even under uncertainties in the simulation, such as unknown material properties.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2302.11864/code)