Generalized Representation for Generalized Dynamics Generation

Yichen Li, Zhiyi Li, Dinghuai Zhang, Brandon Y. Feng, Antonio Torralba

Keywords: dynamics generation

Abstract:

Digital twin worlds with realistic interactive dynamics presents a new opportunity to develop generalist embodied agents in scannable environments with complex physical behaviors. To this end, we present PepGen (Potential Energy Perspective for Generalized dynamics Generation), a framework that seamlessly integrates rigid body, articulated body, and soft body dynamics into a unified, geometry-5
agnostic system. PepGen operates from the governing principle that the potential energy for any stable physical system should be low. This fresh perspective allows us to treat the world as one holistic entity and infer underlying physical properties from simple motion observations. We extend classic elastodynamics by introducing directional stiffness to capture a broad spectrum of physical behaviors, covering soft elastic, articulated, and rigid body systems. We propose a specialized network to model the extended material property and employ a neural field to represent deformation in a geometry-agnostic manner. Extensive experiments demonstrate that PepGen robustly unifies diverse simulation paradigms, offering a versatile foundation for creating interactive virtual environments and training robotic agents in complex, dynamically rich scenarios.

Primary Area: generative models

Submission Number: 23434