Generalized Robot Dynamics Learning and Gen2Real Transfer

Dengpeng Xing, Yiming Yang, Zechang Wang, Jiale Li, Bo Xu

Abstract: Acquiring dynamics is critical for robot learning and is fundamental to planning and control. This paper concerns two fundamental questions: How can we learn a model that covers massive, diverse robot dynamics? Can we construct a model that lifts the data-collection pain and domain expertise required for building specific robot models? We learn the dynamics involved in a dataset containing a large number of serial articulated robots and propose a new concept, “Gen2Real”, to transfer simulated, generalized models to physical, specific robots. We generate a large-scale dataset by randomizing dynamics parameters, topology configurations, and model dimensions, which, in sequence, correspond to different properties, connections, and numbers of robot links. A structure modified from the generative pre-trained transformer is applied to approximate the dynamics of massive heterogeneous robots. In Gen2Real, we transfer the pre-trained model to a target robot using distillation, for the sake of real-time computation. The results demonstrate the superiority of the proposed method in terms of its accuracy in learning a tremendous amount of robot dynamics and its generality to transfer to different robots.