Robotic Manipulation Learning with Equivariant Descriptor Fields: Generative Modeling, Bi-equivariance, Steerability, and Locality

Jiwoo Kim, Hyunwoo Ryu, Jongeun Choi, Joohwan Seo, Nikhil Potu Surya Prakash, Ruolin Li, Roberto Horowitz

Keywords: Robotics, Manipulation, Robotic manipulation, Equivariance, SE(3), SO(3), Energy-based model, Lie group, Representation theory, Equivariant robotics, Roto-translation equivariance, End-to-end, Point clouds, Graph neural networks, Imitation learning, Learning from demonstration, Sample efficient, Few shot, Unseen object, Category-level manipulation, MCMC, Langevin dynamics

TL;DR: We introduce recently proposed Equivariant Descriptor Fields (EDFs), focusing on the four key model properties: generative modeling, bi-equivariance, steerable representation and locality

Abstract: Conventional end-to-end visual robotic manipulation learning methods often face challenges related to data inefficiency and limited generalizability. To mitigate these challenges, recent works have proposed incorporating equivariance into their designs. This paper presents a fresh perspective on the design principles of SE(3)-equivariant methods for end-to-end visual robotic manipulation learning. Specifically, we examine the recently introduced concept of Equivariant Descriptor Fields (EDFs), focusing on four key underlying principles: generative modeling, bi-equivariance, steerable representation, and locality. These principles enable EDFs in achieving impressive data efficiency and out-of-distribution generalizability, even in the absence of prior knowledge. By comparing EDFs with other contemporary equivariant methods based on the four criteria, this paper underscores the importance of these design principles and aims to establish a guiding framework for future research on SE(3)-equivariant robotic manipulation.