Capturing the Hybrid Dynamics of Planar Pushing in RL with Multimodal Categorical Exploration
Keywords: Planar Pushing, Hybrid Dynamics, Reinforcement Learning
TL;DR: We propose a multimodal exploration approach through categorical distributions to capture the hybrid dynamics of planar pushing, which enables the learning of RL policies for arbitrary initial and target object poses.
Abstract: Planar pushing is a hybrid dynamics system due to the different possible contact interaction modes between the robot and the object, such as sticking, sliding, and separation. Previous Reinforcement Learning (RL) literature addressing the planar pushing task achieves low accuracy, non-smooth trajectories, and only simple motions, i.e. without orientation of the manipulated object. We conjecture that previously used unimodal exploration strategies fail to capture the inherent hybrid dynamics of the task. In this paper, we incorporate the hybrid dynamics into an RL framework by proposing a multimodal exploration approach through categorical distributions, which enables us to train planar pushing RL policies for arbitrary initial and target object poses, i.e. positions and orientations, and with improved accuracy. We show that the learned policies are robust to external disturbances, scalable to tasks with multiple pushers, and exhibit smooth pushing trajectories. Furthermore, we validate the transferability of the policies, trained entirely in simulation, to a physical robot hardware using the KUKA iiwa robot arm. See our supplemental video: https://youtu.be/vTdva1mgrk4.
Submission Number: 8
Loading