Continuously Improving Mobile Manipulation with Autonomous Real-World RL

Russell Mendonca, Deepak Pathak

Keywords: Continual RL, exploration, legged mobile manipulation

Abstract:

To build generalist robots capable of executing a wide array of tasks across diverse environments, robots must be endowed with the ability to engage directly with the real world to acquire and refine skills without extensive instrumentation or human supervision. This work presents a fully autonomous real-world reinforcement learning framework for mobile manipulation that can both independently gather data and refine policies through accumulated experience in the real world. It has several key components:
1) automated data collection strategies by guiding the robot’s exploration toward object interactions,
2) using goal cycles for real world RL such that the robot changes goals once it has made sufficient progress, where the different goals serve as resets for one another,
3) efficient control by leveraging basic task knowledge present in behavior priors in conjunction with policy learning and
4) formulating generic rewards that combine human-interpretable semantic information with low-level, fine-grained state information.
We demonstrate our approach on Boston Dynamics Spot robots in continually improving performance on a set of four challenging mobile manipulation tasks and show that this enables competent policy learning, obtaining an average success rate of 80% across tasks, a 3-4 times improvement over existing approaches.

Submission Number: 7