Guided Reinforcement Learning for Robust Multi-Contact Loco-Manipulation

Jean Pierre Sleiman, Mayank Mittal, Marco Hutter

Keywords: Whole-body Loco-Manipulation, Reinforcement Learning, Legged Mobile Manipulators

TL;DR: This work proposes a task-agnostic RL formulation that leverages a single demonstration to train robust policies for complex multi-contact behaviors, such as traversing spring loaded doors.

Abstract:

Reinforcement learning (RL) has shown remarkable proficiency in developing robust control policies for contact-rich applications. However, it typically requires meticulous Markov Decision Process (MDP) designing tailored to each task and robotic platform. This work addresses this challenge by creating a systematic approach to behavior synthesis and control for multi-contact loco-manipulation.
We define a task-independent MDP formulation to learn robust RL policies using a single demonstration (per task) generated from a fast model-based trajectory optimization method. Our framework is validated on diverse real-world tasks, such as navigating spring-loaded doors and manipulating heavy dishwashers. The learned behaviors can handle dynamic uncertainties and external disturbances, showcasing recovery maneuvers, such as re-grasping objects during execution. Finally, we successfully transfer the policies to a real robot, demonstrating the approach's practical viability.

Website: https://leggedrobotics.github.io/guided-rl-locoma/

Student Paper: yes

Submission Number: 678