Mobi-$\pi$: Mobilizing Your Robot Learning Policy

Jingyun Yang, Isabella Huang, Brandon Vu, Max Bajracharya, Rika Antonova, Jeannette Bohg

Keywords: Policy Mobilization, Mobile Manipulation, Robot Learning, Robot Perception

TL;DR: We propose novel metrics, tasks, visualization tools and methods for "policy mobilization", the problem of taking a non-mobile manipulation policy and finding a proper initial robot pose from which to execute it on a mobile platform.

Abstract: Learned visuomotor policies are capable of performing increasingly complex manipulation tasks. However, most of these policies are trained on data collected from limited robot positions and camera viewpoints. This leads to poor generalization to novel robot positions, which limits the use of these policies on mobile platforms, especially for precise tasks like pressing buttons or turning faucets. In this work, we formulate the "policy mobilization" problem: find a mobile robot base pose in a novel environment that is in distribution with respect to a manipulation policy trained on a limited set of camera viewpoints. Compared to retraining the policy itself to be more robust to unseen robot base pose initializations, policy mobilization decouples navigation from manipulation and thus does not require additional demonstrations. With that, our formulation is still compatible with any approach that improves manipulation policy robustness. To study policy mobilization, we introduce the Mobi-$\pi$ framework, which includes: (1) metrics that quantify the difficulty of mobilizing a given policy, (2) a suite of simulated mobile manipulation tasks based on RoboCasa to evaluate policy mobilization, (3) visualization tools for analysis, and (4) several baseline methods. We also propose a novel approach that bridges navigation and manipulation by optimizing the robot's base pose to align with an in-distribution base pose for a learned policy. Our approach utilizes a 3D Gaussian Splatting model for novel viewpoint synthesis, a score function to evaluate pose suitability, as well as sampling-based optimization to identify optimal robot poses. We show that our approach on average outperforms the best baseline by 7.65$\times$ in simulation and 2.38$\times$ in the real world, demonstrating its effectiveness for policy mobilization.

Submission Number: 501