Learning Preconditions of Hybrid Force-Velocity Controllers for Contact-Rich Manipulation

Jacky Liang, Xianyi Cheng, Oliver Kroemer

Keywords: Contact-Rich Manipulation, Hybrid Force-Velocity Controllers, Precondition Learning

TL;DR: Plan with learned preconditions for Hybrid Force-Velocity Controllers to perform contact-rich manipulation tasks in constrained environments (e.g. shelves)

Abstract: Robots need to manipulate objects in constrained environments like shelves and cabinets when assisting humans in everyday settings like homes and offices. These constraints make manipulation difficult by reducing grasp accessibility, so robots need to use non-prehensile strategies that leverage object-environment contacts to perform manipulation tasks. To tackle the challenge of planning and controlling contact-rich behaviors in such settings, this work uses Hybrid Force-Velocity Controllers (HFVCs) as the skill representation and plans skill sequences with learned preconditions. While HFVCs naturally enable robust and compliant contact-rich behaviors, solvers that synthesize them have traditionally relied on precise object models and closed-loop feedback on object pose, which are difficult to obtain in constrained environments due to occlusions. We first relax HFVCs' need for precise models and feedback with our HFVC synthesis framework, then learn a point-cloud-based precondition function to classify where HFVC executions will still be successful despite modeling inaccuracies. Finally, we use the learned precondition in a search-based task planner to complete contact-rich manipulation tasks in a shelf domain. Our method achieves a task success rate of $73.2\%$, outperforming the $51.5\%$ achieved by a baseline without the learned precondition. While the precondition function is trained in simulation, it can also transfer to a real-world setup without further fine-tuning. See supplementary materials and videos at~\url{https://sites.google.com/view/constrained-manipulation/}.

Student First Author: yes

Website: https://sites.google.com/view/constrained-manipulation/