Scaling Multimodal Planning: Using Experience and Informing Discrete Search

Zachary K. Kingston, Lydia E. Kavraki

Abstract: Robotic manipulation is inherently continuous, but typically has an underlying discrete structure, such as if an object is grasped. Many problems like these are <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">multimodal</i> , such as pick-and-place tasks where every object grasp and placement is a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">mode</i> . Multimodal problems require finding a sequence of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">transitions</i> between modes—for example, a particular sequence of object picks and placements. However, many multimodal planners fail to scale when motion planning is difficult (e.g., in clutter) or the task has a long horizon (e.g., rearrangement). This work presents solutions for multimodal scalability in both these areas. For motion planning, we present an experience-based planning framework <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">alef</small> which reuses experience from similar modes both online and from training data. For task satisfaction, we present a layered planning approach that uses a discrete <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">lead</i> to bias search toward useful mode transitions, informed by weights over mode transitions. Together, these contributions enable multimodal planners to tackle complex manipulation tasks that were previously infeasible or inefficient, and provide significant improvements in scenes with high-dimensional robots.