From Skills to TAMP: Learning Portable Symbolic Representations for Task and Motion Planning

Naman Shah, Benned Hedegaard, Yichen Wei, Ziyi Yang, Alper Ahmetoglu, Stefanie Tellex, George Konidaris

Keywords: learning abstractions for skills, task and motion planning, learning for TAMP

TL;DR: Learning spatial and visual predicates from pixels and poses yields portable symbolic abstractions that let robots compose skills via Task and motion planning.

Abstract:

To solve long-horizon tasks, robots must compose previously learned motor skills using task and motion planning (TAMP). However, TAMP presumes the existence of a symbolic task-level abstraction that is sound, complete, and refinable into motion plans.
Designing this representation by hand is brittle, and existing learning methods fail to guarantee the semantics required for TAMP. We present the first approach for \emph{learning portable symbolic representations} from pixels and poses that provably support TAMP.
Our method learns (i) object-centric visual predicates and (ii) generative relational spatial predicates from skill executions.
These predicates serve dually as binary classifiers over low-level states and as samplers for motion-level refinement.
We discuss preliminary experiments on two real-world robot platforms, demonstrating how our approach can learn reusable symbols.
In ongoing experiments, we intend to show how these symbols enable zero-shot synthesis of long-horizon plans across novel environments.

Submission Number: 18