Temporal Representation Alignment: Successor Features Enable Emergent Compositionality in Robot Instruction Following

Vivek Myers, Bill Zheng, Anca Dragan, Kuan Fang, Sergey Levine

Keywords: Robotics, Representation Learning

Abstract:

Effective task representations should facilitate compositionality, such
that after learning a variety of basic tasks, an agent can perform
compound tasks consisting of multiple steps simply by composing the
representations of the constituent steps together. While this is
conceptually simple and appealing, it is not clear how to automatically
learn representations that enable this sort of compositionality. We show
that learning to associate the representations of current and future
states with a temporal alignment loss can improve compositional
generalization, even in the absence of any explicit subtask planning or
reinforcement learning. We evaluate our approach across diverse robotic
manipulation tasks as well as in simulation, showing substantial
improvements for tasks specified with either language or goal images.

Primary Area: Reinforcement learning (e.g., decision and control, planning, hierarchical RL, robotics)

Submission Number: 20943