Successor Representations Enable Emergent Compositional Instruction Following
Keywords: Robot Learning, Instruction Following, Compositional Generalization
TL;DR: Time-contrastive alignment over state and goal representations enables compositional generalization for goal-conditioned robot policies trained with behavioral cloning
Abstract: Behavioral cloning (BC) has seen widespread adoption in scalable robot learning pipelines. These methods struggle to perform compositional generalization, where a new out-of-distribution evaluation task can be viewed as a sequence of simpler in-distribution steps. We augment goal-conditioned BC methods with a temporal alignment loss that learns to associate present and future states. This approach is able to generalize to novel composite tasks specified as goal images or language instructions, without assuming any additional reward supervision or explicit subtask planning. We evaluate our approach across diverse tabletop robotic manipulation tasks, showing substantial improvements for tasks specified with either language or goal images.
Primary Area: applications to robotics, autonomy, planning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 5265
Loading