Compositional Interfaces for Compositional Generalization
Keywords: compositional generalization, modular architectures, generalist agents
TL;DR: end-to-end modular architecture demonstrates compositional generalization wrt observation, action and instruction spaces
Abstract: In this work, we study the effectiveness of a modular architecture for compositional generalization and transfer learning in the embodied agent setting. We develop an environment that allows us to independently vary perceptual modalities and action and task instructions, and use it to carefully analyze the agent's performance in these compositions. Our experiments demonstrate strong zero-shot performance on held-out combinations of perception, action, and instruction spaces; as well as fast adaptation to new perceptual spaces without the loss of performance.
Submission Number: 40