Successes and Limitations of Object-centric Models at Compositional Generalisation

Milton L. Montero, Jeffrey Bowers, Gaurav Malhotra

Keywords: Compositional Generalisation, Object-Centric Models, Representation Learning, Generative Models

TL;DR: Object centric models exhibit strong compositional generalisation capabilities.

Abstract: In recent years, it has been shown empirically that standard disentangled latent variable models do not support robust compositional learning in the visual do- main. Indeed, in spite of being designed with the goal of factorising datasets into their constituent factors of variations, disentangled models show extremely limited compositional generalisation capabilities. On the other hand, object-centric architectures have shown promising compositional skills, albeit these have 1) not been extensively tested and 2) experiments have been limited to scene composition — where models must generalise to novel combinations of objects in a visual scene instead of novel combinations of object properties. In this work, we show that these compositional generalisation skills extend to this later setting. Furthermore, we present evidence pointing to the source of these skills and how they can be improved through careful training. Finally, we point to one important limitation that still exists which suggests new directions of research.

Submission Number: 24