Object-centric Learning with Capsule Networks: A Survey

Fabio De Sousa Ribeiro, Kevin Duarte, Miles Everett, Georgios Leontidis, Mubarak Shah

Abstract:

Capsule networks emerged as a promising alternative to convolutional neural networks for learning objectcentric
representations. The idea is to explicitly model part-whole hierarchies by using groups of neurons
called capsules to encode visual entities, then learn the relationships between these entities dynamically from
data. However, a major hurdle for capsule network research has been the lack of a reliable point of reference
for understanding their foundational ideas and motivations. This survey provides a comprehensive and critical
overview of capsule networks, which aims to serve as a main point of reference going forward. To that
end, we introduce the fundamental concepts and motivations behind capsule networks, such as equivariant
inference. We then cover various technical advances in capsule routing algorithms as well as alternative geometric
and generative formulations. We provide a detailed explanation of how capsule networks relate to
the attention mechanism in Transformers and uncover non-trivial conceptual similarities between them in
the context of object-centric representation learning. We also review the extensive applications of capsule
networks in computer vision, video and motion, graph representation learning, natural language processing,
medical imaging, and many others. To conclude, we provide an in-depth discussion highlighting promising
directions for future work.