Canonical Capsules:Self-Supervised Capsules in Canonical Pose
Overview of Supplementary Material
- The teaser video which briefly summarizes our framework.
- The supplementary appendix which provides architectural details and additional results.
- Qualitative results of canonicalization which show the stability of canonicalization.
- Qualitative results of reconstruction and decomposition in unaligned setup.
Teaser Video
TL;DR: A self-supervised capsule architecture that canonicalizes data while simultaneously decomposing point clouds into parts to perform unsupervised representation learning.
Abstract: We propose an unsupervised capsule architecture for 3D point clouds. We compute capsule decompositions of objects through permutation-equivariant attention, and self-supervise the process by training with pairs of randomly rotated objects. Our key idea is to aggregate the attention masks into semantic keypoints, and use these to supervise a decomposition that satisfies the capsule invariance/equivariance properties. This not only enables the training of a semantically consistent decomposition, but also allows us to learn a canonicalization operation that enables object-centric reasoning. To train our neural network we require neither classification labels nor manually-aligned training datasets. Yet, by learning an object-centric representation in a self-supervised manner, our method outperforms the state-of-the-art on 3D point cloud reconstruction, canonicalization, and unsupervised classification.
Supplementary Appendix
Architectural details, additional ablation studies, and qualitative results for the aligned setup are available in supplementary appendix. For more details, Click the image below to access the PDF
Code
We provide [code in the accompanied subfolder]. Please see [README.md] for detailed instructions regarding the code.
Canonicalization
Below are example videos demonstrating the quality of canonicalization. Our method achieves more stable canonicalization compared to Compass -- shown by the chairs and airplanes being well-aligned despite appearance changes.
| Input | Aligned by Ours | Aligned by Compass | Input | Aligned by Ours | Aligned by Compass |
Reconstruction and Decomposition
We show qualitative highlights, where we decompose 3D point clouds and auto-encode them using Canonical Capsules. We color each Canonical Capsule with a unique colour, and similarly color "patches" from the reconstruction heads of 3D-PointCapsNet and AtlasNetV2. Canonical Capsules provide semantically consistent decomposition that is aligned in the canonical frame, leading to improved reconstruction quality and unsupervised classification performance.