Quaternion Equivariant Capsule Networks for 3D Point Clouds

Yongheng Zhao, Tolga Birdal, Jan Eric Lenssen, Emanuele Menegatti, Leonidas Guibas, Federico Tombari

TL;DR: Deep architectures for 3D point clouds that are equivariant to SO(3) rotations, as well as translations and permutations.

Abstract: We present a 3D capsule architecture for processing of point clouds that is equivariant with respect to the SO(3) rotation group, translation and permutation of the unordered input sets. The network operates on a sparse set of local reference frames, computed from an input point cloud and establishes end-to-end equivariance through a novel 3D quaternion group capsule layer, including an equivariant dynamic routing procedure. The capsule layer enables us to disentangle geometry from pose, paving the way for more informative descriptions and a structured latent space. In the process, we theoretically connect the process of dynamic routing between capsules to the well-known Weiszfeld algorithm, a scheme for solving iterative re-weighted least squares (IRLS) problems with provable convergence properties, enabling robust pose estimation between capsule layers. Due to the sparse equivariant quaternion capsules, our architecture allows joint object classification and orientation estimation, which we validate empirically on common benchmark datasets.

Code: http://s000.tinyupload.com/?file_id=93302435681489799761

Keywords: 3d, capsule networks, pointnet, quaternion, equivariant networks, rotations, local reference frame

Community Implementations: [ 3 code implementations](https://www.catalyzex.com/paper/arxiv:1912.12098/code)