Learning SO(3)-Invariant Correspondence via Point-wise Local Shape Transform
Keywords: Point cloud understanding, 3D dense correspondence, SO(3)-invariance, Part label transfer
TL;DR: We present a novel method to establish SO(3)-invariant dense correspondences between two different 3D instances of the same category, even under arbitrary rotation transformations.
Abstract: Establishing accurate dense 3D correspondences between diverse shapes stands as a pivotal challenge with profound implications for computer vision and robotics. However, existing self-supervised methods for this problem assume perfect input shape alignment, restricting their real-world applicability. In this work, we introduce a novel self-supervised SO(3)-invariant 3D correspondence learner, dubbed LSTNet, that learns to establish dense correspondences between shapes even under challenging intra-class variations. Specifically, LSTNet learns to dynamically formulate an SO(3)-invariant local shape transform for each point, which maps the SO(3)-equivariant global shape descriptor of the input shape to a local shape descriptor. These local shape descriptors are provided as inputs to our decoder to facilitate point cloud self- and cross-reconstruction. Our proposed self-supervised training pipeline encourages semantically corresponding points from different shape instances to be mapped to similar local shape descriptors, enabling LSTNet to establish the dense point-wise correspondences. LSTNet demonstrates state-of-the-art performances on 3D semantic keypoint transfer and part segmentation label transfer given arbitrarily rotated point cloud pairs, outperforming existing methods by significant margins.
Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 1647
Loading