Instance-wise Knowledge Enhancement for 3D Instance Segmentation
Keywords: Computer Vision, 3D Instance Segmentation
Abstract: Recent 3D Instance Segmentation methods typically follow a similar paradigm; they encode hundreds of instance-wise candidates with instance-specific information in various ways and refine them into final masks. However, they have yet to fully explore the benefit of these candidates. They overlook the valuable cues encoded in multiple candidates that represent different parts of the same instance, resulting in fragmented instance masks. Also, they often fail to capture the precise spatial range of complex 3D instances, primarily due to inherent fuzzy noises from sparse and unordered point clouds. In this work, to address these challenges, we propose IKEA, a novel instance-wise knowledge enhancement approach. We first introduce an Instance-wise Knowledge Aggregation to associate scattered single instance details by optimizing correlations among candidates representing the same instance. Moreover, we present Instance-wise Structural Guidance to enhance the spatial understanding of candidates using structural cues from ambiguity-reduced features. Here, we utilize a simple yet effective truncated singular value decomposition algorithm to minimize inherent noises of 3D features. Finally, our instance-wise features are now highly informative for real-world 3D instances. In our extensive experiments on large-scale benchmarks, ScanNetV2, ScanNet200, S3DIS, and STPLS3D, IKEA outperforms existing works. We also demonstrate the effectiveness of our modules based on both kernel and transformer architectures.
Primary Area: applications to computer vision, audio, language, and other modalities
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 7351
Loading