GeoFormer: Learning Point Cloud Completion with Tri-Plane Integrated Transformer

Jinpeng Yu, Binbin Huang, Yuxuan Zhang, Huaxia Li, Xu Tang, Shenghua Gao

Abstract: Point cloud completion aims to recover accurate global geometry and preserve fine-grained local details from partial point clouds. Conventional methods typically predict unseen points directly from 3D point cloud coordinates or use self-projected multi-view depth maps to ease this task. However, these gray-scale depth maps cannot reach multi-view consistency, consequently restricting the performance. In this paper, we introduce a GeoFormer that simultaneously enhances the global geometric structure of the points and improves the local details. Specifically, we design a CCM Feature Enhanced Point Generator to integrate image features from multi-view consistent canonical coordinate maps (CCMs) and align them with pure point features, thereby enhancing the global geometry feature. Additionally, we employ the Multi-scale Geometry-aware Upsampler module to progressively enhance local details. This is achieved through cross attention between the multi-scale features extracted from the partial input and the features derived from previously estimated points. Extensive experiments on the PCN, ShapeNet-55/34, and KITTI benchmarks demonstrate that our GeoFormer outperforms recent methods, achieving the state-of-the-art performance. The code is ready and will be released soon.

Primary Subject Area: [Content] Media Interpretation

Secondary Subject Area: [Experience] Multimedia Applications

Relevance To Conference: Point clouds are a common representation of 3D data in multimedia applications, such as virtual reality, gaming, and 3D modeling. However, point clouds can often be incomplete due to limitations in data acquisition or processing. This work proposes a deep learning-based approach for completing missing points in point clouds, which can improve the accuracy and completeness of 3D data in multimedia applications. Additionally, the proposed method can be applied to multimodal data, such as combining point clouds with RGB or depth images, to further enhance the quality of 3D data. Overall, this work contributes to the advancement of multimedia/multimodal processing by providing a solution for improving the quality of 3D data in various applications.

Submission Number: 587