Cross-View Geometric Collaboration for Generalizable Sparse View Neural Surface Reconstruction

Hang Yang, Le Hui, Jianjun Qian, Jian Yang, Yigong Zhang, Jin Xie

Abstract: Generalizable neural implicit surface reconstruction aims to recover accurate surfaces with sparse views from unseen scenes. Most existing methods suffer from severe incompleteness and inaccuracies in the case of reconstruction with large viewpoint variations, as significant perspective distortions across views lead to unreliable feature correspondence and geometry representations. In this paper, we propose a cross-view geometric collaboration framework for generalizable neural surface reconstruction, which exploits cross-view complementary geometric information to improve the accuracy and robustness of reconstruction from sparse views. Specifically, we propose a cross-view geometry complement module that utilizes the reliable geometric information of different views to refine geometric representations. In addition, we construct a distortion-robust patch-based consistency volume to provide supplementary geometric cues for uncertain regions. For the rendering process, we develop a cross-view geometry transformer to adaptively aggregate reliable cross-view point features by considering geometric context along the ray. Finally, we render per-view depth maps and fuse them to reconstruct the final surface. Extensive experimental results on the DTU, BlendedMVS, and Tanks and Temples datasets demonstrate the superior reconstruction quality and view-combination generalizability of our solution.

External IDs:doi:10.1145/3746027.3755785