MegaSurf: Scalable Large Scene Neural Surface Reconstruction

Yusen Wang, Kaixuan Zhou, Wenxiao Zhang, Chunxia Xiao

Abstract:

We present MegaSurf, a Neural Surface Reconstruction (NSR) framework designed to reconstruct 3D models of large scenes from aerial images. Many methods utilize geometry cues to overcome the shape-radiance ambiguity, which would produce large geometric errors. However, directly using inevitable imprecise geometric cues would lead to degradation in the reconstruction results, especially on large-scale scenes. To address this phenomenon, we propose a Learnable Geometric Guider (LG Guider) to learn a sampling field from reliable geometric cues. The LG Guider decides which position should fit the input radiance and can be continuously refined by rendering loss. Our MegaSurf uses a Divide-and-Conquer training strategy to address the synchronization issue between the Guider and the lagging NSR's radiance field. This strategy enables the Guider to transmit the information it carried to the radiance field without being disrupted by the gradients back-propagated from the lagging rendering loss at the early stage of training. Furthermore, we propose a Fast PatchMatch MVS module to derive the geometric cues in the planer regions that help overcome ambiguity.
Experiments on several aerial datasets show that MegaSurf can overcome ambiguity while preserving high-fidelity details. Compared to SOTA methods, MegaSurf achieves superior reconstruction accuracy of large scenes and boosts the acquisition of geometric cues more than four times.

Primary Subject Area: [Experience] Multimedia Applications

Secondary Subject Area: [Experience] Art and Culture

Relevance To Conference: As one of the most important multimedia processing technologies, 3D reconstruction technology is widely used in smart cities, cultural heritage protection, medicine, education, games and other fields. Neural surface reconstruction can not only obtain a real-world three-dimensional model from input images, but also can render a novel view, thus showing the potential to replace traditional reconstruction methods. However, most of the existing work is focused on small objects, and the reconstruction of large scenes is a necessary step to commercialize NSR. Our method proposes a new solution to the problem of quality degradation of NSR reconstruction in city-level scenes, so that NSR can maintain robustness and preserve realistic details.

Submission Number: 3323