Mesh Splatting for End-to-end Multiview Surface Reconstruction

Ruiqi Zhang, JiachengWU, Jie Chen

Keywords: Multi-View Stereo, Geometry Processing, Meshe Synthesis

TL;DR: We soften meshes into differentiable layers, increasing their 3D receptive field and enabling the learning of intricate geometry via volumetric rendering.

Abstract: Surfaces are typically represented as meshes, which can be extracted from volumetric fields via meshing or optimized directly as surface parameterizations. Volumetric representations occupy 3D space and have a large effective receptive field along rays, enabling stable and efficient optimization via volumetric rendering; however, subsequent meshing often produces overly dense meshes and introduces accumulated errors. In contrast, pure surface methods avoid meshing but capture only boundary geometry with a single-layer receptive field, making it difficult to learn intricate geometric details and increasing reliance on priors (e.g., shading or normals). We bridge this gap by differentiably turning a surface representation into a volumetric one, enabling end-to-end surface reconstruction via volumetric rendering to model complex geometries. Specifically, we soften a mesh into multiple semi-transparent layers that remain differentiable with respect to the base mesh, endowing it with a controllable 3D receptive field. Combined with a splatting-based renderer and a topology-control strategy, our method can be optimized in about 20 minutes to achieve accurate surface reconstruction while substantially improving mesh quality.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 3744