LGR: Local Geometric Refinement in High-Fidelity Surgical Scene Reconstruction

Pengpeng Li, Xiangbo Shu, Chun-Mei Feng, Guo-Sen Xie, Zheng Zhang, Xin Gao, Jinhui Tang

Keywords: Surgical Scene Reconstruction; Local Geometric Refinement; 3D Gaussian Splatting; Surgical Video Analysis

Abstract: Dynamic reconstruction of deformable surgical scenes has the potential to significantly advance robot-assisted surgery. Building on recent advancements in 3D Gaussian splatting (3DGS), current surgical scene reconstruction (SSR) methods have made notable initial progress. Despite this progress, challenges remain in accurately tracking local tissue deformations during surgery, primarily due to the lack of deformation constraints within the local Gaussian neighborhoods of surgical tissues. In this work, we address these issues by proposing a local geometric refinement (LGR) framework based on 3DGS for high-fidelity SSR. Specifically, we first utilize prior visual information to efficiently perform the Gaussian initialization. Following the initialization, we incorporate local geometric constraints to accurately track the local non-rigid deformations occurring in the surgical scene. Furthermore, considering the low-quality scenarios in real surgeries, we apply low-quality enhancement to optimize the fidelity of local details in the preliminarily rendered scene. Experimental results on public datasets demonstrate that LGR outperforms previous state-of-the-art methods. Notably, it achieves an average improvement of over 50\% in terms of LPIPS, a metric that better reflects human perceptual consistency, while maintaining favorable computational cost. These results highlight the great potential of the proposed LGR for promoting practical applications in surgical scenarios. *Our code and model will be released publicly*.

Primary Area: applications to physical sciences (physics, chemistry, biology, etc.)

Submission Number: 11832