AbsGS: Recovering fine details in 3D Gaussian Splatting

Zongxin Ye, Wenyu Li, Sidun Liu, Peng Qiao, Yong Dou

Abstract: Recent advances on neural rendering have shown photo-realistic results in novel view synthesis. As one of the most promising method, 3D Gaussian Splatting (3D-GS) couple 3D Gaussian primitive with differentiable rasterization to obtain high-fidelity 3D scene reconstruction and achieve real-time rendering. The exceptional performance of 3D-GS is attributed to the carefully designed adpative density control strategy, which progressively populate empty areas by splitting/cloning more Gaussians throughout the optimization process. While 3D-GS offers significant advantages, it frequently suffer from over-reconstruction issue in intricate scenes containing high-frequency details, consequently leading to blur. This issue's underlying causes have still been under-explored. In this work, we present an comprehensive analysis of the cause of aforementioned artifacts and we call it gradient collision, which prevent large Gaussians that cover small-scale geometry from splitting. To address this issue, we further propose novel homodirectional gradient as the guidance for densification. Our strategy efficiently identifies large Gaussians in over-reconstructed regions, and recovers fine details by splitting. We evaluate our proposed method on various challenging datasets, and our approach achieves best rendering quality with reduced memory consumption and yields better distributions of 3D Gaussians in world space. Our method is also easy to implement with just few lines of codes and can be incorporated into a wide variety other Gaussian Splatting-based methods. We will open source our codes upon formal publication.

Primary Subject Area: [Content] Media Interpretation

Relevance To Conference: This paper focuses on high-quality novel view synthesis techniques. Novel view synthesis (NVS) is closely related to multimedia, providing important applications and support for multimedia processing. Firstly, NVS techniques can enhance user experience by synthesizing new perspectives, allowing users to observe and interact from different viewpoints, thereby enhancing the appeal and interactivity of multimedia content. Secondly, NVS techniques can expand content creation. By synthesizing new perspectives, creators can produce more creative and attractive multimedia works, enriching and expanding the field of multimedia content creation. Lastly, NVS techniques can improve content editing efficiency: for example, in video editing, NVS techniques can be used to generate different perspectives of dynamic scenes, achieving richer and more refined video editing effects. In summary, NVS technology has broad application prospects in the multimedia field, providing important support and assistance for the creation, editing, presentation, and interaction of multimedia content, and promoting the development and progress of multimedia technology.

Submission Number: 3554