Point Cloud Densification for 3D Gaussian Splatting from Sparse Input Views
Abstract: The technique of 3D Gaussian splatting (3DGS) has demonstrated its effectiveness and efficiency in rendering photo-realistic images for novel view synthesis. However, 3DGS requires a high density of camera coverage, and its performance inevitably degrades with sparse training views, which significantly restricts its applications in real-world products. In recent years, many researchers have tried to use depth information to alleviate this problem, but the performance of their methods is sensitive to the accuracy of depth estimation. To this end, we propose an efficient method to enhance the performance of 3DGS with sparse training views. Specifically, instead of applying depth maps for regularization, we propose a densification method that generates high-quality point clouds for improved initialization of 3D Gaussians. Furthermore, we propose Systematically Angle of View Sampling (SAOVS), which employs Spherical Linear Interpolation (SLERP) and linear interpolation for side view sampling, to determine unseen views outside the training data for semantic pseudo-label regularization. Experiments show that our proposed method significantly outperforms other promising 3D rendering models on the ScanNet dataset and the LLFF dataset. In particular, compared with the conventional 3DGS method, the PSNR and SSIM performance gains achieved by our method are up to 1.71dB and 0.07, respectively. In addition, the novel view synthesis obtained by our method demonstrates the highest visual quality with fewer distortions.
Primary Subject Area: [Content] Vision and Language
Relevance To Conference: Our work on synthesizing photo-realistic novel-view images from sparse input views significantly enhances both the realism and interactivity in multimedia processing, particularly within the realms of AR and VR. The enhanced realism, achieved through these photo-realistic images, provides users with immersive and lifelike experiences, effectively bridging the gap between virtual and physical realities. In addition, the novel-view images foster dynamic interactivity, empowering users to explore virtual environments from unique perspectives. This fusion of realism and interactivity is pivotal in crafting engaging multimedia content. By pushing the boundaries of user experiences in AR and VR, our work makes a substantial contribution to the field of multimedia processing.
Submission Number: 4055
Loading