Space-View Decoupled 3D Gaussians for Novel-View Synthesis of Mirror Reflections

Zhenwu Wang, Zhuopeng Li, Zhenhua Tang, Yanbin Hao, Huasen He

Abstract: Reflection modeling in Neural Radiance Fields has achieved high fidelity but is limited in terms of rendering speed. Recent advancements in 3D Gaussian Splatting (3D-GS) have demonstrated the feasibility of fast and high-quality rendering. However, challenges still exist in modeling reflective mirror scenes. In this paper, we propose a novel 3D-GS based method called Space-View Decoupled 3D Gaussians (SVD-GS) for the Novel-View Synthesis of Mirror Reflections. Instead of relying on computationally expensive ray-tracing methods, our approach leverages Plane-based method to decouple the geometry relations between space and view directions, effectively modeling the objects inside and outside of mirror space through Gaussian splatting. By employing a view-deformable field, our model efficiently predicts view-deformable positions. Additionally, through the use of neural Gaussians and a progressive training strategy, our model effectively fills in sparse deformed points, resulting in reduced computational costs and improved modeling effectiveness. Our SVD-GS achieves real-time rendering on an RTX 4090 GPU at a resolution of 800\(\,\times \,\)800, with a rendering speed of 179 FPS. Importantly, it maintains comparable or superior rendering quality compared to state-of-the-art methods.