Trans-DONeRF for Transparent Object Rendering with Mixed Depth Prior

Jiangnan Ye, Taoqi Bao, Lianrui Mu, Yuchen Yang, Jiedong Zhuang, Xiaoyu Liang, Jiayi Xu, Haoji Hu

Abstract: The advent of Neural Radiance Fields (NeRF) has significantly impacted the field of novel view synthesis, heralding a new era of methodological advancements. Depth Oracle NeRF (DONeRF), as a depth-guided sampling methodology, reaches a real-time rendering efficiency as well as compatibility with other NeRF speeding methods. However, the surface depth of transparent objects from synthetic datasets can result in blurry and chaotic renderings, as it mistakenly focuses on the surface position. The problem is that the observed radiance is not emitted from the transparent surface, but rather from the virtual image of the object behind it. In response to this challenge, we introduce Trans-DONeRF to augment the rendering fidelity of transparent objects whilst preserving DONeRF’s rendering efficiency. Trans-DONeRF incorporates a modular plug-and-play component, Multi-View Grounded-SAM (MV-GSAM), which autonomously segments transparent objects with multi-view consistency by exploiting the textual-and-semantic-aligned features. On this base, we design a refined depth prior Classified Mixed Depth and an SDF-based transparent surface tailored Density Loss for DONeRF training. Comprehensive experiments validate the superiority of our approach in enhancing the quality of multi-view segmentation and transparent object rendering. Furthermore, we release our synthetic dataset of glasses and mirrors to cover the vacancies of related research, which is released at Google Drive (https://drive.google.com/drive/folders/1Fs1OeTY4-6x2ZuS7ZGvgA0ejWGGFP22v?usp=drive_link.)