A Practical Approach to Depth-Aware Augmentation for Neural Radiance Fields

Hamed Razavi Khosroshahi, Jaime Sancho, Daniele Bonatto, Sarah Fachada, Gun Bang, Gauthier Lafruit, Eduardo Juárez, Mehrdad Teratani

Abstract: Neural Radiance Fields (NeRF) have demonstrated exceptional performance in generating novel views of scenes by learning implicit volumetric representations from calibrated RGB images, without depth information. A major limitation is the need for large training datasets in neural network-based view synthesis frameworks. The challenge of effective data augmentation for view synthesis remains unresolved. NeRF models require extensive scene coverage from multiple views to accurately estimate radiance and density. Insufficient coverage reduces the model’s ability to interpolate or extrapolate unseen parts of the scene effectively. In this paper, we propose a novel pipeline to address this data augmentation issue using depth map information. We use depth image-based rendering (DIBR) to overcome the lack of enough views for training NeRF. Experimental results indicate that our approach enhances the quality of rendered images using the NeRF framework, achieving an average peak signal-to-noise ratio (PSNR) increase of 7.2 dB, with a maximum improvement of 12 dB.

External IDs:dblp:conf/vcip/KhosroshahiSBFBLJT24