Multi-3D Occlusion Mask Learning for Flexible Occlusion Removal in Neural Radiance Fields

Zhuoyu Shi, Shuo Zhang, Song Chang, Youfang Lin

Abstract: As NeRFs modeling becomes more widely available, there is an increasing demand for the ability to flexibly and conveniently exclude unnecessary obstructions during the modeling process. Existing methods generally adopt a “ignore” strategy for occlusions, which cannot conveniently and flexibly remove any occlusions in complex scenes. We propose a new method that only requires the introduction of a small number of different external occlusion annotation to model independent 3D masks for different occlusions in space. This “model first, remove later” occlusion removal strategy allows us to model the scene only one process and obtain unobstructed images from desired viewpoint, with any specific or multiple obstruction removed. Experimental results on existing and our synthesized datasets validate the effectiveness of our method and strategy.

External IDs:dblp:conf/prcv/ShiZCL24