Superpixel-based Efficient Sampling for Learning Neural Fields from Large Input
Abstract: In recent years, novel view synthesis methods using neural implicit fields have gained popularity due to their exceptional rendering quality and rapid training speed. However, the computational cost of volumetric rendering has increased significantly with the advancement of camera technology and the consequent rise in average camera resolution. Despite extensive efforts to accelerate the training process, the training duration remains unacceptable for high-resolution inputs. Therefore, the development of efficient sampling methods is crucial for optimizing the learning process of neural fields from a large volume of inputs. In this paper, we introduce a novel method named Superpixel Efficient Sampling (SES), aimed at enhancing the learning efficiency of neural implicit fields. Our approach optimizes pixel-level ray sampling by segmenting the error map into multiple superpixels using the slic algorithm and dynamically updating their errors during training to increase ray sampling in areas with higher rendering errors. Compared to other methods, our approach leverages the flexibility of superpixels, effectively reducing redundant sampling while considering local information. Our method not only accelerates the learning process but also improves the rendering quality obtained from a vast array of inputs. We conduct extensive experiments to evaluate the effectiveness of our method across several baselines and datasets. The code will be released.
Primary Subject Area: [Experience] Multimedia Applications
Secondary Subject Area: [Content] Vision and Language, [Content] Multimodal Fusion
Relevance To Conference: Our work focuses on accelerating the training process of Neural Radiance Fields (NERF) models. NeRF utilizes 2D images as input to reconstruct scenes into neural implicit fields, capable of rendering realistic scene images. In multimedia fields such as Augmented Reality (AR) and Virtual Reality (VR), NERF models enable highly realistic image synthesis, closely intertwined with multimedia. By expediting the training process of NeRF, we can deliver more realistic and immersive experiences for AR and VR applications.
Submission Number: 3313
Loading