Object-Centric Noise Filtering in Neural Radiance Fields via Influence Functions and Segmentation
Keywords: NeRF, robust learning
TL;DR: Enhancing robustness in NeRF through the use of influence functions, a classic technique in robust learning, and segmentation method to improve the precision of noise detection.
Abstract: Neural Radiance Fields (NeRF) is a method for 3D scene modeling that employs fully-connected networks to learn 3D geometric information and synthesizes high-quality novel views. However, NeRF exhibits vulnerability when confronted with distractors in the training images, such as the presence of moving objects like pedestrians or different weather conditions within specific views. Given the difficulty of data curation in NeRF compared to other domains, training a robust model that maintains 3D consistency is an important and timely challenge. Previous approaches have attempted to differentiate distractors by using loss values, but there is a fundamental limitation that hard-to-learn pixels like high-frequency details also show high loss values. In this paper, we propose a noise pruning framework via influence functions to effectively filter out noisy pixels, ultimately enhancing the robustness of NeRF. Furthermore, we improve the precision of detection by incorporating segmentation techniques to refine pixel-level predictions. Our method demonstrates superior performance on benchmark datasets, including synthetic and natural scenes, showcasing its effectiveness across various environments and proficiency in dataset pruning.
Primary Area: representation learning for computer vision, audio, language, and other modalities
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 9421
Loading