Human Perception-Guided Meta-Training for Few-Shot NeRF

Bingyin Li, Xiaoyu Xu, Sheyang Tang, Li Yu, Zhou Wang

Abstract: Neural Radiance Fields (NeRF) have recently achieved impressive results in synthesizing novel views. However, with the few-shot setup, the NeRF is prone to overfitting to the supervision views, which causes artifacts when rendering novel views. To this end, we design a human-perception-guided meta-training framework to enhance the novel views without needing daunting annotation efforts. Specifically, we first analyze that the few-shot NeRF suffers from quality fluctuation, distant view degradation, and geometrical inconsistency. Then, we design a human perception guider to evaluate and select views for enhancement. Further, a human perception score modulated hyper-restoration module is designed to handle different degradation. Finally, we adopt meta-learning to fit enhanced views to avoid any impairment to the supervision views. Besides, we can directly embed any NeRF model into the framework for training and enhancement. Experimental results on two widely used benchmark datasets, LLFF and DTU, demonstrate our superiority to the state-of-the-art methods.