Particle Rendering: Implicitly Aggregating Incident and Outgoing Light Fields for Novel View Synthesis

Tao Hu, Zhiwen Yan, Xiaogang Xu, Gim Hee Lee

Keywords: Neural Radiance Fields, Implicit Rendering, Lighting

TL;DR: Particle rendering method that aggregates the incident light and outgoing light for rendering

Abstract: This paper presents Particle Rendering (PR), a new implicit rendering approach that extends Neural Radiance Fields (NeRF) by incorporating incident light along with traditional outgoing light modeling. In our framework, a 3D scene consists of a mass of particles, each offering a deeper understanding of light interactions by reflecting and emitting light in all directions. Our methodology involves a three-phase training pipeline: 1) Estimating the outgoing light field through a NeRF model; 2) Distilling the incident light field. A simple metric is introduced to assess the quality of the ray for better supervision; 3) Implicit rendering. We propose an implicit method to aggregate incident and outgoing fields that leverages Multilayer Perceptrons (MLP) to directly infer final pixel values, thus avoiding the limitation of traditional physically-based rendering techniques. The effectiveness of PR is demonstrated through state-of-the-art results in various challenging indoor and outdoor scenes, emphasizing its capability to handle complex lighting and reflective materials.

Submission Number: 333