Localised-NeRF: Specular Highlights and Colour Gradient Localising in NeRF

Dharmendra Selvaratnam, Dena Bazazian

Abstract:

Neural Radiance Field (NeRF) based systems predominantly operate within the RGB (Red, Green, and Blue)
space; however, the distinctive capability of the HSV (Hue,
Saturation, and Value) space to discern between specular
and diffuse regions is seldom utilised in the literature. We
introduce Localised-NeRF, which projects the queried pixel
point onto multiple training images to obtain a multi-view
feature representation on HSV space and gradient space
to obtain important features that can be used to synthesise novel view colour. This integration is pivotal in identifying specular highlights within scenes, thereby enriching
the model’s understanding of specular changes as the viewing angle alters. Our proposed Localised-NeRF model uses
an attention-driven approach that not only maintains local
view direction consistency but also leverages image-based
features namely the HSV colour space and colour gradients. These features serve as effective indirect priors for
both the training and testing phases to predict the diffuse
and specular colour. Our model exhibits competitive performance with prior NeRF-based models, as demonstrated
on the Shiny Blender and Synthetic datasets. The code of
Localised-NeRF is publicly available 1.