NeIF: Representing General Reflectance as Neural Intrinsics Fields for Uncalibrated Photometric Stereo

ZONGRUI LI, Qian Zheng, Feishi Wang, Boxin Shi, Gang Pan, Xudong Jiang

Abstract: Uncalibrated photometric stereo is challenging due to the general bas-relief ambiguity. Existing solutions alleviate this ambiguity by either building an explicit relationship between reflectance and lighting or resolving lighting information in a supervised manner before recovering surface normal, which suffers from poor generalization to unseen reflectance or data. In contrast, this paper builds the implicit relationship between general reflectance (specular, cast shadow) and lighting by representing the reflectance as several neural intrinsics fields, based on which we optimize the surface normal and lighting in an unsupervised manner. Specifically, the neural intrinsics fields include reflectance features (i.e., diffuse, specular, diffuse coefficient, specular coefficient, cast shadow) and shading features (i.e., surface normal, lighting information). The implicit relationship is achieved by feeding the lighting information to neural specular & shadow fields and optimizing all intrinsics through a rendering equation in an unsupervised manner, which facilitates the better generalization to unseen reflectance and data. Our method achieves a superior performance advantage over state-of-the-art uncalibrated photometric stereo methods on public datasets in terms of the surface normal & lighting estimation.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/neif-representing-general-reflectance-as/code)