PIR: Photometric Inverse Rendering with Shading Cues Modeling and Surface Reflectance Regularization

Jingzhi Bao, Guanying Chen, Shuguang Cui

Keywords: Neural Rendering, Inverse Rendering

Abstract: This paper addresses the problem of inverse rendering from photometric images. Existing approaches for this problem suffer from the effects of self-shadows, inter-reflections, and lack of constraints on the surface reflectance, leading to inaccurate decomposition of reflectance and illuminations due to the ill-posed nature of inverse rendering. In this work, we propose a new method for neural inverse rendering. Our method jointly optimizes the light source position to account for the self-shadows in the images, and computes indirect illumination using a differentiable rendering layer and an importance sampling strategy. To enhance surface reflectance decomposition, we introduce a new regularization by distilling DINO features to foster accurate and consistent material decomposition. Extensive experiments on synthetic and real datasets demonstrate that our method outperforms state-of-the-art methods in reflectance decomposition.

Submission Number: 140