Inverse Lighting with Differentiable Physically-Based Model

Kazem Meidani, Igor Borovikov, Amir Barati Farimani, Harold Chaput

Abstract: The design of scene lighting in video games and computer graphics can be a challenging and time-consuming task for lighting artists. Automating the lighting in problems such as stadium lighting design in sports games would help the artists by making this tedious process more efficient. In this work, we explore several practical solutions to this problem via optimization and data-driven models. First, we evaluate evolutionary and swarm intelligence gradient-free algorithms with black-box Physically-Based Rendering (PBR) models. Next, by implementing a differentiable PBR model, we leverage gradients to apply gradient-descent optimization to find an optimal solution. We exploit this differentiable model to develop a data-driven framework to learn the mapping from the illumination field to the lighting parameters via minimizing the loss between the illumination and its reconstructed field using a differentiable PBR decoder. Having the learned model, we directly predict the lighting configuration given a user-defined target illumination. In general, we show that all the mentioned methods can reach acceptable solutions, however, based on the conditions, one method can be preferred among others.

External IDs:dblp:conf/lion/MeidaniBFC23