Deep spherical harmonics light probe estimator for mixed reality games

Bruno A. D. Marques, Esteban Walter Gonzalez Clua, Cristina Nader Vasconcelos

Abstract: Highlights•Novel Deep learning based method that estimates the lighting condition of the real scene in interactive time from a raw image.•The method does not require any special equipment or prior knowledge of the scene.•The lighting estimation is an area light that can represent multiple light sources of distinct intensities and directions.•The Method is suitable for indoor and outdoor environments.•A mixed reality framework that incorporates the lighting estimation process to mitigate the lighting mismatch problem in real time.