Spatially and color consistent environment lighting estimation using deep neural networks for mixed reality

Bruno Augusto Dorta Marques, Esteban Walter Gonzalez Clua, Anselmo Antunes Montenegro, Cristina Nader Vasconcelos

Abstract: Highlights•Automatic end-to-end method to estimate the environment lighting in XR applications.•A CNN architecture that learns a latent-space of the environment lighting.•A methodology to generate egocentric mixed-reality-views from HDR panoramas.•Real-time lighting estimation that does not make assumptions about the XR scene.