Equivariant Indoor Illumination Map Estimation from a Single Image

Yusen Ai, Xiaoxue Chen, Xin Wu, Hao Zhao

Abstract: Thanks to the recent development of inverse rendering, photorealistic re-synthesis of indoor scenes have brought augmented reality closer to reality. All-angle environment illumination map estimation of arbitrary locations, as a fundamental task in this domain, is still challenging to deploy due to the requirement of expensive depth input. As such, we revisit the appealing setting of illumination estimation from a single image, using a cascaded formulation. The first stage predicts faithful depth maps from a single RGB image using a distortion-aware architecture. The second stage applies point cloud convolution operators that are equivariant to SO(3) transformations. These two technical ingredients collaborate closely with each other, because equivariant convolution would be meaningless without distortion-aware depth estimation. Using the public Matterport3D dataset, we demonstrate the effectiveness of our illumination estimation method both quantitatively and qualitatively. Code is available at https://github.com/Aitensa/Img2Illum.