HumanIR-MGI: human inverse rendering via jointly optimizing geometry, material, and illumination

Ruhao Wang, Yu Jiang, Huizhi Zhu, Fei Luo, Chunxia Xiao

Abstract: Accurately decoupling geometry, material, and illumination remains a significant challenge in human inverse rendering. Meanwhile, detailed geometric reconstruction and accurate lighting modeling can enhance the quality of material reconstruction. To improve the accuracy of estimated human materials, we propose a novel method, named HumanIR-MGI, which jointly optimizes geometry, materials, and illumination from multi-view images captured under unknown lighting. Specifically, our method employs a two-stage framework. A pretraining stage reconstructs an initial geometry, and a fine-tuning stage jointly refines geometry while estimating illumination and materials. Besides, we treat the outgoing radiance from indirect-bounce points as indirect illumination and estimate visibility to distinguish between direct and indirect illumination. Moreover, we incorporate an anisotropy model to account for the complex material properties of the human body. Through extensive quantitative and qualitative experiments, our method demonstrates its superiority over state-of-the-art methods.