Inverse Rendering using Multi-Bounce Path Tracing and Reservoir Sampling

Yuxin Dai, Qi Wang, Jingsen Zhu, Dianbing Xi, Yuchi Huo, Chen Qian, Ying He

Keywords: Inverse Rendering, Material Decomposition, 3D reconstruction, relighting

TL;DR: We present a novel two-stage inverse rendering framework that jointly reconstructs and optimizes explicit geometry, materials, and light- ing from multi-view images.

Abstract:

We introduce MIRReS, a novel two-stage inverse rendering framework that
jointly reconstructs and optimizes explicit geometry, materials, and lighting
from multi-view images. Unlike previous methods that rely on implicit irradiance fields or oversimplified ray tracing, our method begins with an initial
stage that extracts an explicit triangular mesh. In the second stage, we refine this representation using a physically-based inverse rendering model
with multi-bounce path tracing and Monte Carlo integration. This enables our method to accurately estimate indirect illumination effects, including self-shadowing and internal reflections, leading to a more precise
intrinsic decomposition of shape, material, and lighting. To address the
noise issue in Monte Carlo integration, we incorporate reservoir sampling,
improving convergence and enabling efficient gradient-based optimization
with low sample counts. Through both qualitative and quantitative assessments across various scenarios, especially those with complex shadows,
we demonstrate that our method achieves state-of-the-art decomposition
performance. Furthermore, our optimized explicit geometry seamlessly
integrates with modern graphics engines supporting downstream applications such as scene editing, relighting, and material editing.

Primary Area: applications to computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 9571