Real-time video based lighting using GPU raytracing

Joel Kronander, Johan Dahlin, Daniel Jönsson, Manon Kok, Thomas B. Schön, Jonas Unger

Abstract: The recent introduction of high dynamic range (HDR) video cameras has enabled the development of image based lighting techniques for rendering virtual objects illuminated with temporally varying real world illumination. A key challenge in this context is that rendering realistic objects illuminated with video environment maps is computationally demanding. In this work, we present a GPU based rendering system based on the NVIDIA OptiX [1] framework, enabling real time raytracing of scenes illuminated with video environment maps. For this purpose, we explore and compare several Monte Carlo sampling approaches, including bidirectional importance sampling, multiple importance sampling and sequential Monte Carlo samplers. While previous work have focused on synthetic data and overly simple environment map sequences, we have collected a set of real world dynamic environment map sequences using a state-of-art HDR video camera for evaluation and comparisons. Based on the result we show that in contrast to CPU renderers, for a GPU implementation multiple importance sampling and bidirectional importance sampling provide better results than sequential Monte Carlo samplers in terms of flexibility, computational efficiency and robustness.