Diffusion-Based Photorealistic Bokeh Rendering for Mobile Devices

Linxiao Shi, Siming Zheng, Zerong Wang, Hao Zhang, Jinwei Chen, Bo Li, Shifeng Chen, Peng-Tao Jiang

Keywords: Bokeh Rendering, Mobile Photography

Abstract: Photographs captured by mobile devices are often constrained by physical limitations, \textit{i.e.}, small apertures, making it challenging to achieve the bokeh effects of shallow depth-of-field. Although previous work has primarily focused on learning-based methods to simulate bokeh effects for mobile images, they still face challenges when processing photos captured at high digital zoom levels on mobile devices, which often suffer from reduced resolution and degraded details. Therefore, it is still necessary to improve the quality of these inputs before creating the photorealistic bokeh effects, but this requirement will introduce inefficiencies in the workflow and lead to unnecessary error accumulation. To address the aforementioned issues, we propose MagicBokeh, a unified diffusion-based framework that improves both the quality and efficiency of bokeh rendering for high-zoom mobile photography. With the help of the proposed alternative training strategy and focus-aware mask attention, our approach achieves a unified optimization of bokeh rendering and super-resolution, thus improving both the controllability and quality of mobile bokeh rendering. Additionally, we further optimize depth estimation on low-quality images by degradation-aware depth module. Experiments demonstrate that MagicBokeh efficiently simulates high-quality bokeh effects under complex backgrounds, especially for digital zoom inputs from mobile devices. Code will be made publicly available.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 6563