GBSD: Generative Bokeh with Stage Diffusion

Jieren Deng, Xin Zhou, Hao Tian, Zhihong Pan, Derek Aguiar

Abstract: The bokeh effect is an artistic technique that blurs out-of-focus areas in a photograph and has gained interest due to recent developments in text-to-image synthesis and the ubiquity of smartphone cameras and photo sharing apps. Prior work on rendering bokeh effects have focused on manipulating photographs using classical computer graphics or neural rendering techniques, but have either depth discontinuity artifacts or are restricted to reproducing bokeh effects that are present in the training data. In this paper, we present generative bokeh with stage diffusion (GBSD), the first generative text-to-image model that synthesizes photorealistic images with a bokeh style. Motivated by how image synthesis occurs progressively in diffusion models, our approach combines latent diffusion models with a 2-stage conditioning algorithm to render bokeh effects on semantically defined objects. Since GBSD focuses the blurring effect on objects, this semantic bokeh effect is more versatile than classical rendering techniques. We evaluate GBSD both quantitatively and qualitatively and demonstrate its ability to be applied in both text-to-image and image-to-image settings.