Foreground Harmonization and Shadow Generation for Composite Image

Jing Zhou, Ziqi Yu, Zhongyun Bao, Gang Fu, Weilei He, Chao Liang, Chunxia Xiao

Abstract: We propose a method for light and shadow editing of outdoor disharmonious composite images, including foreground harmonization and cast shadow generation. Most existing work can only perform foreground appearance editing tasks or only focus on shadow generation. In fact, lighting not only affects the brightness and color of objects, but also produces corresponding cast shadows. In recent years, diffusion models have demonstrated their strong generative capabilities, and due to their iterative denoising properties, they have a significant advantage in image restoration tasks. But it fails to preserve content structure of image. In this purpose, we propose an effective model to tackle the problem of foreground light-shadow editing. Specifically, we use a coarse shadow prediction module (SP) to generate coarse shadows for foreground objects. Then, we use the predicted results as prior knowledge to guide the generation of harmony diffusion model. In this process, the primary task is to learn lighting variation to harmonize foreground regions. The secondary task is to generate high-quality cast shadow containing more details. Considering that existing datasets do not support the dual tasks of image harmonization and shadow generation, we construct a real outdoor dataset, IH-SG, covering various lighting conditions. Extensive experiments conducted on existing benchmark datasets and the IH-SG dataset demonstrate the superiority of our method.

Primary Subject Area: [Content] Vision and Language

Secondary Subject Area: [Generation] Generative Multimedia

Relevance To Conference:

In the task of image composition, shadow generation and foreground harmonization are two important subtasks. Shadow generation aims to add realistic shadow effects to the synthesized image, making it look more authentic and natural. This is particularly important when synthesizing images under different lighting conditions, as shadows accurately reflect the direction and intensity of illumination. On the other hand, foreground harmonization task aims to seamlessly integrate foreground objects into the background, making them appear harmonious with the environment without being obtrusive. Effective foreground harmonization ensures the overall consistency and realism of the synthesized image.
ACM MM conference is highly relevant to the tasks of shadow generation and foreground harmonization in image composition. ACM MM conference is one of the top conferences in the multimedia field, covering various areas including image processing, computer vision, and multimedia technology. At ACM MM conference, researchers often submit their latest research findings on image composition tasks, including works on shadow generation and foreground harmonization. These research findings not only drive the advancement of the field of image composition but also provide crucial support and guidance for the progress of multimedia applications and technologies.

Submission Number: 3528