Semantics-Guided Object Removal for Facial Images: with Broad Applicability and Robust Style Preservation

Jookyung Song, Yeonjin Chang, Seonguk Park, Nojun Kwak

Abstract: Object removal and image inpainting in facial images is a task in which objects that occlude a facial image are specifically targeted, removed, and replaced by a properly reconstructed facial image. Two different approaches utilize U-net-based generator and modulated approach, and they respectively have been widely endorsed but notwithstanding each method’s disadvantages of low generative capability and low reconstruction power. Here, we propose a Semantics-Guided Inpainting Network (SGIN), which is the invention of a desirable trade-off between those two methods that can be applied to any form of occluding mask while maintaining a consistent style and preserving high-fidelity details of the original image. By using the guidance of a semantic map, our model is capable of manipulating facial features and styles which grants direction to the one-to-many problem for further practicability.