Local Image-to-Image Translation via Pixel-wise Highway Adaptive Instance Normalization

Wonwoong Cho, Seunghwan Choi, Junwoo Park, David Keetae Park, Tao Qin, Jaegul Choo

Abstract: Recently, image-to-image translation has seen a significant success. Among many approaches, image translation based on an exemplar image, which contains the target style information, has been popular, owing to its capability to handle multimodality as well as its suitability for practical use. However, most of the existing methods extract the style information from an entire exemplar and apply it to the entire input image, which introduces excessive image translation in irrelevant image regions. In response, this paper proposes a novel approach that jointly extracts out the local masks of the input image and the exemplar as targeted regions to be involved for image translation. In particular, the main novelty of our model lies in (1) co-segmentation networks for local mask generation and (2) the local mask-based highway adaptive instance normalization technique. We demonstrate the quantitative and the qualitative evaluation results to show the advantages of our proposed approach. Finally, the code is available at https://github.com/AnonymousIclrAuthor/Highway-Adaptive-Instance-Normalization

Keywords: image to image translation, image translation, exemplar, mutlimodal

Code: [ AnonymousIclrAuthor/Highway-Adaptive-Instance-Normalization](https://github.com/AnonymousIclrAuthor/Highway-Adaptive-Instance-Normalization)

Data: [CelebA](https://paperswithcode.com/dataset/celeba)