Insty-GAN for Image-to-Image Location Transfer

Sanbi Luo

Abstract: Image-to-Image Transformation (I2I) aims to transfer an image from a source domain to a target domain while changing some specified feature representations and preserving some other feature representations. For example, most I2I tasks keep the layout features of an image while changing its color, style, or resolution features. However, image-to-image location transfer only wants to change the layout of objects in an image's foreground, such as LT-GAN and Orthod-GAN. In this paper, we researched the above image-to-image location translation tasks. First, we studied the similarities and differences between malpositioned-teeth images and their corresponding neat-teeth images. Then we analyzed the problems of directly applying the LT-GAN method to the orthodontics transfer task. Finally, we proposed a new approach called Insty-GAN that uses an instance transfer bridge and a style loss to address image-to-image location transfer. Our Insty-GAN is applied to orthodontic and chaos-to-order transfer tasks and improved the FID score by 9.1% and 8.1% over their SOTA methods. LPIPS scores also show that Insty-GAN performs a more excellent production than other baselines in image-to-image location transfer. Additionally, we have annotated 10K instance segmentation labels for Orthod and M2C datasets. These labels could also be used in future works. The added data is available at https://drive.google.com/drive/folders/l5ilnLE_pwIB6fQq1nbWTpMxav6COdU7p?usp=sharing.

External IDs:dblp:conf/ijcnn/Luo23