iFlowGAN: An Invertible Flow-Based Generative Adversarial Network for Unsupervised Image-to-Image Translation

Longquan Dai, Jinhui Tang

Abstract: We propose iFlowGAN that learns an <i>invertible flow</i> (a sequence of invertible mappings) via <i>adversarial learning</i> and exploit it to transform a source distribution into a target distribution for <i>unsupervised image-to-image translation</i> . Existing GAN-based generative model such as CycleGAN [1], StarGAN [2], AGGAN [3] and CyCADA [4] needs to learn a highly under-constraint forward mapping <inline-formula><tex-math notation="LaTeX">$\mathcal {F}: X \rightarrow Y$</tex-math></inline-formula> from a source domain <inline-formula><tex-math notation="LaTeX">$X$</tex-math></inline-formula> to a target domain <inline-formula><tex-math notation="LaTeX">$Y$</tex-math></inline-formula> . Researchers do this by assuming there is a backward mapping <inline-formula><tex-math notation="LaTeX">$\mathcal {B}: Y \rightarrow X$</tex-math></inline-formula> such that <inline-formula><tex-math notation="LaTeX">$\boldsymbol{x}$</tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX">$\boldsymbol{y}$</tex-math></inline-formula> are fixed points of the composite functions <inline-formula><tex-math notation="LaTeX">$\mathcal {B} \circ \mathcal {F}$</tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX">$\mathcal {F} \circ \mathcal {B}$</tex-math></inline-formula> . Inspired by zero-order reverse filtering [5], we (1) understand <inline-formula><tex-math notation="LaTeX">$\mathcal {F}$</tex-math></inline-formula> via contraction mappings on a metric space; (2) provide a simple yet effective algorithm to present <inline-formula><tex-math notation="LaTeX">$\mathcal {B}$</tex-math></inline-formula> via the parameters of <inline-formula><tex-math notation="LaTeX">$\mathcal {F}$</tex-math></inline-formula> in light of Banach fixed point theorem; (3) provide a Lipschitz-regularized network which indicates a general approach to compose the inverse for arbitrary Lipschitz-regularized networks via Banach fixed point theorem. This network is useful for image-to-image translation tasks because it could save the memory for the weights of <inline-formula><tex-math notation="LaTeX">$\mathcal {B}$</tex-math></inline-formula> . Although memory can also be saved by directly coupling the weights of the forward and backward mappings, the performance of the image-to-image translation network degrades significantly. This explains why current GAN-based generative models including CycleGAN must take different parameters to compose the forward and backward mappings instead of employing the same weights to build both mappings. Taking advantage of the Lipschitz-regularized network, we not only build iFlowGAN to solve the redundancy shortcoming of CycleGAN but also assemble the corresponding iFlowGAN versions of StarGAN, AGGAN and CyCADA without breaking their network architectures. Extensive experiments show that the iFlowGAN version could produce comparable results of the original implementation while saving half parameters.