Manipulation Inversion by Adversarial Learning on Latent Statistical Manifold

Jialu Zhang, yifei li, Mai Xu, Shengxi Li, Shuting Liu, Lai Jiang, Shuhang Gu

TL;DR: We proposed a novel image inversion plugin method compatible with existing encoder-based GAN inversion methods and unify the reconstruction and editing into manipulation inversion.

Abstract: The inversion of generative adversarial network (GAN) is able to investigate rich semantics within the generative models, thus receiving increasing research efforts most recently. Existing GAN inversion methods focus on reconstructing images, with relatively less focus on improving the editing realism, the most important criterion for evaluating the semantics achieved by inversion. In this paper, we systematically investigate the latent generating space and prove that both the realism of editing and accuracy of reconstruction can be unified under the umbrella of the inversion against manipulations. Motivated by this, we propose to establish the generating space as latent probabilistic models, followed by the developed statistical manifold to minimise the distribution discrepancy. Based on the manifold, we further propose an adversarial learning strategy to avoid the excessive enumeration when calculating the manipulation inversion metric. We may also need to point out that the proposed method is universal to different architectures, as a novel plugin inversion method. We comprehensively evaluate our method across different types of network architectures, comparing it against the state-of-the-art inversion methods. The experimental results demonstrate that our method is able to achieve superior performances on both reconstruction accuracy and realism of editing.

Primary Area: Deep Learning->Other Representation Learning

Keywords: Image Synthesis, GAN Inversion

Submission Number: 14926