GAN-based Transfer of Interpretable Directions for Disentangled Image Editing in Text-to-Image Diffusion Models

Yusuf Dalva, Hidir Yesiltepe, Pinar Yanardag

Track: Short paper

Keywords: Image Editing, GANs, diffusion models

TL;DR: We transfer editing directions from GANs to diffusion models.

Abstract: The rapid advancement in image generation models has predominantly been driven by diffusion models, which have demonstrated unparalleled success in generating high-fidelity, diverse images from textual prompts. However, these models are often characterized as black boxes due to their complex, less-understood mechanisms, highlighting a significant gap in interpretability research. In contrast, Generative Adversarial Networks (GANs) are praised for their well-structured latent spaces that offer rich semantics, enabling more straightforward exploration and understanding of model behaviors. GAN2Diff bridges this gap by transferring the structured, interpretable latent directions from pre-trained GAN models—representative of specific, controllable attributes—into diffusion models. This approach enhances the interpretability of diffusion models, preserving their generative quality while providing new avenues for exploring and manipulating complex image attributes.

Submission Number: 44