Generative Active Learning for Image Synthesis Personalization

Xulu Zhang, Wengyu Zhang, Xiaoyong Wei, Jinlin Wu, Zhaoxiang Zhang, Zhen Lei, Qing Li

Abstract: This paper presents a pilot study that explores the application of active learning, traditionally studied in the context of discriminative models, to generative models. We specifically focus on image synthesis personalization tasks. The primary challenge in conducting active learning on generative models lies in the open-ended nature of querying, which differs from the closed form of querying in discriminative models that typically target a single concept. We introduce the concept of anchor directions to transform the querying process into a semi-open problem. We propose a direction-based uncertainty sampling strategy to enable generative active learning and tackle the exploitation-exploration dilemma. Extensive experiments are conducted to validate the effectiveness of our approach, demonstrating that an open-source model can achieve superior performance compared to closed-source models developed by large companies, such as Google's StyleDrop. The source code is available at https://github.com/(open\_upon\_acceptance).

Relevance To Conference: This work makes several contributions to multimedia and multimodal processing, specifically in the area of personalized image synthesis. We present a novel approach for applying active learning principles to generative models, especially for the task of personalized image synthesis. This extends active learning, which has traditionally focused on discriminative models, to the generative domain.

Primary Subject Area: [Generation] Generative Multimedia

Secondary Subject Area: [Generation] Generative Multimedia

Submission Number: 1837