Distinct and Shared Concept Discovery for Fine-grained Concept Inversion
Keywords: Vision language models, generative model, concept discovery
TL;DR: We propose DISCOD to discover shared and distinct concepts within and between two sets of images.
Abstract: A real-world object is expressed by composing distinctive characteristics that distinguish it from others and some common properties shared with different objects. Recent advances in generative modeling focus on identifying the shared concepts within images of individual identities. However, it remains unclear how to identify shared concepts beyond multiple identities while preserving the unique concepts inherent to each. In this work, we address this new problem of simultaneously discovering similarities and differences between two sets of images and propose a two-stage framework coined DISCOD (DIstinct and Shared COncept Discovery). In the first stage of DISCOD, we introduce information-regularized textual inversion, focusing on separating representative concepts distinctive from others while capturing the shared concepts among different objects. In the next stage, we further optimize them to align composited concepts of those with the corresponding objects, respectively. We demonstrate the effectiveness of DISCOD by showing that DISCOD discovers the concepts better than baselines, as measured by CLIPScore and success rate. The human study also validates the reasonable discovery capability of DISCOD. Furthermore, we show the practical applicability of our approach by applying to various applications: image editing, few-shot personalization of diffusion models, and group bias mitigation in recognition.
Primary Area: generative models
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 5597
Loading