CoCoIns: Consistent Subject Generation via Contrastive Instantiated Concepts

Lee Hsin-Ying, Kelvin C.K. Chan, Ming-Hsuan Yang

Abstract: While text-to-image generative models can synthesize diverse and faithful content, subject variation across multiple generations limits their application to long-form content generation. Existing approaches require time-consuming fine-tuning, reference images for all subjects, or access to previously generated content. We introduce Contrastive Concept Instantiation (CoCoIns), a framework that effectively synthesizes consistent subjects across multiple independent generations. The framework consists of a generative model and a mapping network that transforms input latent codes into pseudo-words associated with specific concept instances. Users can generate consistent subjects by reusing the same latent codes. To construct such associations, we propose a contrastive learning approach that trains the network to distinguish between different combinations of prompts and latent codes. Extensive evaluations on human faces with a single subject show that CoCoIns performs comparably to existing methods while maintaining greater flexibility. We also demonstrate the potential for extending CoCoIns to multiple subjects and other object categories.

Submission Length: Regular submission (no more than 12 pages of main content)

Code: https://github.com/contrastive-concept-instantiation/cocoins

Assigned Action Editor: ~Tae-Hyun_Oh3

Submission Number: 5255