Dual-head Genre-instance Transformer Network for Arbitrary Style Transfer

Meichen Liu, Shuting He, Songnan Lin, Bihan Wen

Abstract: Arbitrary style transfer aims to render artistic features from a style reference onto an image while retaining its original content. Previous methods either focus on learning the holistic style from a specific artist or extracting instance features from a single artwork. However, they often fail to apply style elements uniformly across the entire image and lack adaptation to the style of different artworks. To solve these issues, our key insight is that the art genre has better generality and adaptability than the overall features of the artist. To this end, we propose a Dual-head Genre-instance Transformer (DGiT) framework to simultaneously capture the genre and instance features for arbitrary style transfer. To the best of our knowledge, this is the first work to integrate the genre features and instance features to generate a high-quality stylized image. Moreover, we design two contrastive losses to enhance the capability of the network to capture two style features. Our approach ensures the uniform distribution of the overall style across the stylized image while enhancing the details of textures and strokes in local regions. Qualitative and quantitative evaluations demonstrate that our approach exhibits its superior performance in terms of visual qualitative and efficiency.

Primary Subject Area: [Experience] Art and Culture

Relevance To Conference: Our work is to achieve the arbitrary style transfer task, which aims to render artistic features from a style reference onto an image while retaining its original content. This work can enhance and personalize multimedia content by applying styles from different artworks or images, enriching audience experiences. Moreover, it empowers content generation and manipulation, facilitating the creation of diverse multimedia assets with unique styles and themes.

Submission Number: 4759