Uni-DlLoRA: Style Fine-Tuning for Fashion Image Translation

Fangjian Liao, Xingxing Zou, Waikeung Wong

Abstract: Image-to-image (i2i) translation has achieved notable success, yet remains challenging in scenarios like real-to-illustrative style transfer of fashion. Existing methods focus on enhancing the generative model with diversity while lacking ID-preserved domain translation. This paper introduces a novel model named Uni-DlLoRA to release this constraint. The proposed model combines the original images within a pretrained diffusion-based model using the proposed Uni-adapter extractors, while adopting the proposed Dual-LoRA module to provide distinct style guidance. This approach optimizes generative capabilities and reduces the number of additional parameters required. In addition, a new multimodal dataset featuring higher-quality images with captions built upon an existing real-to-illustration dataset is proposed. Experimentation validates the effectiveness of our proposed method.

Primary Subject Area: [Experience] Multimedia Applications

Secondary Subject Area: [Generation] Generative Multimedia

Relevance To Conference: the work contributes to multimedia/multimodal processing by enhancing the capabilities of generative models for style transfer in fashion, improving the quality and utility of multimodal datasets, and demonstrating applications of these advancements through effective experimentation.

Submission Number: 4075