P-LoRA: Posterior Knowledge Enables Training-Free Fusion of Subject and Style LoRAs

Qinglong Cao, Xin Xu, Chao Ma, Xiaokang Yang, Yuntian Chen

Keywords: Training-Free LoRA Fusion, Subject and Style LoRAs, Image Generation

TL;DR: we introduce P-LoRA, a fresh training-free fusion paradigm that leverages posterior knowledge of fine-tuned features, fundamentally shifting the fusion process from weight-level heuristics to representation-aware decisions.

Abstract: Recent studies have explored the combination of multiple LoRAs to simultaneously generate learned subjects and styles. However, most existing approaches fuse LoRA weights directly based on their statistical properties, which deviates from the original intent of LoRA, namely learning additional features to adapt to diverse functions. To address this limitation, we introduce \P-LoRA, a fresh training-free fusion paradigm that leverages posterior knowledge from fine-tuned features, fundamentally shifting the fusion process from weight-level heuristics to representation-conditional decisions. Specifically, at each LoRA-applied layer, we compute the KL divergence between the original features and the features generated by subject and style LoRAs, respectively, to adaptively select the most appropriate weights for fusion. Furthermore, objective metrics such as CLIP and DINO scores, which reflect alignment and semantic consistency, are employed as posterior knowledge to dynamically adjust denoised embeddings during the generation process. By incorporating posterior knowledge into the fusion pipeline, P-LoRA effectively preserves the most representative subject and style characteristics without requiring retraining. Extensive experiments across diverse subject-style combinations demonstrate that P-LoRA consistently outperforms existing methods, achieving superior results both qualitatively and quantitatively.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 11864