Post-hoc Compression of LoRA Adapters via Singular Value Decomposition

ICLR 2026 Conference Submission18152 Authors

Keywords: Text-to-Image Generation, Stable Diffusion, LoRA, Model Compression

TL;DR: We show that LoRA adapters for text-to-image models contain significant rank redundancy and can be compressed with SVD without retraining or quality loss.

Abstract:

Low-Rank Adaptation (LoRA) has become a widely used technique for personalizing text-to-image models such as Stable Diffusion. Although LoRA greatly reduces fine-tuning costs by introducing low-rank updates, practical deployments often involve one large backbone model combined with thousands of LoRA adapters. This creates a new challenge: even though each adapter is relatively small, the collective storage and transmission cost of large LoRA libraries becomes substantial.
In this paper, we investigate whether existing LoRA adapters can be further compressed without retraining. We begin from the observation that while LoRA constrains updates to rank $r$ matrices, the effective rank of the learned updates is often significantly smaller, in practice, require only rank 1 to capture its expressive power. Motivated by this redundancy, we introduce a simple singular value decomposition (SVD)–based method to compress pretrained LoRA adapters. Through extensive experiments on a variety of text-to-image LoRA models, we show that trained LoRA models indeed contain considerable rank redundancy and SVD compression can consistently reduce adapter dimension without notable loss in generation quality.

Primary Area: generative models

Submission Number: 18152