LoRA dropout as a sparsity regularizer for overfitting reduction

Yang Lin, Xinyu Ma, Xu Chu, Yujie Jin, Zhibang Yang, Yasha Wang

Abstract: Highlights•Generalization error bound of applying dropout on LoRA.•A more expressive and efficient design to implement dropout on LoRA.•Extensive experiments to verify the effectiveness of LoRA Dropout.

External IDs:dblp:journals/kbs/LinMCJYW25