WaveletFT: Discrete wavelet transform for parameter-efficient fine-tuning

Can Hu, Jie Yang, Shien Song, Wentao Fan, Tao Xie

Abstract: Highlights•WaveletFT shows the core distinctions in the learning paradigms of FT and PEFT.•WaveletFT matches or exceeds LoRA’s performance with far fewer training parameters.•We study parameter scalability to assess our method’s learning capability.