Go to ICLR 2024 Conference homepageQFT: Quantized Full-parameter Tuning of LLMs with Affordable Resources
Large Language Models (LLMs) have showcased remarkable impacts across a wide spectrum of natural language processing tasks. Fine-tuning these pre-trained models on downstream datasets provides further significant performance gains, but this process has been challenging due to its extraordinary resource requirements. To this end, existing efforts focus on parameter-efficient fine-tuning, which, unfortunately, fail to capitalize on the powerful potential of full-parameter fine-tuning. In this work, we propose QFT, a novel quantized full-parameter tuning framework for LLMs that enables memory-efficient fine-tuning without harming performance. Our framework incorporates two novel ideas: (i) we adopt the efficient Lion optimizer, which eliminates the memory usage of variances and enjoys the inherent advantage of performing robust quantization; and (ii) we quantize all model states and store them as integer values, and present a gradient backpropagation and parameter update scheme of the quantized values. As a result, QFT reduces the model state memory to 21% of the standard solution while achieving comparable performance, e.g., tuning a LLaMA-7B model only requires <30GB memory, met by a single A6000 GPU.