Transfer Learning for Finetuning Large Language Models

Tobias Strangmann, Lennart Purucker, Jörg K.H. Franke, Ivo Rapant, Fabio Ferreira, Frank Hutter

Keywords: meta-learning, transfer learning, synthetic dataset, finetuning, LLMs, meta-optimization, Bayesian optimization

Abstract:

As the landscape of large language models expands, efficiently finetuning for specific tasks becomes increasingly crucial.
At the same time, the landscape of parameter-efficient finetuning methods rapidly expands.
Consequently, practitioners face a multitude of complex choices when searching for an optimal finetuning pipeline for large language models.
To reduce the complexity for practitioners, we investigate transfer learning for finetuning large language models.
We aim to transfer knowledge about configurations from related finetuning tasks to a new task.
In this work, we transfer learn finetuning by meta-learning performance and cost surrogate models for grey-box meta-optimization from a new meta-dataset.
Counter-intuitively, we propose to rely only on transfer learning for new dataset.
Thus, we do not use task-specific Bayesian optimization but prioritize knowledge transferred from related tasks over task-specific feedback.
We evaluate our method on eight synthetic question-answer datasets and a meta-dataset consisting of 1,800 runs of finetuning Microsoft's Phi-3.
Our transfer learning is superior to zero-shot, default finetuning, and meta-optimization baselines.
Our results demonstrate the transferability of finetuning to adapt large language models more effectively.

Submission Number: 32