Multi-Fidelity Fine-Tuning of Pre-Trained Language Models

Francesco Tonolini, Jordan Massiah, Nikolaos Aletras, Gabriella Kazai

Keywords: language models, foundation models, multi-fidelity, learning with noisy labels, fine-tuning

TL;DR: A novel method to fine-tune language models with data of mixed quality.

Abstract: We consider the problem of fine-tuning pre-trained language models with a small amount of trusted data (high-fidelity) and a larger amount of data with noisy labels (low-fidelity). We propose Multi-Fidelity Fine-Tuning (MFFT), a novel approach which implicitly determines for new inputs when we can rely on information from high-fidelity data and when instead we need to fall back on knowledge from low-fidelity data. MFFT does not require any architecture changes to the base model and simply provides its fine-tuned version that can be easily deployed for inference. We extensively benchmark MFFT on various classification tasks against several baselines, with both simulated label noise, and in realistic scenarios with LLM generated data. MFFT consistently improves performance compared to using trusted data alone and outperforms all baselines across experiments with macro F1-score improvements of 2-4%. Finally, it provides substantial improvements in uncertainty calibration with expected calibration error (ECE) reductions of 40-60% compared to the best baselines.

Primary Area: foundation or frontier models, including LLMs

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 10754