Predicting the Performance of Foundation Models via Agreement-on-the-line

Rahul Saxena, Aman Mehra, Taeyoun Kim, Christina Baek, J Zico Kolter, Aditi Raghunathan

Keywords: robustness, OOD performance estimation, foundation model safety

TL;DR: Unsupervised estimation of finetuned foundation model performance under distribution shift

Abstract: Estimating out-of-distribution (OOD) performance is critical to safely deploying machine learning models. Recently, Baek et al showed that the phenomenon ``agreement-on-the-line'' can be a reliable method for predicting OOD accuracy of models in an ensemble consisting largely of CNNs trained from scratch. However, it is now increasingly common to lightly fine-tune foundation models, and it is unclear whether such fine-tuning is sufficient to produce enough diversity in models for such agreement-based methods to work properly. In this paper, we develop methods for reliably applying agreement-on-the-line-based performance estimation to fine-tuned foundation models. In particular, we first study the case of fine-tuning a single foundation model, where we extensively study how different types of randomness (linear head initialization, hyperparameter selection, data subsetting, and data shuffling) contribute to the agreement-on-the-line of the resulting model sets; we find, somewhat surprisingly, that it is typically possible to obtain strong agreement via random initialization of the linear head alone. Next, we study how multiple foundation models, pretrained on different data sets but fine-tuned on the same task, may or may not produce agreement; we show, again rather surprisingly, that the diversity of such models is already sufficient and not too disparate for them to all lie on the same agreement line. In total, these methods enable reliable and efficient estimation of OOD accuracy for fine-tuned foundation models, without leveraging any labeled OOD data.

Submission Number: 59