Continuous Language Model Interpolation yields Dynamic and Controllable Text Generation

Sara Kangaslahti, David Alvarez-Melis

Abstract: As large language models (LLMs) have gained popularity for a variety of use cases, making them adaptable and controllable has become increasingly important, especially for user-facing applications. In particular, linear interpolation between model parameters forms the backbone for many recent approaches to adapting models to user preferences. While the existing literature on LLM adaptation primarily focuses on finding methods that optimize for some set of performance criteria or user preferences, here we instead seek to better understand and characterize the behavior of dense, continuous interpolation between models. Specifically, we use low-rank updates to fine-tune a base model to various different domains, yielding a set of anchor models with distinct generation profiles. Then, we use the weight updates of these anchor models to parametrize the entire (infinite) class of models contained within their convex hull. We empirically show that varying the interpolation weights yields predictable and consistent change in the model outputs with respect to all of the controlled attributes simultaneously. We find that there is little entanglement between most attributes and identify and discuss the pairs of attributes for which this is not the case. Our results suggest that parameter merging facilitates flexible model adaptation due to its predictable behavior within the full interpolation region.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission: Camera ready

Video: https://youtu.be/mm9zi-y7amg

Code: https://github.com/skangasl/continuous-lm-interpolation

Assigned Action Editor: ~Tal_Schuster1

Submission Number: 4849