Merge-Friendly Post-Training Quantization for Multi-Target Domain Adaptation

Juncheol Shin, Minsang Seok, Seonggon Kim, Eunhyeok Park

TL;DR: We analyze the impact of quantization on model merging through the lens of error barriers and highlight key considerations for quantization in the context of multi-target domain adaptation via model merging.

Abstract: Model merging has emerged as a powerful technique for combining task-specific weights, achieving superior performance in multi-target domain adaptation. However, when applied to practical scenarios, such as quantized models, new challenges arise. In practical scenarios, quantization is often applied to target-specific data, but this process restricts the domain of interest and introduces discretization effects, making model merging highly non-trivial. In this study, we analyze the impact of quantization on model merging through the lens of error barriers. Leveraging these insights, we propose a novel post-training quantization, HDRQ - Hessian and distant regularizing quantization - that is designed to consider model merging for multi-target domain adaptation. Our approach ensures that the quantization process incurs minimal deviation from the source pre-trained model while flattening the loss surface to facilitate smooth model merging. To our knowledge, this is the first study on this challenge, and extensive experiments confirm its effectiveness.

Lay Summary:

Merging models trained on different datasets can produce a single model that performs well across all data. However, in real-world scenarios, models are often quantized to low precision to meet resource constraints-especially on edge devices like mobile phones. Unfortunately, merging these quantized models typically results in a loss of performance.

We explored whether improved quantization techniques could help preserve model quality when models are merged. Using a metric called the error barrier, we analyzed the behavior of quantized models during merging and found that encouraging the weights of each quantized model to remain close and converge to flatter regions of the loss landscape can lead to better merged performance. Based on these insights, we developed HDRQ (Hessian and Distance Regularization Quantization), a novel quantization algorithm that guides models toward flatter minima and keeps their weights close together.

HDRQ enables effective merging of low-precision models without significant degradation in quality. This makes it possible to combine models directly on small devices without server involvement. As a result, users can generate versatile models that work across mutiple datasets in demand, all while staying within tight resource budgets

Link To Code: https://github.com/ewsn1593/HDRQ

Primary Area: Deep Learning->Algorithms

Keywords: Post-Training Quatnization, Multi-target Domain Adaptation, Model Merging

Submission Number: 10825