Domain Bridging: Enabling Adaptation without Peeking at Target Data

Xia Sihan, Jing Li, Zou Yuanben, Ivor Tsang, Yew-Soon Ong

Keywords: domain bridging, evaluation-based adaptation, zeroth-order optimization, proprietary target data

TL;DR: Domain Bridging introduces an efficient framework that learns source data perturbations to bridge domain gaps, enabling effective model fine-tuning for target domains without requiring direct access to proprietary target data.

Abstract: Adapting models to target domains with proprietary data remains a challenging problem. One possible setup to enable adaptation is to allow target domain owners to privately evaluate candidate models on their own data. For example, model providers consider how to adjust models to better fit the unseen target data, relying solely on returned model performance. Existing methods adopt Zeroth-Order (ZO) optimization to refine model parameters or employ a two-stage learning process that first identifies the target-related samples in the source data and then retrains the model. However, we find that these methods struggle to generalize well for the target tasks during inference, primarily because of the failure to account for data-statistical shifts between source and target domains. To address this limitation, we introduce the concept of domain bridging in the context of model adaptation for proprietary target data. The core idea is to bridge the domain gap by learning target-aligned perturbations on source data, enabling the fine-tuned model to achieve better performance on target domains. A natural attempt is to extend ZO optimization to this setting. However, this approach fails to produce reliable perturbations on real datasets. To address this, we design a target-aligned, sample-wise perturbation learner, enabling reliable adaptation from performance-only feedback. We provide theoretical convergence guarantees and demonstrate through experiments on five datasets across image and text modalities that our domain bridging method achieves state-of-the-art performance, improving accuracy by approximately 4\%.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 8980