Theoretical Performance Guarantees for Partial Domain Adaptation via Partial Optimal Transport

Jayadev Naram, Fredrik Hellström, Ziming Wang, Rebecka Jörnsten, Giuseppe Durisi

Abstract: In many scenarios of practical interest, labeled data from a target distribution are scarce while labeled data from a related source distribution are abundant. One particular setting of interest arises when the target label space is a subset of the source label space, leading to the framework of partial domain adaptation (PDA). Typical approaches to PDA involve minimizing a domain alignment term and a weighted empirical loss on the source data, with the aim of transferring knowledge between domains. However, a theoretical basis for this procedure is lacking, and in particular, most existing weighting schemes are heuristic. In this work, we derive generalization bounds for the PDA problem based on partial optimal transport. These bounds corroborate the use of the partial Wasserstein distance as a domain alignment term, and lead to theoretically motivated explicit expressions for the empirical source loss weights. Inspired by these bounds, we devise a practical algorithm for PDA, termed WARMPOT. Through extensive numerical experiments, we show that WARMPOT is competitive with recent approaches, and that our proposed weights improve on existing schemes.

Lay Summary:

Machine learning systems often rely on large amounts of labeled data to learn effectively. But in many real-world situations, such data are only available in related—but not identical—contexts. For example, a model trained to recognize a wide range of objects may need to be adapted to a new setting where only a smaller set of objects is relevant, and labeled examples in the new setting are scarce. This challenge is known as partial domain adaptation.

In our work, we take a fresh look at how to best transfer knowledge from one setting to another in these cases. We provide new theoretical insights into how and why certain adaptation techniques work, especially those that try to align patterns between the old and new data. Based on our findings, we propose a practical new method, called WARMPOT, that learns more effectively which parts of the old data are actually useful in the new setting.

Our experiments show that WARMPOT performs as well as or better than current top methods, and can even reach state-of-the-art results on a widely used benchmark. Overall, this work helps make machine learning models more adaptable and reliable when working with limited data in real-world applications.

Link To Code: https://github.com/JayD2106/WARMPOT

Primary Area: Theory->Domain Adaptation and Transfer Learning

Keywords: Partial Domain Adaptation, Optimal Transport, Generalization Bounds

Submission Number: 11867