DISPEL: Domain Generalization via Domain-Specific Liberating

Chia-Yuan Chang, Yu-Neng Chuang, Guanchu Wang, Samson Zhou, Vladimir Braverman, Mengnan Du, Na Zou

Keywords: Domain Generalization, Transfer Learning

TL;DR: A post-processing framework for improving generalization of frozen prediction models.

Abstract: Domain generalization aims to learn a generalization model that can perform well on unseen test domains by only training on limited source domains. However, existing domain generalization approaches often bring in prediction-irrelevant noise or require the collection of domain labels. To address these challenges, we consider the domain generalization problem from a different perspective by categorizing the underlying feature groups into domain-shared and domain-specific features. Nevertheless, domain-specific features are difficult to be identified and distinguished from the input data. In this work, we propose $\underline{\mathrm{D}}$oma$\underline{\mathrm{I}}$n-$\underline{\mathrm{SPE}}$cific $\underline{\mathrm{L}}$iberating (DISPEL), a post-processing fine-grained masking approach that can filter out undefined and indistinguishable domain-specific features in the embedding space. Specifically, DISPEL utilizes a mask generator that produces a unique mask for each input data to filter domain-specific features. The DISPEL framework is highly flexible to apply to fine-tuned models. We derive a generalization error bound to guarantee the generalization performance by optimizing a designed objective loss. The experimental results on five benchmarks demonstrate that DISPEL outperforms existing methods and can further generalize various algorithms.

Primary Area: general machine learning (i.e., none of the above)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 409