Diversity Sampling Regularization for Multi-Domain Generalization

TMLR Paper6478 Authors

Abstract:

Domain Generalization (DG) seeks to create models that can successfully generalize to new,
unseen target domains without the need for target domain data during training. Traditional
approaches often rely on data augmentation or feature mixing techniques, such as MixUp;
however, these methods may fall short in capturing the essential diversity within the feature
space, resulting in limited robustness against domain shifts. In this research, we revisit the
importance of diversity in DG tasks and propose a simple yet effective method to improve DG
performance through diversity-sampling regularization. Specifically, we calculate entropy
values for input data to assess their prediction uncertainty, and use these values to guide
sampling through Determinantal Point Process (DPP), which prioritizes selecting data sub-
sets with high diversity. By incorporating DPP-based diversity sampling as a regularization
strategy, our framework enhances the standard Empirical Risk Minimization (ERM) objec-
tive, promoting the learning of domain-agnostic features without relying on explicit data aug-
mentation. We empirically validate the effectiveness of our method on standard DG bench-
marks, including PACS, VLCS, OfficeHome, TerraIncognita, and DomainNet, and through
extensive experiments show that it consistently improves generalization to unseen domains
and outperforms widely used baselines and S.O.T.A without relying on any task-specific
heuristics.

Submission Type: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission:

Considering the colelctive suggestion of all three reviewers, we have updated the manuscript with the following new changes.

1. Addition of new figure in Fig. 5(f).
2. Addition of algorithm and the high level discussion of the algorithm in the appendix section (A.1 Trainign DSR: Algorithm) of the paper.
3. Edited Table. 4 (section A.5) of appendix. This edit include the addition of total number of classes that reflects the scalability of the DSR to large datasets.
4. Editing and correction of minor typos and erratas as reviewed by the reviewers.
5. Editing of the Section 3.1 of the paper. This edit includes the explanation of the domain discrepancy term $\mathcal{C}$.

Assigned Action Editor: ~Qi_CHEN6

Submission Number: 6478