Provable Domain Generalization via Information Theory Guided Distribution Matching
Primary Area: learning theory
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: domain generalization, out-of-distribution generalization, information theory
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Abstract: Domain generalization (DG) aims to learn predictors that perform well on unseen data distributions by leveraging multiple related training environments. To this end, DG is commonly formulated as an average or worst-case optimization problem, which however either lacks robustness or is overly conservative. In this work, we propose a novel probabilistic framework for DG by minimizing the gap between training and test-domain population risks. Our formulation is built upon comprehensive information-theoretic analysis and enables direct optimization without stringent assumptions. Specifically, we establish information-theoretic upper bounds for both source and target-domain generalization errors, revealing the key quantities that control the capability of learning algorithms to generalize on unseen domains. Based on the theoretical findings, we propose Inter-domain Distribution Matching (IDM) for high-probability DG by simultaneously aligning inter-domain gradients and representations, and Per-sample Distribution Matching (PDM) for high-dimensional and complex data distribution alignment. Extensive experimental results validate the efficacy of our methods, showing superior performance over various baseline methods.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
Supplementary Material: zip
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 4934
Loading