Target Conditioned Representation Independence (TCRI); from Domain-Invariant to Domain-General Representations

Olawale Elijah Salaudeen, Oluwasanmi O Koyejo

Keywords: Domain Generalization, Out-of-distribution Generalization, Transfer Learning, Distribution Shift, Covariate Shift

TL;DR: We propose a Target Conditioned Representation Independence (TCRI) objective to learn domain-general representations and predictors.

Abstract: We propose a Target Conditioned Representation Independence (TCRI) objective for domain generalization. TCRI addresses the limitations of existing domain generalization methods due to incomplete constraints. Specifically, TCRI implements regularizers motivated by conditional independence constraints that are sufficient to strictly learn complete sets of invariant mechanisms, which we show are necessary and sufficient for domain generalization. Empirically, we show that TCRI is effective on both synthetic and real-world data. TCRI is competitive with baselines in average accuracy while outperforming them in worst-domain accuracy, indicating desired cross-domain stability.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Deep Learning and representational learning