Measuring Invariance in Representation Learning: A Robust Evaluation Framework

ICLR 2026 Conference Submission14968 Authors

Keywords: invariant representation learning, domain generalization

Abstract: Distribution shifts challenge reliable deployment even when in-distribution accuracy is high. Invariant representation learning aims to mitigate this challenge by learning feature spaces that remain invariant across diverse out-of-distribution (OOD) scenarios. However, a critical gap exists in directly and efficiently evaluating the true invariance of learned representations across varied environments. To address this, we introduce DRIC, a novel and computationally efficient criterion designed for the direct assessment of invariant representation performance. DRIC establishes a formal link between the conditional expectation of invariant predictors and environmental diversity through the density ratio, providing a theoretically sound and practical evaluation framework. We validate the effectiveness and robustness of DRIC through extensive numerical experiments on both synthetic and real-world datasets, demonstrating its utility in quantifying and comparing the invariance of learned representations, ultimately contributing to the development of more robust machine learning models.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 14968