Class-Conditional Domain Alignment via Kernel Cauchy-Schwarz Mutual Information

Ahmad Abdel-Qader, Anas Chaaban, Ahmed Mostafa Soliman Radwan, Omar Abdelaziz, Mahmoud Soliman, Mohamed S. Shehata

Keywords: Information Theory, Domain Generalization, Mutual Information, out-of-distribution generalization

TL;DR: We derive a closed-form alignment objective from kernel mutual information that enforces class-conditional invariance, ensuring robust generalization to unseen domains.

Abstract: Domain Generalization (DG) seeks to learn models that are robust to unseen distribution shifts, a critical challenge for real-world machine learning applications. A dominant paradigm is to enforce domain invariance by aligning feature distributions from multiple source domains. However, aligning marginal feature distributions indiscriminately can discard critical class-discriminative information, especially when class priors vary across domains. We address this limitation with Domain Alignment via Kernel Cauchy-Schwarz Mutual Information (DAS-MI), a novel framework that advances principled class-conditional alignment. The core principle is to maximize the statistical dependence between same-class features across different domains. We operationalize this using the Cauchy-Schwarz Quadratic Mutual Information (CS-QMI), a powerful information-theoretic measure. Critically, and in contrast to prior work relying on complex approximations or adversarial training, our approach yields a closed-form, non-parametric alignment objective derived from kernel density estimates. This results in a stable loss that integrates seamlessly into deep learning pipelines. Extensive experiments across five benchmark datasets demonstrate performance comparable to state-of-the-art methods. DAS-MI offers a theoretically-grounded and practically efficient solution to domain generalization that robustly preserves discriminative information.

Primary Area: learning theory

Submission Number: 8026