DRIP: Invariance-preserving Data Reduction for Domain Generalization

Mingze Sun, Zhoutong Ye, Zhe He, Yuntao Wang, Chun Yu, Yuanchun Shi

Keywords: domain generalization, data augmentation, causality, invariance

TL;DR: We propose DRIP, a data augmentation paradigm that theoretically proven to improve domain generalization. Evaluations confirm its superior performance on both visual and IMU datasets.

Abstract: Domain Generalization (DG) aims to enable deep models trained on several source domains to generalize to unseen target domains. Existing works assume that data contain both invariant features, whose relationship with label is invariant across domains, and spurious features, which are spuriously correlated to the label. It is widely recognized that retaining invariant features while suppressing spurious ones is critical to achieving DG. However, despite this inspiration, current methods often struggle to guarantee a separation between these features, or their guarantees need relatively strong assumptions of data. In this work, we propose DRIP (Data Reduction with Invariance-Preserving), a data augmentation paradigm theoretically proven to improve domain generalization. We prove that such DRIP can reduce spurious features while preserving invariant features, forcing the model to rely more on generalizable invariant features during training. Following the principles of DRIP, we propose several low cost implementations and verify that one of those indeed meets the criteria of DRIP. Experiments show that it consistently outperforms existing DG baseline in two modalities, vision and IMU.

Primary Area: transfer learning, meta learning, and lifelong learning

Submission Number: 10564