Causal-guided strength differential independence sample weighting for out-of-distribution generalization

Haoran Yu, Weifeng Liu, Yingjie Wang, Bao-di Liu, Dapeng Tao, Honglong Chen

Abstract: Most machine learning methods often perform vulnerably in the real open world due to the unknown distribution shifts between training and testing distribution. Out-of-Distribution (OOD) generalization aims to make stable predictions under unknown distribution shifts by exploring invariant patterns to address this problem. One of the representative methods is independence sample weighting learning. It eliminates spurious correlations to make the model explore the true relationship between features and labels for stable prediction by learning a set of sample weights to eliminate dependencies between features. However, existing independence sample weighting methods roughly eliminate the correlation between all features, resulting in the loss of critical information and affecting the model’s performance. To address this problem, we propose a causal-guided independence sample weighting (CIW) algorithm. CIW first evaluates the causal effect of features on labels by constructing a cross domain-invariant directed acyclic graph (DAG). Subsequently, it generates a strength guiding mask based on the causal effect to differentially eliminate the correlation between different features avoiding redundant elimination of correlations between causal features. We perform extensive experiments in different experimental settings and experimental results demonstrate the effectiveness and superiority of our method.