Dual-stream Feature Augmentation for Domain Generalization
Abstract: Domain generalization (DG) task aims to learn a robust model from source domains that could handle the out-of-distribution (OOD) issue. In order to improve the generalization ability of the model in unseen domains, increasing the diversity of training samples is an effective solution. However, existing augmentation approaches always have some limitations. On the one hand, the augmentation manner in most DG methods is not enough as the model may not see the perturbed features in approximate the worst case due to the randomness, thus the transferability in features could not be fully explored. On the other hand, the causality in discriminative features is not involved in these methods, which is harm for the generalization of model due to the spurious correlations. To address these issues, we propose a Dual-stream Feature Augmentation (DFA) method by constructing some hard features from two perspectives. Firstly, to improve the transferability, we construct some targeted features with domain related augmentation manner. Through the guidance of uncertainty, some hard cross-domain fictitious features are generated to simulate domain shift. Secondly, to take the causality into consideration, the spurious correlated non-causal information is disentangled by an adversarial mask, then the more discriminative features can be extracted through these hard causal related information. Different from previous fixed synthesizing strategy, the two augmentations are integrated into a unified learnable model with disentangled feature strategy. Based on these hard features, contrastive learning is employed to keep the semantics consistent and improve the robustness of the model. Extensive experiments on several datasets demonstrated that our approach could achieve state-of-the-art performance for domain generalization.
Primary Subject Area: [Content] Vision and Language
Secondary Subject Area: [Experience] Multimedia Applications
Relevance To Conference: In this paper, a dual stream feature augmentation is proposed based on the disentanglement framework. Previous work always applied random perturbations on style, however, they do not exploit potentialities of feature transferability. Differently, on the one hand, we construct domain related hard features to explore harder and broader style spaces while preserving semantic consistency. On the other hand, considering that the spurious correlated non-causal information can harm the discriminability of model, the causal related hard features are also constructed to better disentangle the non-causal information hidden in domain-invariant features, thereby improving the generalization and robustness of the model. Trough dual-stream feature augmentation based on a stable feature disentanglement framework, we successfully learn causal related domain-invariant features, and a variety of experiments demonstrate the effectiveness of our method. In the future, we will try to integrate our work with the challenging multimodel learning task.
Submission Number: 1380
Loading