Towards Robust Classification Model by Counterfactual and Invariant Data Generation

Chun-Hao Chang, George Alexandru Adam, Anna Goldenberg

Abstract:

Despite the success of machine learning applications in science, industry, and society in general, many approaches
are known to be non-robust, often relying on spurious correlations to make predictions. Spuriousness occurs when
some features correlate with labels but are not causal; relying on such features prevents models from generalizing
to unseen environments where such correlations break. In this work, we focus on image classification and propose two
data generation processes to reduce spuriousness. Given human annotations of the subset of the features responsible
(causal) for the labels (e.g. bounding boxes), we modify this causal set to generate a surrogate image that no longer has
the same label (i.e. a counterfactual image). We also alter non-causal features to generate images still recognized as
the original labels, which helps to learn a model invariant to these features. In several challenging datasets, our data
generations outperform state-of-the-art methods in accuracy when spurious correlations break, and increase the saliency
focus on causal features providing better explanations.