Align-Deform-Subtract: An interventional framework for explaining object differences

Cian Eastwood, Li Nanbo, Chris Williams

Keywords: causality, goodness-of-fit, object similarity, disentanglement

TL;DR: We propose an interventional framework for explaining object-image differences in terms of the underlying object properties (e.g. pose, shape, appearance).

Abstract: Given two object images, how can we explain their differences in terms of the underlying object properties? To address this question, we propose Align-Deform-Subtract (ADS)---an interventional framework for explaining object differences. By leveraging semantic alignments in image-space as counterfactual interventions on the underlying object properties, ADS iteratively quantifies and removes differences in object properties. The result is a set of "disentangled" error measures which explain object differences in terms of their underlying properties. Experiments on real and synthetic data illustrate the efficacy of the framework.