Identifying Challenges for Generalizing to the Pearl Causal Hierarchy on Images

Matej Zečević, Moritz Willig, Devendra Singh Dhami, Kristian Kersting

Keywords: causality, computer vision, domain generalization, challenges

Abstract: Towards the ultimate goal of AI that features agents capable of generalizing to unseen domains, many researchers have recently voiced their support towards Pearl's counterfactual theory of causation as a key milestone. As in any other growing subfield, patience seems to be a virtue since significant progress on integrating notions from both fields takes time, yet, major challenges such as the lack of ground truth benchmarks or a unified perspective on classical problems such as computer vision seem to hinder the momentum of the research movement. This work takes a first, informal look at the Pearl Causal Hierarchy (PCH) for image data. We moreover discuss several challenges that naturally arise when applying key concepts from causality to the study of image data.

Submission Number: 1