Algorithmic causal structure emerging through compression

Liang Wendong, Simon Buchholz, Bernhard Schölkopf

Keywords: algorithmic causality, compression, symmetry, Kolmogorov complexity

TL;DR: This paper proposes a framework where causality emerges through compression across multiple environments, employing an algorithmic notion of causality to handle cases lacking identifiable causal structures, and discussing links to machine learning.

Abstract: We explore the relationship between causality, symmetry, and compression. We build on and generalize the known connection between learning and compression to a setting where causal models are not identifiable. We propose a framework where causality emerges as a consequence of compressing data across multiple environments. We define algorithmic causality as an alternative definition of causality when traditional assumptions for causal identifiability do not hold. We demonstrate how algorithmic causal and symmetric structures can emerge from minimizing upper bounds on Kolmogorov complexity, without knowledge of intervention targets. We hypothesize that these insights may also provide a novel perspective on the emergence of causality in machine learning models, such as large language models, where causal relationships may not be explicitly identifiable.

Submission Number: 117