Learning Causally Emergent Representations
Keywords: Representation Learning, Information Theory, Emergence
TL;DR: Machine learning method for discovering emergent variables in time series data that leverages a recent information-theoretic characterisation of emergence and advances in mutual information estimation from data with neural networks.
Abstract: Cognitive processes usually take place at a macroscopic scale in systems characterised by emergent properties, which make the whole `more than the sum of its parts.' While recent proposals have provided quantitative, information-theoretic metrics to detect emergence in time series data, it is often highly non-trivial to identify the relevant macroscopic variables a priori. In this paper we leverage recent advances in representation learning and differentiable information estimators to put forward a data-driven method to find emergent variables. The proposed method successfully detects emergent variables and recovers the ground-truth emergence values in a synthetic dataset. This proof-of-concept paves the ground for future analyses uncovering the emergent structure of cognitive representations in biological and artificial intelligence systems.
Submission Number: 10
Loading