Symmetry Acquisition in Predictive Coding Networks

Adam Shaw, Jiayu Li, Michael Sperling, Michael Kim, Alvin Jin

Keywords: symmetry acquisition, persistent homology, predictive coding networks, representational symmetry, topological simplification, weight-space symmetries, approximate symmetry, invertibility

TL;DR: Predictive coding networks acquire representational symmetries across layers, measurable with persistent homology; when this happens too early, reconstruction worsens, revealing an abstraction–invertibility tradeoff in generative models.

Abstract: Predictive coding networks (PCNs) perform inference through recurrent, bidirectional dynamics, requiring representations that support both abstraction and approximate inversion. We study how PCNs learn representations through the lens of weight-space symmetry, using persistent homology to track when and where PCN layers merge previously distinct regions of the data manifold. This provides a topological probe of representation-space identifications induced by the network’s learned dynamics. We find that this merging occurs across layers, but its timing is strongly controlled by model capacity ($0.72 \leq \rho \leq 0.79$) and activation smoothness. Critically, networks that merge representations earlier perform worse in input reconstruction tasks ($\rho = -0.58$). This reveals a measurable cost of early representational symmetry acquisition in generative architectures, and demonstrates that topological measures can expose structural properties of learned representations that loss values alone cannot.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 70