Compositionality Unlocks Deep Interpretable Models

AAAI 2025 Workshop CoLoRAI Submission8 Authors

Keywords: Neural Network, Tensor Network, Mechanistic Interpretability, Compositional AI, Model Interpretation, Model Compression, Low-Rank Representations

TL;DR: The X-net architecture combines the compositional structure of tensor networks with the expressivity of deep neural networks, enabling diagonalisation, truncation and intrinsic interpretation.

Abstract: We propose $\chi$-net, an intrinsically interpretable architecture combining the compositional multilinear structure of tensor networks with the expressivity and efficiency of deep neural networks. $\chi$-nets retain equal accuracy compared to their baseline counterparts. Our novel, efficient diagonalisation algorithm, ODT, reveals linear low-rank structure in a multilayer SVHN model. We leverage this toward formal weight-based interpretability and model compression.

Submission Number: 8