BottleneckMLP: Graph Explanation via Implicit Information Bottleneck

ICLR 2026 Conference Submission21969 Authors

Keywords: Graph Neural Networks, Explainability, Information Bottleneck, Mutual Information, Representation Learning

TL;DR: BottleneckMLP is a general module which implicitly enforces IB, effectively replacing explicit IB loss terms in existing ante-hoc graph explanation frameworks.

Abstract: The success of Graph Neural Networks (GNNs) in modeling unstructured data has heightened the demand for explainable AI (XAI) methods that provide transparent, interpretable rationales for their predictions. A prominent line of work leverages the Information Bottleneck (IB) principle, which frames explanation as optimizing for representations that maximize predictive information $I(Z;Y)$ while minimizing input dependence $I(X;Z)$. We show that explicit IB-based losses in GNN explainers provide little benefit beyond standard training: the fitting and compression phases of IB emerge naturally, whereas the variational bounds used in explicit objectives are too loose to meaningfully constrain mutual information. To address this, we propose BottleneckMLP, a simple architectural module that implicitly enforces the IB principle. By injecting Gaussian noise inversely scaled by node importance, followed by architectural compression, BottleneckMLP amplifies the reduction of $I(X;Z)$ while increasing $I(Z;Y)$. This yields embeddings where important nodes remain structured and clustered, while unimportant nodes drift toward Gaussianized, high-entropy distributions, consistent with progressive information loss under IB. BottleneckMLP integrates seamlessly with current explainers, as well as subgraph recognition tasks, replacing explicit IB terms and consistently improving predictive performance and explanation quality across diverse datasets.

Primary Area: interpretability and explainable AI

Submission Number: 21969