WILTing Trees: Interpreting the Distance Between MPNN Embeddings

Masahiro Negishi, Pascal Welke, Thomas Gärtner

Keywords: Weisfeiler Leman test, Graph Neural Networks, Interpretability, Graph metric, Graph distance

TL;DR: We show that message passing neural networks (MPNNs) are implicitly trained to respect graph functional distances, and introduce the weighted Weisfeiler Leman Labeling Tree (WILT) to identify subgraphs that MPNNs consider functionally important.

Abstract:

We investigate the distance function implicitly learned by message passing neural networks (MPNNs) on specific tasks.
Our goal is to capture the functional distance that is implicitly learned by an MPNN for a given task.
This contrasts previous work which relates MPNN distances on arbitrary tasks to structural distances that ignore the task at hand.
To this end, we distill the distance between MPNN embeddings into an interpretable graph distance.
Our distance is an optimal transport on the Weisfeiler Leman Labeling Tree (WILT), whose edge weights reveal subgraphs that strongly influence the distance between MPNN embeddings.
Moreover, it generalizes the metrics of two well-known graph kernels and is computable in linear time.
Through extensive experiments, we show that MPNNs define the relative position of embeddings by focusing on a small number of subgraphs known by domain experts to be functionally important.

Primary Area: learning on graphs and other geometries & topologies

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 10046