[Re] GNNInterpreter: A probabilistic generative model-level explanation for Graph Neural Networks

TMLR Paper2196 Authors

Abstract:

Graph Neural Networks have recently gained recognition for their performance on graph
machine learning tasks. The increasing attention on these models’ trustworthiness and
decision-making mechanisms has instilled interest in the exploration of explainability tech-
niques, including the model proposed in "GNNInterpreter: A probabilistic generative model-
level explanation for Graph Neural Networks." (Wang & Shen (2022)). This work aims to
reproduce the findings of the original paper, by investigation the main claims made by its
authors, namely that GNNInterpreter (i) generates faithful and realistic explanations with-
out requiring domain-specific knowledge, (ii) has the ability to work with various node and
edge features, (iii) produces explanations that are representative for the target class and
(iv) has a much lower training time compared to XGNN, the current state-of-the-art model-
level GNN explanation technique. To reproduce the results, we make use of the open-source
implementation and we test the interpreter on the same datasets and GNN models as in
the original paper. We conduct an enhanced quantitative and qualitative evaluation, and
additionally we extend the original experiments to include another real-world dataset. Our
results show that we are not able to validate the first claim, due to significant hyperpa-
rameter and seed variation, as well as due to training instability. Furthermore, we partially
validate the second claim by testing on datasets with different node and edge features, but
we reject the third claim due to GNNInterpreter’s failure to outperform XGNN in producing
dataset aligned explanations. Lastly, we are able to confirm the last claim.

Submission Length: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Nadav_Cohen1

Submission Number: 2196