Reproducibility Study of ”Label-Free Explainability for Unsupervised Models”

Julius Wagenbach, Gergely Papp, Niklas Mather, Laurens Jan de Vries

Keywords: Explainable AI, Unsupervised Learning, Feature Importance, Example Importance

TL;DR: We reproduce the paper "Label-Free Explainability for Unsupervised Models". For the most part, the results are reproducible. We extend by testing on more datasets and add new methodology to test the claims..

Abstract: In this work, we present our reproducibility study of "Label-Free Explainability for Unsupervised Models", a paper that introduces two post‐hoc explanation techniques for neural networks: (1) label‐free feature importance and (2) label‐free example importance. Our study focuses on the reproducibility of the authors’ most important claims: (i) perturbing features with the highest importance scores causes higherlatent shift than perturbing random pixels, (ii) label‐free example importance scores help to identify training examples that are highly related to a given test example, (iii) unsupervised models trained on different tasks show moderate correlation among the highest scored features and (iv) low correlation in example scores measured on a fixed set of data points, and (v) increasing the disentanglement with β in a β‐VAE does not imply that latent units will focus on more different features. We reviewed the authors’ code, checked if the implementation of experiments matched with the paper, and also ran all experiments. The results are shown to be reproducible. Moreover, we extended the codebase in order to run the experiments on more datasets, and to test the claims with other experiments.

Paper Url: https://arxiv.org/abs/2203.01928

Paper Venue: Other venue (not in list)

Confirmation: The report pdf is generated from the provided camera ready Google Colab script, The report metadata is verified from the camera ready Google Colab script, The report contains correct author information., The report contains link to code and SWH metadata., The report follows the ReScience latex style guides as in the Reproducibility Report Template (https://paperswithcode.com/rc2022/registration)., The report contains the Reproducibility Summary in the first page., The latex .zip file is verified from the camera ready Google Colab script

Venue Name: MLRC 2022

Journal: ReScience Volume 9 Issue 2 Article 24

Doi: https://www.doi.org/10.5281/zenodo.8173711

Code: https://archive.softwareheritage.org/swh:1:dir:0df843f2fc8c0868968429afb8908db7d3a76a3c