Reproducibility study:Generative causal explanations of black-box classifiers

Gerson Foks, Tomáš Fábry, Simon Mariani, Friso Verweij

Keywords: Black Box, Causal explanation, reproducing

Abstract:

centering Summary
Scope of reproducibility
In the paper by O’Shaughnessy et al. the authors claim to have created a conceptual framework that is able to explain black-box classifiers using latent variables and an appropriate generative model. They also claim that this framework works for classifier and that the latent variables are disentangled (the latent variable describe independent features).

Methodology
In this report, we re-implement their code and aim to generate results similar to theirs. Additionally, we extent their research by testing the framework on new datasets. These datasets are the complete MNIST set (in constrast to just the digits 3 and 8) and the CIFAR-10 dataset. Finally, we purposefully used an poorly performing model to discover if the framework is able to find out why the model struggles.

Results
It was possible to reproduce the results presented in the paper. Furthermore, the framework was able to highlight the cause of the poor performance of the purposefully poorly performing model. When applying their framework to a more complex dataset, the framework proved to be effective.

What was easy?
Although the original code did not work immediately once the code was fixed it was easy to get similar results.
What was difficult?
The available code of the original authors did not work completely. There where some issues with broken references and lengthy code which could be hard to understand therefore reusing parts and rewriting others was more time efficient than debugging the available code. Furthermore understanding algorithm 1 in their paper was difficult as many details were left out, such as how to choose an appropriate step size.

Communication with original authors
We have not communicated with the original authors.

Paper Url: https://openreview.net/forum?id=tdG6Fa3Y6hq&referrer=%5BML%20Reproducibility%20Challenge%202020%5D(%2Fgroup%3Fid%3DML_Reproducibility_Challenge%2F2020)