<br/><br/>
<p align="center">
  <img width="250" src="logo.png">
<h3 align="center"><b>Task-agnostic Interpretability Evaluator</b></h3>
<p align="center">PyTorch</p>
<br/><br/>

This repository contains the code and experiments for the paper **["Explanation Faithfulness is Alignment: A Unifying and Ge-
ometric Perspective on Interpretability Evaluation"](Link)** by anonymous et al., 2024. 

[![Getting started!](https://colab.research.google.com/assets/colab-badge.svg)](anonymous)-->

## Citation

If you find this work interesting or useful in your research, use the following Bibtex annotation to cite us:

```bibtex
@InProceedings{}
```

<!--This work has been published ...........-->

## Repository overview

The repository is organised as follows:
- The `src/` folder contains all necessary functions.
- The `nbs/` folder includes notebooks for generating the plots in the paper and for benchmarking experiments.
- The `assets/` folder contains all files to reproduce the experiments.
- The `tests/` folder contains the tests.

All evaluation metrics used in these experiments are implemented in [Quantus](https://github.com/understandable-machine-intelligence-lab/Quantus), a widely-used toolkit for metric-based XAI evaluation. Benchmarking is performed with tools from [MetaQuantus](https://github.com/annahedstroem/MetaQuantus/), a specialised framework for meta-evaluating metrics in explainability.

## Paper highlights 📚


Overview of the three-step GEF evaluation method (Algo. 1) for estimating GEF (Def. 5). First, model parameter scaling is performed to obtain a perturbed model used for evaluation. Second, model and explanation distortions are computed, with the pullback operation. By integrating along the path, as represented by the blue and red areas, we capture the continuous changes in the explanation, providing a more comprehensive measure of how perturbations affect the explanation. Third, distortion vectors are constructed and correlated to obtain a final GEF quality estimate.

</p>
<p align="center">
  <img width="800" src="algorithm.png"> 
</p>

INSERT TEXT

## Installation

Install the necessary packages using the provided [requirements.txt](link):

```bash
pip install -r requirements.txt
```

## Package requirements 

Required packages are:

```setup
python>=3.10.1
torch>=2.0.0
quantus>=0.5.0
metaquantus>=0.0.5
captum>=0.6.0
```

### Thank you

We hope our repository is beneficial to your work and research. If you have any feedback, questions, or ideas, please feel free to raise an issue in this repository. Alternatively, you can reach out to us directly via email for more in-depth discussions or suggestions. 

📧 Contact us: 
- anonymous: [anonymous@gmail.com](mailto:anonymous@gmail.com)

Thank you for your interest and support!


