# CoLo: Contrastive Localizing and Mitigatng Option Bias in LLMs

## Install

```
conda create -n icd python==3.10
conda activate icd
pip install -r requirements.txt
cd ./transformers
pip install --editable ./
```

## Run
Try the following command to test our method to Localizing head
```
cd ./exp_scripts
bash find_layer_head.sh
```

## Evaluate
```
cd ./exp_scripts
bash mmlu.sh
```

Then get a json file. Evalute in `src/ipynb/choice_analysis.ipynb`

## Reference

Modify from [ICD method](https://github.com/HillZhang1999/ICD/tree/main)