# README

You can find the scripts of running gpt-3.5-turbo or causal language models in `scripts`. Note that for local running you should set the env variable `HF_MODELS` that indicates the save folder of LLMs.

For PriDe, simply modify the hyperparameters in `debias_pride.py` and run it.

If you find this repository useful or our work is related to your research, please kindly cite it:

```latex
@inproceedings{
  llm-mcq-bias,
  title={Large Language Models Are Not Robust Multiple Choice Selectors},
  author={Chujie Zheng and Hao Zhou and Fandong Meng and Jie Zhou and Minlie Huang},
  booktitle={The Twelfth International Conference on Learning Representations},
  year={2024},
  url={https://openreview.net/forum?id=shr9PXz7T0}
}
```

## Experimental Results

Our experimental results are released in another data repo: https://github.com/chujiezheng/LLM-MCQ-Bias_data 
