# PreCoT: Problem Representation Enhances Reasoning in Large Language Models
Official Implementation of PreCoT.

### Evaluation

Follow the steps below to reproduce the evaluation results as presented in our paper. You'll need to enter your API key in the `run.sh` before proceeding. The available models, datasets, and frameworks are listed in the `run.sh`.
```
pip3 install -r requirements.txt
chmod +x run.sh
./run.sh [MODEL_NAME] [DATASET_NAME] [FRAMEWORK_NAME]
```

### License Notice
We formatted all of the datasets used in our evaluation in the Big-Bench Hard style. The datasets we used are each under the following licenses.

- [GSM8K](https://github.com/openai/grade-school-math): MIT license.
- [GSM-IC](https://github.com/openai/grade-school-math): Modified version of GSM8K.
- [SVAMP](https://github.com/arkilpatel/SVAMP): MIT license.
- [AQuA-RAT](https://github.com/google-deepmind/AQuA): Apache License 2.0.
- [StrategyQA](https://github.com/eladsegal/strategyqa) ([Big-Bench](https://github.com/google/BIG-bench) split): MIT license (StrategyQA), Apache License 2.0 (Big-Bench)
- [SocialIQA](https://leaderboard.allenai.org/socialiqa/submissions/get-started): CC-BY-4.0.
- [Big-Bench Hard](https://github.com/suzgunmirac/BIG-Bench-Hard) tasks: MIT license.