# RCOT: Detecting and Rectifying Factual Inconsistency in Reasoning by Reversing Chain-of-Thought

## Method
“RCoT, a novel method for detecting and rectifying condition hallucination, overlooking and question misinterpretation in CoT“



The illustration of active-prompt is shown in the figure below.


## Requirements
Make sure you have Python>=3.8 installed on your machine.
pip install torch==1.8.2+cu111 torchtext==0.9.2 -f https://download.pytorch.org/whl/lts/1.8/torch_lts.html

`pip install openai`
Put your `OpenAI API KEY` to the `API_KEY` variable of `utils.py`.


## Quick Start
For a quick start, you can directly run inference with RCoT(proposed in the paper).

# zero-shot-cot
```shell
python inference.py --dataset="gsm8k" --model="gpt-3.5-turbo" --method="zero_shot_cot" --random_seed=99 --multipath=1 --temperature=0 --api_time_interval=0.5 --test_size=256 --shot=0 --flag="gpt"
```
# few-shot-cot(Active-prompting)
```shell
python inference.py --dataset="asdiv" --model="gpt-3.5-turbo" --method="few_shot_cot" --random_seed=24 --multipath=1 --temperature=0 --api_time_interval=0.5 --test_size=256 --shot=4 --flag="gpt" --prompt_path=./inference_prompts/gsm8k_k=10 --output_dir=./few_shot_results

```

## Important arguments
   * `--dataset`: The name of a dataset. `choices = [gsm8k, svamp, aqua, asdiv, singleeq, addsub, Date]`.
   * `--model`: GPT-3 model. `choices = [gpt-3.5-turbo]`.
   * `--method`: zero-shot-cot or few-shot-cot.
   * `--test_size`: number of test questions.
   * `--api_time_interval`: the time interval when requesting GPT-3 API. Suggested interval is larger than 1 second.
   * `--shot`: number of demonstration examples.
