
# Install
```bash
conda create -n masom python=3.9
pip install -r requirements.txt
```



# Run
1. edit `api.py` and add your api-key 
2. run the following command:
```bash
python generate.py --role 0 --dataset mmlu --repeat 1 --turn 3 --api_idx 0 --api_account 0 --experiment_type main --n_case 50 --model gpt-3.5-turbo --agent 3 --save_path ./
```
3. evaluate. run the following command:
```bash
python evaluate.py --dataset mmlu --metric acc --repeat 1 --experiment_type main --turn 3 --agent 3 --n_case 50 --which_turn 9 --turn 3
```