# Branching Factor for LLM

## Dependencies

Create a conda environment, and then install the dependencies from `requirements_conda.txt` and install our package in editable mode:
```bash
conda create -p ./env --file requirements_conda.txt
conda activate ./env
pip install -e .
```

Download chat_templastes used to set up aligned model prompting (for backward compatibility to older version of HF):
```bash
git clone https://github.com/chujiezheng/chat_templates.git
```

Update: we recently find CoT hub provides performance-verified prompt templates for various tasks, which are more reliable than the ones in `chat_templates`. 
We recommend using them instead. That repo is released under MIT license. We have adjusted part of our codes to accomodate it.  

Download prompt template files from Chain-of-Thoughts hub:
```bash
git clone https://github.com/FranxYao/chain-of-thought-hub.git
```
But in general, you can put any prompt template files in the `prompt_templates` folder under `root_path` (see instruction below to set it up), and the code will automatically load them.

## Usage
1. Setup Environment Variable

    In order to run the code, you need to set the environment variable:

    In `uncertainty_quantification/consts.py`, set `root_path` to be the absolute path to the current project directory.

    Also change all `/path/to/project` in all shell scripts to run the codes.

2. Run shell scripts under `mmlu`, `cognac`, `storytelling`, `language_modeling` to replicate experiments in the paper. `stepX_xxx.sh` specify the ordering to run the scripts. `visualization` provide codes to create figures in the paper.