
# VSCoDe: Visual-Augmentation Selection for Contrastive Decoding

### Environment Setup

```bash
conda create -yn vscode python=3.9
conda activate vscode
cd VSCoDe
pip install -r requirements.txt
```

### How to Use VSCoDe

We modified LLaVA-1.5, Qwen_VL, and InstructBLIP to apply VSCoDe at the sampling stage according to the algorithm for contrastive decoding.

VSCoDe algorithm can be found in 'vacode_utils/vacode_sampling.py'. The code operates identically to the algorithm written in our Paper.

For the experiment with VSCoDe, please follow the guide below:

- Fill the appropriate dataset image path, question path, and model checkpoint path in 'experiments/eval/paths.py'.

- Configure the hyperparameter you want to experiment by modifying ‘config/llava/main.yaml’.

- Experiment on the data set for which you want an appropriate model according to 'scripts/run_model.sh'.