# Mix-of-Show Experiments

This repository builds on scaled GD project [Riemannian Preconditioned LoRA](https://arxiv.org/abs/2402.02347), and [Mix-of-Show](https://arxiv.org/abs/2305.18292).

## Repository Overview

* [datasets/](datasets) contains the training images.
* [mixofshow/](mixofshow) contains the source code used for models.
* [options/](options) contains training and testing hyperparameters.

## Requirements
```bash
pip install -r requirements.txt
 ```
See the [Mix-of-Show](https://github.com/TencentARC/Mix-of-Show/tree/main) repository for requirement details.

## Quickstart
1. Download pretrained model
 ```bash
mkdir experiments
mkdir experiments/pretrained_models
cd experiments/pretrained_models
git-lfs clone https://huggingface.co/windwhinny/chilloutmix.git
 ```
2. Create model directory
```bash
cd ../..    
mkdir experiments/8101_EDLoRA_potter_Cmix_B4_Repeat500
mkdir experiments/8101_EDLoRA_potter_Cmix_B4_Repeat500/models
 ```
3. Train (specify learning rates in <code>-opt</code> files)
```bash
accelerate launch train_edlora.py -opt options/train/EDLoRA/real/8101_EDLoRA_potter_Cmix_B4_Repeat500.yml --optimizer solora --optimizer_reg 0
```

Here <code>sgd, scaled_gd, lora_pro_sgd, solora_sgd, adamw, scaled_adamw, lora_pro_adamw, solora</code> are all valid choices for <code>--optimizer</code>. Trained models will be saved to <code>experiments/8101_EDLoRA_potter_Cmix_B4_Repeat500/models</code>.

4. Create image directory
```bash
mkdir results
mkdir results/8101_EDLoRA_potter_Cmix_B4_Repeat500
 ```

5. Test
```bash
python test_edlora.py -opt options/test/EDLoRA/human/8101_EDLoRA_potter_Cmix_B4_Repeat500.yml
```
Figures will be saved to <code>results/8101_EDLoRA_potter_Cmix_B4_Repeat500/visualization</code>.

6. Evaluate FID and CLIP score
```bash
# Environment
conda install pillow numpy tqdm scikit-learn scipy -y
pip install ftfy regex
pip install git+https://github.com/openai/CLIP.git
pip install pytorch-fid
```

## Parameter Reference
Refer to Appendix of our paper.


