# Discriminator Gradient Flow Implementation for ICLR 2021 Review

This directory contains an implementation of Discriminator Gradient Flow (DGflow) for the CIFAR10 dataset and can be used to reproduce some experiments in the paper **Refining Deep Generative Models via Wasserstein Gradient Flows**.  

## Enviroment Setup

* Install Libraries:
    * torch (Tested on `torch==1.6.0`)
    * torchvision (Tested on `torchvision==0.7.0`)
    * cupy (Tested on `cupy-cuda101==7.6.0`)
    * chainer (Tested on `chainer==7.2.0`)
    * tensorflow (Required for porting InceptionV3, Tested on `tensorflow-gpu==1.14.0`)
    * Pillow
    * tqdm
    * scipy
    * pyyaml

* Run the following scripts.
```bash
$ python load_dataset.py --root torchdata/ --data cifar10      # download CIFAR-10 and making training_data/CIFAR.npy
$ mkdir metric
$ python download.py --outfile metric/inception_score.model    # download inception model
$ python get_mean_cov_2048featurespace.py --data CIFAR --gpu 0 # calculating mean&cov in 2,048 feature space and saving it to metric/CIFAR_inception_mean.npy and metric/CIFAR_inception_cov.npy
```
* Download pretrained SNGANs for CIFAR10 from [Tanaka, 2019]'s official repository. (Not included in this zip due to size limitations).
* Download pretrained OCFGAN for CIFAR10 from [Ansari et al., 2020]'s official repository. (Included in this zip with the corresponding density ratio corrector).

## Usage

To refind GAN samples using DGflow run the following:

```
python dgflow.py --config <path to config file>
```

Example for SN-DCGAN (hinge):

```
python dgflow.py --config configs/sngan-hi.yml
```

The results will be saved in `./exps/`.

### Example of a Config File

```yaml
dataset: cifar10
image_size: 32
eval_file_prefix: CIFAR
gen_type: t_cnn # Type of generator
gen_path: trained_models/OCFGAN_CIFAR.pth # Path to generator checkpoint
disc_type: c_sndcgan # Type of discriminator
disc_path: trained_models/DCGAN_D_CIFAR_SNGAN_NonSaturating_150001.npz # Path to discriminator checkpoint
corr_type: t_sndcgan # Type of density ratio corrector
corr_path: trained_models/OCFGAN_CIFAR_DRC.pth # Path to density ratio corrector checkpoint
f_div: KL # One of 'KL', 'logD', or 'JS'
eta: 0.1 # Step-Size
gamma: 0.01 # Noise regularizer
save_interval: 5 # Save samples every save_interval steps
keep_samples: true # Keep samples on disk after execution
steps: 25 # Number of update steps
bottom_width: 4 
num_imgs: 50000 # Number of images to generate
batch_size: 500
exp_root: ./exps/
z_dim: 32 # Dimension of the prior distribution
```

### Acknowledgement 

This repo contains code that's based on [Miyato et al., 2018], [Tanaka, 2019], [Ansari et al., 2020]'s repos.

### References
**[Miyato et al., 2018]** Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. In ICLR, 2018.    
**[Tanaka, 2019]** Akinori Tanaka. Discriminator optimal transport. In NeurIPS, 2019.    
**[Ansari et al., 2020]** Abdul Fatir Ansari, Jonathan Scarlett, and Harold Soh. A characteristic function approach to deep implicit generative modeling. In CVPR, 2020.