1. Train a baseline ensemble using
python train_baseline.py --dataset=DATASET		# DATASET can be either cifar10 or cifar100

2. Train GAL ensemble using (Requires Tensorflow v1.x)
python train_gal.py

3. Train PARL Ensemble using (Requires Tensorflow v1.x)
python train_parl.py --dataset=DATASET --num_layers=NUM		# DATASET can be either cifar10 or cifar100 and NUM can be 3, 5, or 7

4. Train ADP and DVERGE ensembles following the links provided below
a. Training procedure for an ADP Ensemble is provided in https://github.com/P2333/Adaptive-Diversity-Promoting
b. Training procedure for a DVERGE Ensemble is provided in https://github.com/zjysteven/DVERGE

5. Train a surrogate model for black-box transfer attacks
a. Train another baseline model using
python train_baseline.py --dataset=DATASET		# DATASET can be either cifar10 or cifar100
b. Rename the saved model to DATASET_baseline_surrogate.h5

We have provided some of the pre-trained models inside pretrained_models folder. We were not able to provide all the model because of space constraints.

6. Craft adversarial samples from previously selected benign samples from the surrogate model using
python generate_black_box_adversarial_samples.py --dataset=DATASET --attack=ATTACK --strength=EPS
# DATASET can be either cifar10 or cifar100
# ATTACK can be fgsm, bim, mim, or pgd
# EPS can be 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, or 0.07

The benign samples are provided inside benign_samples folder. The crafted adversarial samples will be stored inside adversarial_examples folder.

7. Evaluate performance of the ensembles against the crafted adversarial samples using
python evaluate_black_box_transfer_attacks.py --dataset=DATASET --attack=ATTACK --random_start=RAND --strength=EPS --model=MODEL --num_layers=NUM
# DATASET can be either cifar10 or cifar100
# ATTACK can be fgsm, bim, mim, or pgd
# RAND can be any integer between [0, 1] if ATTACK is pgd
# EPS can be 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, or 0.07
# MODEL can be baseline, adp, gal, or parl
# NUM can be 3, 5, or, 7 if MODEL is parl

8. The DVERGE ensemble is designed using PyTorch library. Use the samples from adversarial_examples folder to test the performance of DVERGE using the link https://github.com/zjysteven/DVERGE