# Hard-label Manifolds

## Supported environments:
* Linux x86

## Get started
1. Install Anaconda for Linux: https://www.anaconda.com/products/individual#linux
2. Install project dependencies by running `setup.sh` in the project root. 



## Numerical Experiments (Section 3)
1. Navigate to project root and add it to PYTHONPATH
2. `cd research_pool`
3. Run `python mutual_information.py`
4. Result is saved to `research_pool/figures`.


## Empirical Experiments (Section 4 and 5)

### Configuration files
Scripts rely on `.json` configuration files which hold the hyperparameters for each attack run.

To test an attack given configuration file at `/path/to/config.json` use:
```
python test_attack.py --config /path/to/config.json --gpuid 0
```
Note that CUDA support is required.


### Paper results
The (anonymized) files to re-create paper results are stored in `research_pool/config_autogen`.

#### Datasets
MNIST and CIFAR-10 are taken from torchvision library. ImageNet (ILSVRC 2012 version) must be downloaded from http://www.image-net.org/. We use the train and val splits.  

#### Robust model checkpoints
A script is included which attempts to download and validate checkpoints for all of the robust models used in our paper:
1. `cd communty`
2. `python init_ckpts.py`



### General information about this repo:
#### Supported attack
* Hard-label Black-box attack
    * $\ell_2$ and $\ell_\infty$ attack:
        * OPT-attack: https://arxiv.org/abs/1807.04457
        * Sign-OPT: https://arxiv.org/abs/1909.10773
        * HSJA: https://arxiv.org/abs/1904.02144
        

#### Supported dataset
* MNIST
    * Natural variant (Conv. network)
    
* Robust MNIST ($\ell_2$)
    * Madry adversarial training
    * TRADES 
    * Robust manifold defense

* CIFAR-10
    * Natural variant (ResNet)
    
* Robust CIFAR-10 ($\ell_\infty$)
    * Madry adversarial training
    * TRADES
    * Interpolation training
    * Feature scattering
    * SENSE
    
* Robust CIFAR-10 ($\ell_2$)
    * Certified smoothing 
    
* ImageNet 
    * Natural variant (ResNet)
    
* Robust ImageNet ($\ell_\infty$)
    * Madry adversarial training (ε=8/255)

* Robust ImageNet ($\ell_2$)
    * Certified smoothing
    
