# Calibrating the Rigged Lottery: Making All Tickets Reliable

This is the code repository of RigL-based models in the following paper: Calibrating the Rigged Lottery: Making All Tickets Reliable.

Parts of this code repository is based on the following works by the machine learning community.

* https://github.com/Shiweiliuiiiiiii/In-Time-Over-Parameterization

## Requirements

The library requires Python 3.6.7, PyTorch v1.0.1, and CUDA v9.0.

You can download it via anaconda or pip, see [PyTorch/get-started](https://pytorch.org/get-started/locally/) for further information. 

## Training 
Our implementation includes the code for two dynamic sparse training methods SET (https://www.nature.com/articles/s41467-018-04316-3) and RigL (https://arxiv.org/abs/1911.11134). The main difference is the weight regorwing method: using --growth random for SET; using --growth gradient for RigL.


### ImageNet with ResNet-50
To train ResNet-50 on ImageNet with **RigL-ITOP**, run the following command:
```
cd ImageNet

CUDA_VISIBLE_DEVICES=0,1 python $1multiproc.py --nproc_per_node 2 $1main.py --multiplier 1 --growth gradient --master_port 4545 -j5 -p 500 --arch resnet50 -c fanin --update_frequency 4000 --label-smoothing 0.1 -b 64 --lr 0.1 --warmup 5 --epochs 100 --density 0.2 $2 ../../../data/ --save save/ITOP/
```
change path of data ../../../data/ to the saved imagenet directory before running.
