# Accuracy Boosters: Epoch-Driven Mixed-Mantissa Block Floating-Point for DNN Training 

## Requirements

To install requirements:

```setup
pip install -r requirements.txt
```

>📋  We have tested our implementation on 2 NVIDIA V100 GPUs (Driver Version: 470.57.02 & CUDA Version: 11.4) with Python 3.9.7 and PyTorch 1.11.0

## Training

To train the model(s) in the paper, run the scripts under the folder cnn/. For example, to run a WideResNet on CIFAR100 with Accuracy Boosters between 0-10th and 240-250th epochs:

```train
bash cnn/scripts/wideresnet/booster_10,240_wrn.sh
```

>📋  This bash script includes the command to run the specified model on specified dataset (e.g., python main.py --type cnn --arch wideresnet28 --data cifar100) and rest of the arguments are the hyperparameters for that training task. These arguments are followed by HBFP arguments (i.e., --num_format bfp --rounding_mode stoc --mant_bits 5 --bfp_tile_size 16 --weight_mant_bits 15 --manual_seed 2222) and finally the Accuracy Booster epoch numbers (--mixed_precision 10,240)

## Evaluation

To evaluate a trained model that has checkpoints: 

```eval
python main.py --type eval_cnn --world_size 2 --checkpoint <path-to-the-checkpoint/model_best.pth.tar>
```
