# barlowtwins-Fusion Active function and SyncNorm

## Note
This is the new barlowtwins version with Fusion Active function and SyncNorm

⚠️WARNING: if you want to set the Boolean variables to Flase, please leave them with "" (e.g. `--reload=` `--spiking=` ) to make sure them will be correctly parsed by argparse.

## Experiment groups(DDP is available)
⚠️Please notice the number of batch size（2048 in total for ImageNet as same as in Barlow Twins paper, 256 for CIFAR10) assigned to each GPU, if GPU number changes.

General setting for all experiment groups: epoch=1000, time-step=4 if use LIF.

## ImageNet

|||Settings in Command Line|
|:-------:|:-------:|:-------:|
|Method|Step| ResNet50-ImageNet|
| spiking, 1stlast temporal loss, MixedLIF|SSL training|``` CUDA_VISIBLE_DEVICES=0,1,2,3  python main.py --gpus=4 --batch_size=40 --model="resnet50" --dataset="ImageNet" --n_classes=1000 --lr=1e-3 --spiking=True --temporal_loss=True --cross_temporal=True --simplified_loss=True --sync_norm=True --epochs=1000 --projection_dim=8192 --act_func="MixedLIF" --model_path="save/imagenet/resnet50_spk_1l_mixlif" --num_workers 1 --dataset_dir /mnt/vstor/CSE_ECSE_GXD234/data/imagenet```|
||linear evaluation|```python linear_evaluation.py --model="resnet50" --dataset="ImageNet" --n_classes=1000 --epoch_num=1000 --spiking=true --sync_norm=True --act_func="MixedLIF" --model_path="save/imagenet/resnet50_spk_1l_mixlif"```|
||fine tune on 1%/10% labels|```python finetune_semi_supervised_learning.py --eval_proportion=0.01 --logistic_epochs=60 --model="resnet50" --dataset="ImageNet" --n_classes=1000 --epoch_num=1000 --spiking=true --sync_norm=True --model_path="save/imagenet/resnet50_spk_1l_mixlif"```, ```python finetune_semi_supervised_learning.py --eval_proportion=0.1 --logistic_epochs=30 --model="resnet50" --dataset="ImageNet" --n_classes=1000 --epoch_num=1000 --spiking=true --sync_norm=True --model_path="save/imagenet/resnet50_spk_1l_mixlif"```|
||transfer_learning|```python transfer_learning.py --eval_proportion=1 --logistic_epochs=500 --model="resnet50" --dataset="CIFAR10" --n_classes=10 --epoch_num=1000 --spiking=true --sync_norm=True --model_path="save/imagenet/resnet50_spk_1l_mixlif"```, ```python transfer_learning.py --eval_proportion=1 --logistic_epochs=500 --model="resnet50" --dataset="oxford_flowers102" --n_classes=102 --epoch_num=1000  --spiking=true --sync_norm=True --model_path="save/imagenet/resnet50_spk_1l_mixlif"```|


## Cifar10

|||Settings in Command Line|
|:-------:|:-------:|:-------:|
|Method|Step| ResNet34-CIFAR10|
| Nonspiking(ANN)|SSL training|``` CUDA_VISIBLE_DEVICES=0,1  python main.py --gpus=2 --batch_size=256 --model="resnet34" --dataset="CIFAR10" --n_classes=10 --lr=1e-3 --epochs=1000 --projection_dim=1024 --model_path="save/cifar10/resnet34"```|
||linear evaluation|```python linear_evaluation.py --model="resnet34" --dataset="CIFAR10" --n_classes=1000 --epoch_num=1000 --model_path="save/cifar10/resnet34"```|
||fine tune on 1%/10% labels|```python finetune_semi_supervised_learning.py --eval_proportion=0.01 --logistic_epochs=60 --model="resnet34" --dataset="CIFAR10" --n_classes=10 --epoch_num=1000 --model_path="save/cifar10/resnet34"```, ```python finetune_semi_supervised_learning.py --eval_proportion=0.1 --logistic_epochs=30 --model="resnet34" --dataset="CIFAR10" --n_classes=10 --epoch_num=1000 --model_path="save/cifar10/resnet34"```|
| spiking, 1stlast temporal loss, MixedLIF|SSL training|``` CUDA_VISIBLE_DEVICES=0,1  python main.py --gpus=2 --batch_size=256 --model="resnet34" --dataset="CIFAR10" --n_classes=10 --lr=1e-3 --spiking=True --temporal_loss=True --cross_temporal=True --simplified_loss=True --sync_norm=True --epochs=1000 --projection_dim=1024 --act_func="MixedLIF" --model_path="save/cifar10/resnet34_spk_1l_mixlif"```|
||linear evaluation|```python linear_evaluation.py --model="resnet34" --dataset="CIFAR10" --n_classes=10 --epoch_num=1000 --spiking=true --sync_norm=True --act_func="MixedLIF" --model_path="save/cifar10/resnet34_spk_1l_mixlif"```|
||fine tune on 1%/10% labels|```python finetune_semi_supervised_learning.py --eval_proportion=0.01 --logistic_epochs=60 --model="resnet34" --dataset="CIFAR10" --n_classes=10 --epoch_num=1000 --spiking=true --sync_norm=True --model_path="save/cifar10/resnet34_spk_1l_mixlif"```, ```python finetune_semi_supervised_learning.py --eval_proportion=0.1 --logistic_epochs=30 --model="resnet34" --dataset="CIFAR10" --n_classes=10 --epoch_num=1000 --spiking=true --sync_norm=True --model_path="save/cifar10/resnet34_spk_1l_mixlif"```|
|Method|Step| VGG16-CIFAR10|
| spiking, 1stlast temporal loss, MixedLIF|SSL training|``` CUDA_VISIBLE_DEVICES=0,1  python main.py --gpus=2 --batch_size=256 --model="vgg16" --dataset="CIFAR10" --n_classes=10 --lr=1e-3 --spiking=True --temporal_loss=True --cross_temporal=True --simplified_loss=True --sync_norm=True --epochs=1000 --projection_dim=1024 --act_func="MixedLIF" --model_path="save/cifar10/vgg16_spk_1l_mixlif"```|
||linear evaluation|```python linear_evaluation.py --model="vgg16" --dataset="CIFAR10" --n_classes=10 --epoch_num=1000 --spiking=true --sync_norm=True --act_func="MixedLIF" --model_path="save/cifar10/vgg16_spk_1l_mixlif"```|
||fine tune on 1%/10% labels|```python finetune_semi_supervised_learning.py --eval_proportion=0.01 --logistic_epochs=60 --model="vgg16" --dataset="CIFAR10" --n_classes=10 --epoch_num=1000 --spiking=true --sync_norm=True --model_path="save/cifar10/vgg16_spk_1l_mixlif"```, ```python finetune_semi_supervised_learning.py --eval_proportion=0.1 --logistic_epochs=30 --model="vgg16" --dataset="CIFAR10" --n_classes=10 --epoch_num=1000 --spiking=true --sync_norm=True --model_path="save/cifar10/vgg16_spk_1l_mixlif"```|

### Ablation study:

|||Settings in Command Line|
|:-------:|:-------:|:-------:|
| spiking, 1stlast temporal loss, LIF|SSL training|``` CUDA_VISIBLE_DEVICES=0,1  python main.py --gpus=2 --batch_size=256 --model="resnet34" --dataset="CIFAR10" --n_classes=10 --lr=1e-3 --spiking=True --temporal_loss=True --cross_temporal=True --simplified_loss=True --sync_norm=True --epochs=1000 --projection_dim=1024 --act_func="LIFt" --model_path="save/cifar10/ab_1"```
||linear evaluation|```python linear_evaluation.py --model="resnet34" --dataset="CIFAR10" --n_classes=10 --epoch_num=1000 --spiking=true --sync_norm=True --act_func="LIFt" --model_path="save/cifar10/ab_1"```|
| spiking, cross temporal loss, MixedLIF|SSL training|``` CUDA_VISIBLE_DEVICES=0,1  python main.py --gpus=2 --batch_size=256 --model="resnet34" --dataset="CIFAR10" --n_classes=10 --lr=1e-3 --spiking=True --temporal_loss=True --cross_temporal=True --simplified_loss= --sync_norm=True --epochs=1000 --projection_dim=1024 --act_func="MixedLIF" --model_path="save/cifar10/ab_2"```
||linear evaluation|```python linear_evaluation.py --model="resnet34" --dataset="CIFAR10" --n_classes=10 --epoch_num=1000 --spiking=true --sync_norm=True --act_func="MixedLIF" --model_path="save/cifar10/ab_2"```|
| spiking, non-temporal loss, MixedLIF|SSL training|``` CUDA_VISIBLE_DEVICES=0,1  python main.py --gpus=2 --batch_size=256 --model="resnet34" --dataset="CIFAR10" --n_classes=10 --lr=1e-3 --spiking=True --temporal_loss= --cross_temporal= --simplified_loss= --sync_norm=True --epochs=1000 --projection_dim=1024 --act_func="MixedLIF" --model_path="save/cifar10/ab_3"```
||linear evaluation|```python linear_evaluation.py --model="resnet34" --dataset="CIFAR10" --n_classes=10 --epoch_num=1000 --spiking=true --sync_norm=True --act_func="MixedLIF" --model_path="save/cifar10/ab_3"```|
| spiking, cross temporal loss, LIF|SSL training|``` CUDA_VISIBLE_DEVICES=0,1  python main.py --gpus=2 --batch_size=256 --model="resnet34" --dataset="CIFAR10" --n_classes=10 --lr=1e-3 --spiking=True --temporal_loss=True --cross_temporal=True --simplified_loss= --sync_norm=True --epochs=1000 --projection_dim=1024 --act_func="LIFt" --model_path="save/cifar10/ab_4"```
||linear evaluation|```python linear_evaluation.py --model="resnet34" --dataset="CIFAR10" --n_classes=10 --epoch_num=1000 --spiking=true --sync_norm=True --act_func="LIFt" --model_path="save/cifar10/ab_4"```|
| spiking, non-temporal loss, LIF|SSL training|``` CUDA_VISIBLE_DEVICES=0,1  python main.py --gpus=2 --batch_size=256 --model="resnet34" --dataset="CIFAR10" --n_classes=10 --lr=1e-3 --spiking=True --temporal_loss= --cross_temporal=True --simplified_loss= --sync_norm=True --epochs=1000 --projection_dim=1024 --act_func="LIFt" --model_path="save/cifar10/ab_5"```
||linear evaluation|```python linear_evaluation.py --model="resnet34" --dataset="CIFAR10" --n_classes=10 --epoch_num=1000 --spiking=true --sync_norm=True --act_func="LIFt" --model_path="save/cifar10/ab_5"```|

## dvs-Cifar10
To implement Vit-4-38 based Barlow twins on cifar10, please conduct
```
cd cifar10dvs-resnet
``` 

⚠️Note: basic settings and needed modifications are same as ImageNet, which is mentioned above. Exception, please modify ```batch_size``` value  and  ```resume``` in ```train.py```. And please put dvs dataset ```dvs-cifar10``` under this root.

|||Settings in Command Line|
|:-------:|:-------:|:-------:|
|Method|Step| resnet34-Cifar10dvs|
|firstlast-timestep loss, MixedLIF|ssl training|   ``` CUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --nproc_per_node=4 train.py --dataset /mnt/vstor/CSE_ECSE_GXD234/data/cifar-dvs/cifar-dvs -b 80 --workers 2 --resume save/Cifar10-DVS_T10/checkpoint_epoch505 --epochs 1000 ```|
||linear evaluation|```python evaluation.py```|



## Ablations Study
#### Temporal loss study: cross temporal loss v.s. 1stlast temporal loss on cifar10: 

cross Temporal loss

1stlast temporal loss


#### MixedLIF study on cifar10: 

Without MixedLIF

With MixedLIF


#### Dependencies
```
torch
torchvision
pyyaml
numpy
```
