# Bridging Inter-task Gap of Continual Self-supervised Learning with External Data
This repository is the official implementation of Bridging Inter-task Gap of Continual Self-supervised Learning with External Data. 



## Requirements

To install requirements:

```setup
conda create --name bge python=3.8
conda activate bge
pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 torchaudio==0.13.1 --extra-index-url https://download.pytorch.org/whl/cu117
pip install pytorch-lightning==1.5.4 lightning-bolts wandb scikit-learn einops
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist --upgrade nvidia-dali-cuda110
```


## Training
To change the external datasets you want to use, you can modify the "dataselect" in config file (e.g. bash_files/continual/cifar/barlow.sh).
Remember to change the external dataset path in "cassle/utils/pretrain_dataloader.py" (for CIFAR100 experiments)

### Fine-tuning

#### CIFAR100
```
DATA_DIR=/path/to/data/dir/ CUDA_VISIBLE_DEVICES=0 python job_launcher.py --script bash_files/continual/cifar/barlow.sh
```


### CaSSLe and PFR
```
PRETRAINED_PATH=/path/to/task0/checkpoint/ DATA_DIR=/path/to/data/dir/ CUDA_VISIBLE_DEVICES=0 python job_launcher.py --script bash_files/continual/cifar/barlow_distill.sh
```
```
PRETRAINED_PATH=/path/to/task0/checkpoint/ DATA_DIR=/path/to/data/dir/ CUDA_VISIBLE_DEVICES=0 python job_launcher.py --script bash_files/continual/cifar/barlow_my_methods.sh
```
## Linear Evaluation

#### CIFAR100
```
PRETRAINED_PATH=/path/to/last/checkpoint/ DATA_DIR=/path/to/data/dir/ bash bash_files/linear/cifar/barlow_linear.sh
```

