# Improving Few-Shot Visual Classification with Unlabelled Examples
**NOTE: Unfortunately, due to the 100MB size limit on supplementary material, we're unable to provide the model checkpoints in this package. However, since we use the same checkpoints as those released by Bateni et al. for Simple CNAPS, we've provided instructions here so that the reviewers are still able to recreate our results, should that be of interest to them**

This directory contains the code for the paper, "Improving Few-Shot Visual Classification with Unlabelled Examples", submitted to ICLR 2021. This repository has been submitted as supplementary material for the paper, should the reviewers want to look at the code or replicate experiments. This epository will be made public upon paper acceptance.

This code base builds upon the source code released for "Improving Few-Shot Visual Classification" by Bateni et al., and we primarly change the model under ```model.py``` and the set encoder under ```set_encoder.py```. To see the original Simple CNAPS repository, please visit https://github.com/peymanbateni/simple-cnaps.

## Dependencies (Same as Simple CNAPS)
This code requires the following:
* Python 3.5 or greater
* PyTorch 1.0 or greater
* TensorFlow 1.14 or greater (although we recommend refraining from using TensorFlow 2.0 since as of writing, the source code for the Meta-Dataset has not been updated to new TF version.)

## GPU Requirements (Same as Simple CNAPS)
The GPU requirements for Simple CNAPS are:
* 2 GPUs with 16GB or more memory for training Simple AR-CNAPS
* 1 GPU with 16GB or more memory for training Simple CNAPS
We recommend the same settings for testing.

## Installation (Same as Simple CNAPS)
Our installation process is same as Simple CNAPS:
1. Clone or download this repository.
2. Configure Meta-Dataset:
    * Follow the the "User instructions" in the Meta-Dataset repository (https://github.com/google-research/meta-dataset)
    for "Installation" and "Downloading and converting datasets". This will take some time.
3. Install additional test datasets (MNIST, CIFAR10, CIFAR100):
    * Change to the $DATASRC directory: ```cd $DATASRC```
    * Download the MNIST test images: ```wget http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz```
    * Download the MNIST test labels: ```wget http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz```
    * Download the CIFAR10 dataset: ```wget https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz```
    * Extract the CIFAR10 dataset: ```tar -zxvf cifar-10-python.tar.gz```
    * Download the CIFAR100 dataset: ```wget https://www.cs.toronto.edu/~kriz/cifar-100-python.tar.gz```
    * Extract the CIFAR10 dataset: ```tar -zxvf cifar-100-python.tar.gz```
    * Change to the ```simple-cnaps/src``` directory in the repository.
    * Run: ```python prepare_extra_datasets.py```

## Usage
To train and test Transductive CNAPS on Meta-Dataset:

1. First run the following three commands:
    
    ```ulimit -n 50000```

    ```export META_DATASET_ROOT=<root directory of the cloned or downloaded Meta-Dataset repository>```
    
    ```export RECORDS=<root directory of where you have produced meta-dataset records>```
    
    Note that you may need to run the above commands every time you open a new command shell.
    
2. Unfortunately, due to the 100MB size limit on supplementary material, we're unable to provide the model checkpoints in this package. However, the checkpoint used in producing Transductive CNAPS results at test-time is the same as the one released by Bateni et al. for Simple CNAPS. You can download this checkpoint by simply running ```wget https://github.com/peymanbateni/simple-cnaps/best_simple_cnaps.pt``` or downloading it from the Simple CNAPS repository at https://github.com/peymanbateni/simple-cnaps. After you have downloaded and placed the checkpoint in this repository's main directory, you can use the following commands to test the provided Simple CNAPS and Simple AR-CNAPS models:

    For Transductive CNAPS:
    
    ```cd src```
    ```python run_transductive_cnaps.py --data_path $RECORDS --feature_adaptation film --mode test -m ../best_simple_cnaps.pt```
    
    To re-create initial Simple CNAPS results, simply set ```max_em_steps``` to zero:
    
    ```cd src```
    ```python run_transductive_cnaps.py --data_path $RECORDS --feature_adaptation film+ar --mode test -m ../best_simple_ar_cnaps.pt --max_em_steps 0```

    Note that while the parameters are the same, since for testing, we sample a set of tasks from each dataset, variations may be seen in terms of reproducing results. That said, the discrepancies should be within the confidence intervals provided and should still match the references results considering statistical significance.
    
3. If you would like to train/test the models from scratch, use the following two commands:

    For training without tranduction (as is the case with Transductive CNAPS):
    
    ```cd src```
    ```python run_transductive_cnaps.py --data_path $RECORDS --feature_adaptation film --checkpoint_dir <address of the directory where you want to save the checkpoints> --max_em_steps 0```
    
    For training with transduction (which is the case in the Transductive+ CNAPS ablation):
    
    ```cd src```
    ```python run_transductive_cnaps.py --data_path $RECORDS --feature_adaptation film --checkpoint_dir <address of the directory where you want to save the checkpoints>```
    
    Depending on your environment, training may take anywhere from 1-5 days. For reference, training on 2 T4 GPUs with 64G of memory and 8 dedicated GPUs took 2 days and 4 hours for Simple CNAPS and over 3 days for Simple AR-CNAPS.

## Citing Original Simple CNAPS
```
@misc{bateni2019improved,
    title={Improved Few-Shot Visual Classification},
    author={Peyman Bateni and Raghav Goyal and Vaden Masrani and Frank Wood and Leonid Sigal},
    year={2019},
    eprint={1912.03432},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}
```
