# Code for ROMA

## Setup

First create python environment and install the required packages:
```
conda env create -f env.yml
```
The required packages including: Pytorch version 1.12, torchvision version 0.13, timm version 0.3.2, etc.

## Pretrained models

Follow [MAE](https://github.com/facebookresearch/mae) pre-train the model or directly utilize the official released weights pretrained on ImageNet-1k. The models were pretrained with 224x224 resolution. 

Download the pretrained checkpoints that are self-supervised pretrained on ImageNet-1k: [mae_pretrain_vit_large_full.pth](https://dl.fbaipublicfiles.com/mae/pretrain/mae_pretrain_vit_base_full.pth) including both encoder and decoder weights.

## Example:

### Finetuning on In-distribution Dataset

Finetuning MAE with self-supervised learning (SSL) regularization on different ID datasets as follows:

```shell
python3 main_finetune_selfsupervised.py --output_dir ./output_dir/cifar100 --log_dir ./output_dir/cifar100 --data_path ../data/cifar100 --nb_classes 100 --device cuda:0 --batch_size 64 --epochs 100  --finetune ../pretrain_chkpt/mae_pretrain_vit_base_full.pth --save_freq 10 --blr 5e-4
```

Change the parameter `data_path` with path of datasets including CIFAR-10, CIFAR-100, ImageNet-1k to fine-tune on.

### Performing OOD detection on Out-of-distribution Dataset

#### Standard OOD detection

`eval_features2.py` is used for evaluating model OOD detection performance. Also, we utilize the OpenOOD benchmark code for fair evaluating. In this python file, change the `setup_config` function with configs of corresponding ID and OOD datasets. Then, run the following command:

```shell
python3 eval_with_features2.py --output_dir output_dir/cifar100 --data_path ../data/cifar100 --resume output_dir/cifar100/checkpoint-99.pth --device cuda:0 --ood_dataset cifar10 --pre ft_99 --nb_classes 100
```

#### Hidden OOD detection

In `eval_features2.py`, change the `setup_config` function with configs of corresponding ID and **hidden-OOD** datasets. Then, run the following command:

```shell
python3 eval_with_features2.py --output_dir output_dir/cifar100 --data_path ../data/cifar100 --resume output_dir/cifar100/checkpoint-99.pth --device cuda:0 --ood_dataset cifar10 --pre ft_99_hidden --nb_classes 100
```

Also, the hidden-OOD samples we have selected in the paper are zipped into `hiddenOOD_datasets.zip`.

### Choosing Hidden-OOD Samples

run the following command and parse the corresponding FPR value for results of both pretrained models and finetuned models:

```shell
python3 analyze_hiddenOOD.py --id cifar10 --pre 0.008 --ft 0.06
```

