# CreAT

Dependency:

* Pytorch>=1.7
* Transformers

## Quick Start

**Sentence classification**

GLUE data can be downloaded from https://gluebenchmark.com/tasks.

Please put the task data in `data/`.

Training BERT on MNLI:

```bash
CUDA_VISIBLE_DEVICES=0 python run_sent_clas.py \
  --do_train \
  --do_eval \
  --task_name MNLI \
  --learning_rate 3e-5 \
  --train_batch_size 128 \
  --output_dir mnli \
  --do_lower_case \
  --adv_lr 1e-1 \
  --model_type bert-base-uncased \
  --fp16
```

For vanilla AT, please add `--tau 0`.

Testing:

```bash
CUDA_VISIBLE_DEVICES=0 python run_sent_clas.py \
  --do_test \
  --task_name MNLI \
  --eval_batch_size 128 \
  --output_dir mnli \
  --do_lower_case \
  --model_type bert-base-uncased \
  --load_model_path mnli/xxx.bin
```

**Multi-choice**

DREAM data is provided.

```bash
CUDA_VISIBLE_DEVICES=0 python run_multi_cho.py \
  --do_train \
  --do_eval \
  --task_name DREAM \
  --learning_rate 2e-5 \
  --train_batch_size 16 \
  --output_dir dream \
  --do_lower_case \
  --model_type bert-base-uncased \
  --fp16 \
  --eval_on test \
  --num_train_epochs 6
```

HellaSWAG data can be downloaded from https://allenai.org/data/hellaswag.

```bash
CUDA_VISIBLE_DEVICES=0 python run_multi_cho.py \
  --do_train \
  --do_eval \
  --task_name HELLASWAG \
  --learning_rate 2e-5 \
  --train_batch_size 16 \
  --output_dir hellaswag \
  --do_lower_case \
  --model_type bert-base-uncased \
  --fp16
```
