# Applying Second Order Optimizer to Deep Transformers with Parameter-efficient Tuning

### Install requirements and OpenDelta from source
```
pip install -r requirements.txt
cd OpenDellta
python setup.py install
```

### Generate configs
```
cd examples_text_classification
python gen_config.py
```

### Run
#### Text classification on RTE dataset with AdamW and K-FAC
```
bash run_newoptim_cls.sh
```

#### Sequence-to-Sequence on RTE dataset with AdamW and K-FAC
```
bash run_seq2seq.sh
```