# Mixture of Prompts
## :octocat: Installation
- Env: must be **python 3.8**

- Run ATTEMPT
```ruby
/bin/bash scripts/run_attempt.sh
```
- Modify model in `./attempt/seq2seq/third_party/models/t5/modeling_t5.py`

## :steam_locomotive: Train Source Prompts with Multiple GPUs
```ruby
/bin/bash scripts/train_source.sh
```

## :notes: Baselines
### Single teachers
```ruby
/bin/bash scripts/train_source.sh
```

### Fewshot-test single-teacher/PPT
```ruby
/bin/bash scripts/test_vote_attn_fs.sh
```

## Table 1
### Fewshot FT
```ruby
/bin/bash scripts/pt.sh
```
### PT
1. Get the few-shot prompts.
```ruby
/bin/bash scripts/fewshot_init.sh
```
The results are saved in `./outputs/baselines/init_pt_fewshot_table`

2. Test the results by
```ruby
/bin/bash scripts/test_pt.sh
```

### SPOT-t
1. We need to get the relation between initial soft prompt with source
```ruby
python ./attempt/seq2seq/calc_sim.py
```
2. Run test by
```ruby
/bin/bash test_spot.sh
```

### PPT
1. train
```ruby
/bin/bash scripts/train_ppt.sh
```
2. Fewshot single-teacher/PPT
```ruby
/bin/bash scripts/fewshot_ppt.sh
```
3. Test fewshot ppt
```ruby
/bin/bash scripts/test_ppt.sh
```

### Fewshot ATTEMPT
```ruby
/bin/bash scripts/fewshot.sh
```

<!--
### Fewshot Soft Prompt initial
This is for calc the sim of prefix; SPoT-t.
```ruby
/bin/bash scripts/fewshot_init.sh
```
--> 

### Uniform Ensemble
```ruby
/bin/sh scripts/test_uniform.sh
```

### FW Ensemble
1. Get eval vote results for eah source task.
```ruby
/bin/bash scripts/test_source_val.sh
```
2. get pickle for loading weight.
```ruby
python eval_acc.py
```
3. test
```ruby
/bin/bash scripts/test_eval_vote.sh
```

## Table 2
### Sim-prompt
```ruby
python ./attempt/seq2seq/calc_sim.py
/bin/bash scripts/test_sim_prompt.sh
```
### Eval-acc
```ruby
/bin/bash scripts/test_source_eval.sh
python cal_val_acc.py
/bin/bash scripts/test_sim_prompt.sh
```

## Table 3 - Full

### PT source
```ruby
/bin/bash scripts/full_source.sh
```

### ATTEMPT
```ruby
/bin/bash scripts/run_attempt.sh
```

### :notes: Our model
```ruby
/bin/bash scripts/vote_attn_full.sh
```
### PPT
```ruby
/bin/bash scripts/pt_ppt_full.sh
```
