1. Incase not trained, download the pretrained files from https://anonfiles.com/35xde1Icye/output_zip
2. Then run inference using the following command:
`python load_and_eval_s2s.py --train_file_path=Wiki1M/train.jsonl --test_file_path=Wiki1M/test_unseen.jsonl --output_dir=experiments/test_run_results/t5_small_wiki_seed3/epoch2 --eval_batch_size 16 --decode_beams 15`
Update the dataset paths and checkpoint dirs accordingly.
3. Finally calculate evaluation metrics using:
`python eval.py --guess output/experiments/test_run_results/t5_small_wiki_seed3/epoch2/test_preds_filtered_sum_prob_aida.jsonl --gold Wiki1M/test_unseen.jsonl  --ks 1,2,3,5,10,20,50,100`