1. Download the datasets from https://anonfiles.com/kdxeebI4y5/datasets_zip
2. Unzip the datasets
3. You can train the weights using the following command:
`python finetune_s2s.py --train_file_path=Wiki1M/train.jsonl --test_file_path=Wiki1M/test.jsonl --train_batch_size=32 --eval_batch_size=32 --output_dir=test_run_results/t5_small_wiki_seed3 --model_name_or_path t5-small --use_multisoftmax --data_parallel --save_after_every_eval --eval_every_k_epoch 1 --num_epochs 2`. Change the data paths and checkpoint directories accordingly.
