Codebase for PETRA, built on Megatron-LM.

"data_preprocess" contains codes for data preprocessing.
"model_training" contains codes for model training and some supplementary codes.
"results" contains experimental results.

Independent "Readme.txt" about the usage of data preprocess code and model training code can be found in their directories respectively. 

