For GPT-3, we use OpenAI API, wrapped with a python script. The public version is at https://openai.com/api/.

For FiD, we use its official code, which can be downloaded as https://github.com/facebookresearch/FiD. 

For OPT, we also use its official instrcutions to download and run their models.

Although most of the code has been released publicly, we will make our code and trained checkpoints public after this paper is accepted, in order to make it easier to reproduce our experimental results.