In min-max-linear-RL/ code is provided to reproduce the results and figures from sections 5.1 and 5.2.1.

In nonlinear-exp/ our reference policies and algorithm implementations (algorithms 2, 3, and 4)  are provided for the non-linear RL experiments.