README for Muti-Critic Actor Leanring Supplementary Code Submission

NOTE: 
- The code provided with this initial submission is admittedly not in its cleanest form and will be cleaned up and better documented for public release.
- If you wish to verify our results however, the code provided should allow you to run experiments for the Path Following and Pong experiments discussed in our main paper

TO INSTALL:
1. Please pip install the PyGame-Learning-Environment code in PLE_Mod/PyGame-Learning-Environment
2. Please install pleMod_gym in PLE_Mod ("pip install -e ." from the PLE_Mod folder root)
3. Please install shapeFollow_gym in the ShapeFollow folder (additional README in the folder explains installation)
4. Please install spinup from the spinningup folder.


Spinup usage:
Once installed, running experiments works the same way as with the general spinup code (https://spinningup.openai.com/en/latest/)

Example:

python -m spinup.run mcn_sac_pytorch --hid "[8,8]" --env TriShapeFollow2D-v1 --exp_name mcn_sac_trishapetest --epochs 100 --seed 0

To run MN-MultiCriticAL with SAC on the 3-shape Path following environment presented in our paper

To view possible pong run configurations, please look in PLE_Mod/pleMod_gym/__init__.py for a list of gym environments that will be registered for use upon installation.

P.S. Results for the paper were generated using the pytorch implementations of the algorithms.


TO RUN SONIC GAMES:
- To run training on the Sonic games, first install gym-retro following instructions from https://contest.openai.com/2018-1/details/
- Once installed, the wrappers in spinning/spinup/sonic_utils can be used to wrap the sonic games and run training on