# RCR implementation based on RAD
Codebase for ICLR2022

# Directory List
    |-- Data Aug Visualization.ipynb
    |-- README.md
    |-- TransformLayer.py
    |-- agent.py    
    |-- conda_env.yml
    |-- data_augs.py
    |-- data_sample.npy
    |-- eval.py            
    |-- logger.py        
    |-- package-lock.json
    |-- train.py            # Train our agent
    |-- train.sh            # Script for training on Cartpole
    |-- train_cheetah.sh    # Script for training on Cheetah
    |-- train_walker.sh     # Script for training on Walker
    |-- utils.py            # Utility functions
    |-- utilsmod.py         # Utility modules
    |-- video.py            
    |-- algorithms          # Our agent is implemented here 
    |   |-- factory.py      # Main Interface
    |   |-- rad.py           
    |   |-- rad_byol.py      
    |   |-- rad_simsiam.py      
    |   |-- sac.py           
    |   |-- sac_byol.py     
    |   |-- sac_simsiam.py  

# Environment Preparation and Running

1. ```
   conda env create -f conda_env.yml
   ```

2. Make sure you installed `mujoco`, `gym`; 

3. Specify `MUJOCO_GL` if you met `GLFW` error: 

   1. Solution 1 (for non-root users): `export MUJOCO_GL=osmesa` and you need to install `osmesa` through conda first
   2. Solution 2 ( for users with root permission): `xvfb-run -a -s "-screen 0 1400x900x24 bash"`

4. ```
   export CUDA_VISIBLE_DEVICES=0
   bash train_xxx.sh # trains PCR on task xxx with parameters specifies in the script file. Alternatively, you can directly specifies the parameters by calling `train.py` following tran.sh
   ```

# Running Ablation Studies

1. For `PCR without predictive consistency`, run `sh train_walker_noinv.sh` to train it on walker. You can change the task name in the script.
2. For `PCR with Constrastive Loss`, run `sh train_walker_cmc.sh` to train it on walker. You can change the task name in the script.