# PoLiCER (Image-based)
Our implementation is based on the official codebase of [B-Pref](https://github.com/rll-research/BPref) and [SURF](https://github.com/alinlab/SURF/tree/pixel). For meta-world environment wrapper, we also benefit from [DrM](https://github.com/XuGW-Kevin/DrM?tab=readme-ov-file). We appreciate their insightful works.

## How to Install
Same with vector-based control branch

### Docker Setting and Install Mujoco 2.1.0

```bash
# run docker container
docker run -it -d --shm-size=64g --gpus=all -v /your/drive/location:/mnt/hdd/workspace -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=unix$DISPLAY --env QT_X11_NO_MITSHM=1 --name PoLiCER_image pytorch/pytorch:1.11.0-cuda11.3-cudnn8-runtime
apt-get update
apt-get install sudo
sudo apt update
cd ../mnt/hdd/workspace
apt-get install wget
apt-get install nano

# install mujoco
wget https://mujoco.org/download/mujoco210-linux-x86_64.tar.gz
mkdir /root/.mujoco
tar -xvzf mujoco210-linux-x86_64.tar.gz -C /root/.mujoco
sudo apt install libglew-dev libgl-dev -y

# copy and paste following commands into /root/.bashrc
export MUJOCO_GL=egl
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/root/.mujoco/mujoco210/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/nvidia
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/x86_64-linux-gnu
export PATH="$LD_LIBRARY_PATH:$PATH"
export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libGL.so:/usr/lib/x86_64-linux-gnu/libGLEW.so
export DISPLAY=:1

sudo apt-get install x11-apps

conda install -c conda-forge xorg-libx11 xorg-libxcomposite xorg-libxcursor xorg-libXdamage xorg-libXext xorg-libXfixes xorg-libXi xorg-libXinerama xorg-libXrandr -y

cd root
source .bashrc

sudo apt-get install python3-dev build-essential libssl-dev libffi-dev libxml2-dev -y
sudo apt install libosmesa6-dev libgl1-mesa-glx libglfw3 patchelf libegl1 libopengl0 -y


sudo apt-get install libxslt1-dev zlib1g-dev python3-pip
conda install libffi==3.3
conda install git
git clone https://github.com/openai/mujoco-py
cd mujoco-py
pip install -r requirements.txt
pip install -r requirements.dev.txt

pip3 install -e . --no-cache

sudo apt install -y mesa-utils
```

### Install Dependencies
Python 3.8.12

```bash
pip install mujoco==2.3.5
pip install gym==0.25.2
pip install dm_control==1.0.12
pip install git+https://github.com/denisyarats/dmc2gym.git
pip install tensorboard termcolor pybullet scikit-image
pip install hydra-core==1.0.4
pip install "cython<3"

pip install opencv-python
pip install imageio imageio[ffmpeg]
pip install einops
```

### Metaworld Dependency
<!-- All experiments are conducted on old metaworld version, same with previous methods. However, current Metaworld releases are only available for 2.0.0, which is incompatible for our implementation. Therefore, we attach previous metaworld package with their license in our code.

Original install guide for metaworld package (Current version 2.0.0)
```bash
pip install git+https://github.com/Farama-Foundation/Metaworld.git@master#egg=metaworld
``` -->
The Metaworld package is still under development and continually updated. Therefore, to replicate our experiments, you need to install Metaworld using the following commands:

```bash
wget https://github.com/Farama-Foundation/Metaworld/archive/refs/tags/v2.0.0.tar.gz
tar -xvzf v2.0.0.tar.gz
cd Metaworld-2.0.0
pip install metaworld -e .
# cp -r metaworld /mnt/hdd/workspace
```

### Hydra Dependency
We used hydra with current version of 1.0.4, while original [B-Pref](https://github.com/rll-research/BPref) used version 0.x.
We slightly modified hydra configuration in **config** folder and *hydra.main()* args in all **train_x.py**. It does not affect the experiment, but only affect compatibility for hydra version.
For more details, please refer to [hydra config path changes](https://hydra.cc/docs/upgrades/0.11_to_1.0/config_path_changes/)

## How to run
### Synthetic annotator experiment with image-based observation (DMC)
```bash
export device=0
bash scripts/dmc.sh
```

### Synthetic annotator experiment with image-based observation (Meta-world)
```bash
unset LD_PRELOAD
export LD_PRELOAD=""
export device=0
bash scripts/meta.sh
```

