## Install

```shell
# Install dependence with a new conda env called corl2022
conda create -n corl2022 python=3.8
conda activate corl2022
pip3 install gym open3d scipy opencv-python numpy nlopt scipy transforms3d imageio nvitop setuptools opencv-contrib-python tensorboard --upgrade
pip3 install https://storage1.ucsd.edu/wheels/sapien-dev/sapien-2.0.0.dev20220531-cp38-cp38-manylinux2014_x86_64.whl
pip3 install torch torchvision --extra-index-url https://download.pytorch.org/whl/cu113
pip3 install wandb moviepy h5py
```

Download data file for hand detector and scene
from [Google Drive Link](https://drive.google.com/file/d/1R88RCkjBJHldKUpPWaiRUqERp1d149MV/view?usp=sharing).
Place the `day.ktx` at `assets/misc/ktx/day.ktx`.

```shell
# Download ktx for visualization purpose
pip3 install gdown
gdown https://drive.google.com/uc?id=1R88RCkjBJHldKUpPWaiRUqERp1d149MV
unzip teleop_static_file.zip
```

## File Structure

- `hand_teleop`: main entry for the environment, utils, and other staff needs for teleoperation and RL training.
- `assets`: robot and object models, and other static files
- stable_baselines3: modified from the original stable_baseline3 package for RL training
- `main`: entry file

## Acknowledge

- For all the single file code that we download from the Github, we cite the author and the source on each individual
  file.
- For rl training, we leverage the [stable_baseline3](https://github.com/DLR-RM/stable-baselines3) code with our
  modification
- For object assets, we use the [YCB Dataset](https://www.ycbbenchmarks.com/)
  and [ShapeNet Dataset](https://shapenet.org/) that are included in the code.

## Environment Using Examples:

```shell
export PYTHONPATH=`pwd`:$PYTHONPATH
python3 main/example_use_imagination_env.py # For the environemnt with our proposed imaged finger point cloud
python3 main/example_use_pc.py # For the environment with the normal point cloud
```

## Evaluation with Pre-Trained Model

```shell
export PYTHONPATH=`pwd`:$PYTHONPATH
python3 main/eval_relocate_policy.py 
```

## Train from scratch

```shell
export PYTHONPATH=`pwd`:$PYTHONPATH
python3 main/train_relocate_pc_img.py # Train policy with imagined point cloud and normal point cloud
python3 main/train_relocate_pc.py # Train policy with normal point cloud 
```





