## PEAR ##
### Code repository for PEAR: Primitive enabled Adaptive Relabeling for boosting Hierarchical Reinforcement Learning ###

This is a tensorflow based implementation for our approach PEAR: Primitive enabled Adaptive Relabeling for boosting Hierarchical Reinforcement Learning. The approach leverages a handful of expert demonstrations to generate subgoal dataset using our lower primitive's margin objective Q-function. Subsequently, PEAR uses imitation learning regularization to regularize higher level using joint optimization using hierarchical reinforcement learning and imitation learning.

1. Running demo
  ```Shell
  # For Maze navigation environment
  python experiment/play.py --dir=maze_pear_0 --render=1 --rollouts=10

  # For Pick and place environment
  python experiment/play.py --dir=pick_pear_0 --render=1 --rollouts=10

  # For bin environment
  python experiment/play.py --dir=bin_pear_0 --render=1 --rollouts=10

  # For hollow environment
  python experiment/play.py --dir=hollow_pear_0 --render=1 --rollouts=10

  # For rope manipulation environment
  python experiment/play.py --dir=rope_pear_0 --render=1 --rollouts=10

  # For Franka kitchen environment
  python experiment/play.py --dir=kitchen_pear_0 --render=1 --rollouts=10

  ```

2. Training code
  ```Shell
  # For Maze navigation environment
  python experiment/train.py --env="FetchMazeReach-v1" --logdir="pick_pear_0" --n_epochs=2100 --num_cpu=1 --seed=0 --bc_loss=1 --bc_loss_upper=1  --adversarial_loss=1 --num_hrl_layers=2 --populate=1

  # For Pick and place environment
  python experiment/train.py --env="FetchPickAndPlace-v1" --logdir="pick_pear_0" --n_epochs=2100 --num_cpu=1 --seed=0 --bc_loss=1 --bc_loss_upper=1  --adversarial_loss=1 --num_hrl_layers=2 --populate=1

  # For bin environment
  python experiment/train.py --env="FetchPickAndPlaceBin-v1" --logdir="bin_pear_0" --n_epochs=2200 --num_cpu=1 --seed=0 --bc_loss=1 --bc_loss_upper=1  --adversarial_loss=1 --num_hrl_layers=2 --populate=1

  # For hollow environment
  python experiment/train.py --env="FetchPickAndPlaceHollow-v1" --logdir="hollow_pear_0" --n_epochs=3500 --num_cpu=1 --seed=0 --bc_loss=1 --bc_loss_upper=1  --adversarial_loss=1 --num_hrl_layers=2 --populate=1

  # For rope manipulation environment
  python experiment/train.py --env="RoboticRope-v1" --logdir="rope_pear_0" --n_epochs=600 --num_cpu=1 --seed=0 --bc_loss=1 --bc_loss_upper=1  --adversarial_loss=1 --num_hrl_layers=2 --populate=1

  # For Franka kitchen environment
  python experiment/train.py --env="kitchen-complete-v0" --logdir="kitchen_pear_0" --n_epochs=1900 --num_cpu=1 --seed=0 --bc_loss=1 --bc_loss_upper=1  --adversarial_loss=1 --num_hrl_layers=2 --populate=1

  ```

3. Plot progress
  ```Shell
  # For Maze navigation environment
  python experiment/plot.py --dir1=maze_pear_0:pear --plot_name="maze"

  # For Pick and place environment
  python experiment/plot.py --dir1=pick_pear_0:pear --plot_name="pick"

  # For bin environment
  python experiment/plot.py --dir1=bin_pear_0:pear --plot_name="bin"

  # For hollow environment
  python experiment/plot.py --dir1=hollow_pear_0:pear --plot_name="hollow"

  # For rope manipulation environment
  python experiment/plot.py --dir1=rope_pear_0:pear --plot_name="rope"

  # For Franka kitchen environment
  python experiment/plot.py --dir1=kitchen_pear_0:pear --plot_name="kitchen"
  
  ```
