****************************************************************************************************
----------------------------------------------------------------------------------------------------
Due to the size limitation of the supplementary files that can be uploaded to OpenReview.net, we do not provide the data and pre-trained checkpoints here. Please download the full version (1.1GB) from https://drive.google.com/file/d/1cUjWb_AqYHnJG10M3bkEnfokOPcNsc7j/view?usp=sharing to access the data and pre-trained models.
----------------------------------------------------------------------------------------------------
****************************************************************************************************


************************
*****  For  GIL  *****
************************


To train GIL on the mimic training dataset:

python mimic_lstm_GIL_train.py 

[options]
-no_gpu	bool	"Train w/o using GPUs"	default=False
-gpu 	int 	"Select which GPU to use" 	default=0
-lstm_hidden_size	int	"Set the size of LSTM hidden states"	default=1024
-lr_prediction_model	float	"Set learning rate for training the LSTM prediction model"	default=0.0005
-lr_actor	float	"Set learning rate for training the actor"	default=0.0005
-lr_critic	float	"Set learning rate for training the critic"	default=0.0001
-decay_step	int	"Set exponential decay step"	default=500
-decay_rate	float	"Set exponential decay rate"	default=1.0
-decay_lr_actor	float	"Set decay rate the learning rate of the actor"	default=0.965
-decay_lr_critic	float	"Set decay rate the learning rate of the critic"	default=0.965
-training_steps	int	"Set max number of training epochs"	default=2000
-seed	int	"Set random seed"	default=2599
-exploration_prob	float	"Initial probability of random exploration (p3 in Appendix D) in the behavioral policy"	default=0.6
-heuristic_prob	float	"Initial probability of following the heuristic (p2 in Appendix D) in the behavioral policy"	default=0.15
-exploration_prob_decay	float	"Rate of decaying the probability of random exploration in each step"	default=0.95
-heuristic_prob_decay	float	"Rate of decaying the probability of following the heuristic in each step"	default=0.95
-replay_buffer	int	"Size of experience replay buffer for training actor and critic. Default to 10**5 but can be reduced to 10**4 if training too slow or occupies too much RAM."	default=10**5

----------------------------------------------------------------------------------------------------------------

To evaluate GIL on the mimic testing dataset:

python mimic_lstm_GIL_eval.py -ckpt_path <PATH_TO_CKPT_FILES>

[options]
-no_gpu	bool	"Train w/o using GPUs"	default=False
-gpu 	int 	"Select which GPU to use" 	default=0

----------------------------------------------------------------------------------------------------------------

To load the checkpoint of GIL pre-trained using the default parameters:

python mimic_lstm_GIL_eval.py -ckpt_path ./saved_model/MIMIC_LSTM_GIL_CKPT/

----------------------------------------------------------------------------------------------------------------

**************************
*****  For  GIL-D  *****
**************************


To train GIL-D on the mimic training dataset:

python mimic_lstm_GIL-D_train.py 

[options]
-no_gpu	bool	"Train w/o using GPUs"	default=False
-gpu 	int 	"Select which GPU to use" 	default=0
-lstm_hidden_size	int	"Set the size of LSTM hidden states"	default=1024
-lr_prediction_model	float	"Set learning rate for training the LSTM prediction model"	default=0.0005
-lr_actor	float	"Set learning rate for training the actor"	default=0.0005
-lr_critic	float	"Set learning rate for training the critic"	default=0.0001
-decay_step	int	"Set exponential decay step"	default=500
-decay_rate	float	"Set exponential decay rate"	default=1.0
-decay_lr_actor	float	"Set decay rate the learning rate of the actor"	default=0.965
-decay_lr_critic	float	"Set decay rate the learning rate of the critic"	default=0.965
-training_steps	int	"Set max number of training epochs"	default=2000
-seed	int	"Set random seed"	default=2599
-exploration_prob	float	"Initial probability of random exploration (p3 in Appendix D) in the behavioral policy"	default=0.6
-heuristic_prob	float	"Initial probability of following the heuristic (p2 in Appendix D) in the behavioral policy"	default=0.15
-exploration_prob_decay	float	"Rate of decaying the probability of random exploration in each step"	default=0.95
-heuristic_prob_decay	float	"Rate of decaying the probability of following the heuristic in each step"	default=0.95
-replay_buffer	int	"Size of experience replay buffer for training actor and critic"	default=10**4

----------------------------------------------------------------------------------------------------------------

To evaluate GIL-D on the mimic testing dataset:

python mimic_lstm_GIL-D_eval.py -ckpt_path <PATH_TO_CKPT_FILES>

[options]
-no_gpu	bool	"Train w/o using GPUs"	default=False
-gpu 	int 	"Select which GPU to use" 	default=0

----------------------------------------------------------------------------------------------------------------

To load the checkpoint of GIL-D pre-trained using the default parameters:

python mimic_lstm_GIL-D_eval.py -ckpt_path ./saved_model/MIMIC_LSTM_GIL-D_CKPT/

----------------------------------------------------------------------------------------------------------------

**************************
*****  For  GIL-H  *****
**************************


To train GIL-H on the mimic training dataset:

python mimic_lstm_GIL-H_train.py 

-no_gpu	bool	"Train w/o using GPUs"	default=False
-gpu 	int 	"Select which GPU to use" 	default=0
-lstm_hidden_size	int	"Set the size of LSTM hidden states"	default=1024
-lr_prediction_model	float	"Set learning rate for training the LSTM prediction model"	default=0.005
-decay_step	int	"Set exponential decay step"	default=500
-decay_rate	float	"Set exponential decay rate"	default=0.95
-training_steps	int	"Set max number of training epochs"	default=2000
-seed	int	"Set random seed"	default=2599

----------------------------------------------------------------------------------------------------------------

To evaluate GIL-H on the mimic testing dataset:

python mimic_lstm_GIL-H_eval.py -ckpt_path <PATH_TO_CKPT_FILES>

[options]
-no_gpu	bool	"Train w/o using GPUs"	default=False
-gpu 	int 	"Select which GPU to use" 	default=0

----------------------------------------------------------------------------------------------------------------


****************************************************************************************************
----------------------------------------------------------------------------------------------------
Due to the size limitation of the supplementary files that can be uploaded to OpenReview.net, we do not provide the data and pre-trained checkpoints here. Please download the full version (1.1GB) from https://drive.google.com/file/d/1cUjWb_AqYHnJG10M3bkEnfokOPcNsc7j/view?usp=sharing to access the data and pre-trained models.
----------------------------------------------------------------------------------------------------
****************************************************************************************************




