The code is for paper submission: On Trajectory Augmentations for Off-Policy Evaluation

This folder contains the code to (1) potential sub-trajectories (PSTs) mining and (2) train the VAE-MDP using PSTs as inputs. We provide the augmented PSTs and training checkpoints obtained from the pen-human dataset as an example for checking reproducibility.

Folders:
augmented_dataset -- contains the augmented trajectories
processed_data -- contains the mined PSTs
raw_data -- contains original trajectories and temporal discrete sequences obtained from mtticc
saved_augmented_data -- contains augmented PSTs
saved_dist -- stores (pre-trained) behavior policy checkpoints that can be loaded as behavior policy
saved_models -- stores (pre-trained) OAT checkpoints that can be loaded as trajectory augmentation models

############################################
(1) PSTs Mining 
############################################

Excecute all the the jupyter notebook PSTmining.ipynb
Dependencies:
Python 3
tensorflow 1.15.0
gym 0.21.0
numpy 1.21.2
pandas 1.3.5
csv 1.0
sklearn 1.0.2

############################################
(2) Train/Evaluate the MDP-VAE Model 
############################################
Step1. Excecute all the the jupyter notebook learn_behavior.ipynb
Step2. Excecute all the the jupyter notebook LSTM_VAE_train.ipynb
Step3. Excecute all the the jupyter notebook LSTM_VAE_eval_withBehavior.ipynb

Dependencies:
Python 3
tensorflow 1.15.0
gym 0.21.0
numpy 1.21.2
pandas 1.3.5
csv 1.0
sklearn 1.0.2


****************************************************************************************************
----------------------------------------------------------------------------------------------------
Due to the size limitation of the files, we do not provide all the data, pre-converted PSTs and pre-trained checkpoints here. These data will be made public once manuscript is accepted.
----------------------------------------------------------------------------------------------------
****************************************************************************************************
