# ICLR 2021 ID 2721

Please use the code for review only. Not disclosed. Many thanks again. 

- Anonymous Supplementary ID 2721 -- Causal Inference Q-Network: Toward Resilient Reinforcement Learning

- Please check the appendix in the main pdf for reproducible parameter details.

- We will open source the code. 

## Run Reproducivble Exp.

### Part 1. Gym setup

- conda create --name drlnd python=3.6

- source activate drlnd

Use the open source Udacity baseline setup

- git clone https://github.com/udacity/deep-reinforcement-learning.git

- cd deep-reinforcement-learning/python

- pip install .

#### State Cartpole

$ cd 1_env_cartpole

$ python 3-dqn-c-runme.py

### Unity Setup (no repulicate worker id)

#### Banana 3D

$ cd 2_env_banana/Run_Code

$ python 1-dqn-rep-banana-runme.py

## Part 2. Rendering!

$ `export DISPLAY=:0`  

### Monitor Rendering - LunarLander

$ `python 0-lunarlander-runme.py`

#### Pixel Cartpole

$ `python 0-dqn-pixel-runme.py`

### Part 2. ATE Computation


    numpy
    scipy
    scikit-learn
    pandas
    networkx (for analyzing causal graphs)
    matplotlib (for general plotting)
    sympy (for rendering symbolic expressions)


$ `pip install dowhy`

$ `cd CausalATE`

$ `python causal_analysis/generate_causal_data.py`