# Learning Memory-Based Policies for Robust POMDPs
Robust POMDP planning framework using RNNs and FSCs.

## Running inside docker (RECOMMENDED)

We ran our experiments inside a Docker container and provide it for straightforward reproducing of the results.

### Build

Build the image with:

```bash
docker build --build-arg make_threads=4 -t rnn-rpomdp:1.8.1-2.10.1 . -f docker/Dockerfile
```

### Run

Run experiments of both rFSCNet and the baseline, including the memory and different configurations with:
```bash
mkdir -p output/cache
docker run -v "$(pwd):/usr/uPOMDP/uPOMDP" --name run-robust-and-baseline rnn-rpomdp:1.8.1-2.10.1 bash run_large.sh
```

This will start 20 runs in parallel, beware. Then, the terminal will show a lot of information, and results should gather in `data/output`.

## Manual installation

Instructions are tested on (Docker containers running) Ubuntu 22.04. For manual installation, install dependencies for the storm model checker (instructions for other systems can be found [here](https://www.stormchecker.org/documentation/obtain-storm/dependencies.html)) and Java for PRISM. This can be done by executing the commands of the docker file manually.

# Reproduce experiments

Run the file `run_large.sh` to reproduce the RNN-based experiments shown in the paper.

## SCP

To run all the SCP problems, we provide the convenience files `run_exp.bash` and `run_mem_exp.bash`. Its dependencies are also in the the `exact_requirements.py` file and included in the Dockerfile definition.

For the SCP method, the following should work:

```bash
docker run -v "$(pwd):/usr/uPOMDP/uPOMDP" --name run-robust-and-baseline rnn-rpomdp:1.8.1-2.10.1 bash -c "cd upomdp_cvx_SCP && run_exp.bash && run_mem_exp.bash"
```

# Gurobi

The model sizes we consider in `gurobi-py` requires a Gurobi license. Fortunately, one can acquire a free academic license for non-commercial use.

The license `gurobi.lic` is expected to be in the same folder as *this* README.md file.

## Python module version

The exact Python versions (a `pip freeze` dump) can be found for reference in the aptly named file `requirements.txt` and are explicitly stated in the Dockerfile.
