# MBPO-MVE Supplementary Material

This repository is a modification of the Facebook `mbrl-lib` to evaluate MBPO-MVE. For historical reasons, this is caused `uasc` or `unrolled actor soft critic`.

The key unrolling implementation is contained entirely in this file:

    mbrl/third_party/unrolled_actor_soft_critic/agent/unrolling.py

To log q-values, you can re-run an experiment with the additional argument `algorithm.log_q_values.from_states` pointing to a directory of a previous experiment run:

    python -m mbrl.examples.main \
        algorithm=uasc overrides=uasc_hopper \
        overrides.actor_train_unroll_merge=mean seed=3 \
        algorithm.log_q_values.from_states="$(pwd)/exp/uasc/gym___Hopper-v2/mean/2021.08.04/111103-lph-H0-3" \
        experiment="qvals_logged"

These values can then be plotted in `graphics/Explaining MVE.ipynb`.
