# Source Code for the ACDA Algorithm and the Interaction Layer

**NOTE: The algorithm called ACDA in the paper is called MBPAC in this
artifact. The name of the algorithm was changed after the results were
collected. Any references to MBPAC under this directory refers to ACDA.**

The dependencies are best installed using the `conda` package manages. This can
be installed from https://github.com/conda-forge/miniforge by downloading one
of their releases.

We provide two environment files with necessary dependencies:

 * `condaenv-cpu.yaml`: Install CPU-only dependencies.
 * `condaenv-cuda.yaml`: Install dependencies with CUDA support. Highly
   recommended if training with ACDA.

From this point on we assume that the CUDA dependencies will be used. Install
them with:

```sh
conda env create -f condaenv-cuda.yaml
conda activate latency-env-cuda
```

All training jobs are provided as makefile rules. Each training job consists of
one algorithm training on a single Gymnasium environment under a specific delay
process. We provide 3 delay processes:

 * `extreme`: the GE(1,23) delay process in the paper,
 * `extreme32`: the GE(4,32) delay process in the paper, and
 * `mm1q`: the MM1 delay process in the paper.

We use noisified versions of MuJoCo environments from Gymnasium that we
evaluate on. These are identified by `noisy-ant`, noisy-humanoid`,
`noisy-halfcheetah`, `noisy-hopper`, and `noisy-walker2d`. These correspond to
the environments `Ant-v4`, `Humanoid-v4`, `HalfCheetah-v4`, `Hopper-v4`, and
`Walker2d-v4`, with 5% action noise added.

The general command to run for starting an ACDA training job is

```sh
TEST_PROFILE=<DELAY_PROCESS> make test-mbpac_memdelay-<ENVIRONMENT>
```

where `<DELAY_PROCESS>` and `<ENVIRONMENT>` is replaced by the delay process
and environment that we want to evaluate on. For example, we run the following
command to evaluate the Humanoid-v4 under the MM1 delay process:

```sh
TEST_PROFILE=mm1q make test-mbpac_memdelay-noisy-humanoid
```
