# AD2S (Enhancing Offline-to-Online Reinforcement Learning by Adaptive Experience Aligned Diffusion Sampling)

The source code of the Adaptive Data Alignment Diffusion Sampling (AD2S) algorithm, proposed by ICLR 2026 submission "Enhancing Offline-to-Online Reinforcement Learning by Adaptive Experience Aligned Diffusion Sampling".

## Getting started

### Install MuJoCo.

- Download [MuJoCo Key](https://www.roboti.us/license.html) and [MuJoCo 2.1 binaries](https://mujoco.org/download/mujoco210-linux-x86_64.tar.gz)
- Extract the downloaded `mujoco210` and `mjkey.txt` into `~/.mujoco/mujoco210` and `~/.mujoco/mjkey.txt`

Add the following environment variables into `~/.bashrc`

```bash
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/.mujoco/mujoco210/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/nvidia
```

### Install Anaconda environment.

To install the necessary environments, we use

```bash
conda create -n ad2s python=3.8
conda activate ad2s
pip install -r requirements/requirements_dev.txt
```

### Run the code.

To run the Cal-ql-AD2S ,use the command

```bash
python -u algorithms/cal_ql_ad2s.py --env hopper-random-v2 --td_guide --density --advantage --seed 48
```

