# run OSD-BC
OSD-BC is implemented based on the official implementation of OptiDICE 

## Installation
Install and use the included Ananconda environment
1. Create conda environment and activate it:
     ```
     cd OSD-BC
     conda env create -f environment.yml
     conda activate optidice
     ```

2. Install `d4rl`:
     ```
     pip install git+https://github.com/rail-berkeley/d4rl@master#egg=d4rl
     ```

## run OSD-BC
     ```
     $cd osd-BC
     $python osdbc.py --env_name=hopper-medium-v2 --device=cuda:0 --seed=0 --alpha=0.0001
     ```


# run OSD-CQL and OSD-TD3BC
OSD-CQL and OSD-TD3BC algorithms are implemented basd on an offline reinforcement learning library OfflineRL-Kit.

## Installation
```shell
git clone https://github.com/yihaosun1124/OfflineRL-Kit.git
cd OfflineRL-Kit
python setup.py install
```
## Run Experiments
You can run OSD-CQL experiments using the following command:
```
$cd osd-BR
$python run_osdcql.py --task=hopper-medium-v2  --device=cuda:0 --seed=0 --osd_alpha=1.0
```

You can run OSD-TD3BC experiments using the following command:
```
$cd osd-BR
$python run_osdtd3bc.py --task=hopper-medium-v2  --device=cuda:0 --seed=0 --osd_alpha=1.0
```

You can run CQL-based baselines using the following command:
```
$cd reweighing-baselines
$python run_reweigh_cql.py --task=hopper-medium-v2  --device=cuda:0 --seed=0 --mode='AW' ('uniform', 'RW','OPER-A', 'top10')
```

You can run TD3BC-based baselines using the following command:
```
$cd reweighing-baselines
$python run_reweigh_td3bc.py --task=hopper-medium-v2  --device=cuda:0 --seed=0 --mode='AW' ('uniform', 'RW','OPER-A', 'top10')
```

You can run DW baseline using the following command:
```
$cd DW
$python run_dw.py --task=hopper-medium-v2  --device=cuda:0 --seed=0 
```


# run OSD-CQL on Antmaze tasks
To obtain a better performance on Antmaze tasks, we also implement OSD-CQL  basd on https://github.com/young-geng/CQL.

## Installation

Install and use the included Ananconda environment
```
$cd osd-CQL
$conda env create -f environment.yml
$source activate SimpleSAC
```

## Run Experiments
You can run OSD-CQL experiments on Antmaze using the following command:
```
$cd osd-CQL
$python -m SimpleSAC.conservative_sac_main \
--env 'antmaze-medium-play-v0' \
--name 'antmaze-medium-play-v0' \
--cql.osd_alpha=1.0 \
--device='cuda:0' \
--seed=0 \
--logging.output_dir './experimental_output'
```

