## Installation

### Mem1 environment
```bash
# Install requirements. This may take a few minutes to complete.
conda env create -f environment.yml

# verl
conda activate mem1
pip install -e .
```

### Retriever environment (optional)
If you would like to call a local retriever as the search engine, you can install the environment as follows. (We recommend using a separate environment.)
```bash
conda create -n retriever python=3.10
conda activate retriever

# we recommend installing torch with conda for faiss-gpu
conda install pytorch==2.4.0 torchvision==0.19.0 torchaudio==2.4.0 pytorch-cuda=12.1 -c pytorch -c nvidia
pip install transformers datasets pyserini

## install the gpu version faiss to guarantee efficient RL rollout
conda install -c pytorch -c nvidia faiss-gpu=1.8.0

## API function
pip install uvicorn fastapi
```

## Quick Start

1. Download the necessary data:
```bash
cd setup/
python download.py
```

2. Pre-process the multi-objective QA dataset (change the batch size for varying number of objectives)
```bash
cd Mem1/
bash gen_data/scripts/data_process_multi.sh --batch_size 2
```

3. Launch a local retrieval server:
```bash
# First remember to change the `file_path` field of `Mem1/train/retrieval_launch.sh` to the path of your RAG files
cd Mem1/train/
bash retrieval_launch.sh
```

4. Train the Mem1 model:
```bash
cd Mem1/train/
bash train_ppo.sh
```
