# RPM: Reasoning-Level Personalization for Black-Box Large Language Models

This repository is the official implementation of "RPM: Reasoning-Level Personalization for Black-Box Large Language Models".

## Requirements

To install requirements:

```setup
pip install -r requirements.txt
```

If there is an any error installing with requirements.txt, you can just install Python with version 3.10.13 and install required libraries while running the code.

## Running the Pipeline

To run the complete pipeline from data preparation to inference:

### LaMP3

```
cd LaMP3
bash script/run_pipeline.sh
```

### GOQA

```
cd GOQA
bash script/run_pipeline.sh
```

Each script automatically executes the entire pipeline process including factor construction, reasoning construction and reasoning-aligned generation.

## Results

Our Framewrok achieves the following performance on :

### LaMP benchmark, GOQA dataset

| Model name | LaMP-2 Accuracy | LaMP-2 F1 | LaMP-3 MAE | LaMP-3 RMSE | LaMP-5 Rouge-1 | LaMP-5 Rouge-L | GOQA Accuracy |
| ---------- | --------------- | --------- | ---------- | ----------- | -------------- | -------------- | ------------- |
| RPM        | 0.561           | 0.463     | 0.259      | 0.548       | 0.492          | 0.416          | 0.852         |
