# Supervised Fine-Tuning (SFT) Guide

This guide introduces how to perform supervised fine-tuning on the System1 model using data generated by `dataprepare.py`.

## Dependencies Installation

Before running training scripts, please ensure all necessary dependencies are installed:

```bash
pip install -r ../../requirements.txt
```

## Data Preparation

First, ensure you have run the `dataprepare.py` script to generate training data:

```bash
python dataprepare.py
```

This will generate `train.json` and `val.json` files.

## Model Training

Use the following command to start supervised fine-tuning training:

```bash
python sft_train.py
```

The training script will use Qwen/Qwen3-1.7B as the base model and apply LoRA fine-tuning through the PEFT library. Training checkpoints will be saved in the `./sft_checkpoints` directory.

## Model Structure

To allow System1 and System2 to share the same base model while System1 loads LoRA weights, we adopt the following structure:

- Base model weights are stored in the model repository
- LoRA weights are stored in the `./sft_checkpoints` directory
- During inference, the base model is loaded first, followed by the LoRA weights

This structure enables System1 and System2 to share the same base model while loading different adapter weights.

## Model Evaluation

Use the `eval.py` script to evaluate model performance:

### Evaluate Original Qwen Model

```bash
cd ../../../DyBBT
python finetune/sft/eval.py \
    --model_name_or_path Qwen/Qwen3-1.7B \
    --dataset_path data/sft_data/val.json \
    --output_path results/qwen_original_results.json
```

### Evaluate SFT Fine-tuned Model

```bash
cd ../../../DyBBT
python finetune/sft/eval.py \
    --model_name_or_path Qwen/Qwen3-1.7B \
    --adapter_path ./sft_checkpoints/checkpoint-best \
    --dataset_path data/sft_data/val.json \
    --output_path results/sft_finetuned_results.json
```

### Parameters Description

- `--model_type`: Model type, options are `qwen` or `sft`
- `--model_path`: Model path
- `--adapter_path`: Adapter path for SFT model (required only when model_type is sft)
- `--test_data`: Test data path

Evaluation results will display accuracy by domain separately and provide overall average accuracy.