# HOFT: Householder Orthogonal Fine-Tuning

<div align="center">
  <img src="assets/hoft_desc.png" width="1100"/>
</div>

## Overview

Fine-tuning large foundation models often relies on low-rank adaptation techniques. Orthogonal fine-tuning is another promising approach, offering strong generalization but at the cost of increased time and memory requirements. Here, we introduce **Householder Orthogonal Fine-Tuning (HOFT)**, a new orthogonal adaptation method designed to cut down on both runtime and storage overhead. We also explore some theoretical aspects of orthogonal fine-tuning, leading us to propose **Scaled Householder Orthogonal Fine-Tuning (SHOFT)**. We evaluate HOFT and SHOFT across several benchmarks—commonsense reasoning, machine translation, subject-driven image generation, and mathematical reasoning—and show that they match or outperform current state‑of‑the‑art adaptation methods.

## Environment Setup

1. **General Experiments (excluding qualitative subject-driven generation):**

   ```bash
   conda env create -f requirements.yml
   ```

2. **Qualitative Subject-Driven Generation:**

   ```bash
   conda env create -f requirements_sdxl.yml
   ```

Activate the relevant environment before running any experiments. Additionnaly, install custom PEFT library in both conda environments usig the following command 
```bash
pip install peft/
```

## Experiments

### 1. Commonsense Reasoning

1. Navigate to the commonsense reasoning folder:

   ```bash
   cd commonsense_reasoning
   ```

2. Run the fine-tuning script:

   ```bash
   ./finetune.sh $model $peft $lr
   ```

   * `$model`: `llama3.1`, `qwen2.5`, `phi4`, `qwen2.5-14B`
   * `$peft`: `base`, `lora`, `dora`, `hoft`, `shoft`
   * `$lr`: any floating-point value

   **Example:**

   ```bash
   ./finetune.sh qwen2.5 hoft 9e-5
   ```

3. To explore different ranks, use:

   ```bash
   ./finetune_rank_exploration.sh $model $peft $lr $rank
   ```

   * `$rank`: integer value

   **Example:**

   ```bash
   ./finetune_rank_exploration.sh qwen2.5 hoft 9e-5 4
   ```

### 2. Machine Translation

1. Change directory:

   ```bash
   cd machine_translation
   ```

2. Run one of the fine-tuning scripts:

   ```bash
   ./finetune_nllb.sh $source_lang $peft $lr
   ./finetune_llama2.sh $source_lang $peft $lr
   ./finetune_llama3.sh $source_lang $peft $lr
   ```

   * `$source_lang`: `sl`, `de`, `fr`, `lv`
   * `$peft`: `baseline`, `lora`, `dora`, `hoft`, `shoft`
   * `$lr`: any floating-point value

   **Example:**

   ```bash
   ./finetune_nllb.sh sl shoft 5e-4
   ```

3. Download the Common Voice Corpus 4 data for your source language from [Mozilla Common Voice](https://commonvoice.mozilla.org/en/datasets) and place it under:

   ```bash
   covost2/$source_lang/
   ```

### 3. Quantitative Subject-Driven Generation
This section is created based on [HRA](https://github.com/DaShenZi721/HRA).
#### Dataset Preparation

1. Go to the image generation folder and download the DreamBooth dataset:

   ```bash
   cd image_generation
   bash download_dreambooth.sh
   ```
2. After downloading, your folder should look like:

   ```plain
   dreambooth
   ├── dataset
   │   ├── backpack
   │   └── backpack_dog
   ```
3. You can add custom images under `dreambooth/dataset`.

#### Fine-Tuning

```bash
prompt_idx=$0   # 0–24
class_idx=$0    # 0–29
./train_dreambooth_hoft.sh $prompt_idx $class_idx $peft $rank
```

* `$prompt_idx`: integer (0–24)
* `$class_idx`: integer (0–29)
* `$peft`: `hoft` or `shoft`
* `$rank`: integer value

#### Evaluation

```bash
python evaluate.py $image_dir
python get_result.py $image_dir
```

* `$image_dir`: path to generated images

### 4. Qualitative Subject-Driven Generation
This section is created based on [train dreabooth lora sdxl file](https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/train_dreambooth_lora_sdxl.py).
<div align="center">
  <img src="assets/llama.png" width="600"/>
</div>

1. Change into the SDXL folder:

   ```bash
   cd sdxl
   ```
2. Run the SDXL fine-tuning script:

   ```bash
   ./sdxl_finetune_dreambooth.sh $dataset_name $peft
   ```

   * `$dataset_name`: `merve/lego_sets_latest` or `linoyts/3d_icon`
   * `$peft`: `lora`, `hra`, `oft`, `hoft`, `shoft`

### 5. Mathematical Reasoning
This section is created based on [HRA](https://github.com/DaShenZi721/HRA).
1. Navigate to the mathematical reasoning folder:

   ```bash
   cd mathematical_reasoning
   ```
2. Run fine-tuning:

   ```bash
   ./finetune.sh $peft $lr
   ```
   where `$peft` is either `hoft` or `shoft`.
3. Results are saved in `gsm8k-$peft-$lr.txt` and `MATH-$peft-$lr.txt`.

### 6. Mathematical Reasoning with Quantized Models

1. Change to the quantized experiments folder:

   ```bash
   cd qhoft
   ```
2. Run fine-tuning for each model:

   ```bash
   ./finetune_llama2.sh $dataset_name $peft $lr
   ./finetune_llama3.sh $dataset_name $peft $lr
   ```

   * `$dataset_name`: `openai/gsm8k` or `microsoft/orca-math-word-problems-200k`
   * `$peft`: `baseline`, `lora`, `dora`, `hoft`, `shoft`
   * `$lr`: any floating-point value

### 7. Notebooks

In the ``notebooks`` folder there are several jupyter notebooks:
* `approximation_error.ipynb`: Computes the results of Figure 1
* `hyperespherical_energy.ipynb`: Computes the results of Figure 5
* `mean_value.ipynb`: Computes the results of Appendix E.1


