# Dataless Weight Disentanglement in Task Arithmetic via Kronecker-Factored Approximate Curvature

This guide explains how to:
- Install project dependencies
- Compute the approximate curvature matrix (Fisher/GGN) and cache it
- Create task vectors using the Fisher cache
- Evaluate the saved task vectors

Commands are tailored for Linux with bash. Adjust paths to your local environment.

---

## 1) Install dependencies

Make sure you have Python 3.9+ and an active virtual environment (recommended):

```bash
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt
```

If you use a GPU, ensure you install the correct PyTorch build (CUDA or ROCm) for your setup.

---

## 2) Compute and cache Fisher/GGN

Replace the value of `--fisher_cache` with the path where you want to save the GGN/Fisher (the folder will be created if it does not exist):

```bash
python3 utils/main.py --clip_backbone=ViT-B/32 --batch_size=8 --model=clip_ft_fisher_v2 \
  --fitting_mode=epochs --lr=3e-4 --optimizer=adamw --optim_wd=0.0 --seed=42 \
  --n_epochs=0 --fitting_mode=iters --n_iters=0 \
  --fisher_num_samples_expectation=1 \
  --fisher_cache="/home/your_user/path/to/save_GGN" \
  --train_percent=64 --reg_lambda=0.0 --load_fisher=0 \
  --dataset=seq-8vision --tangent=0 --fisher_use_pca=0
```

Notes:
- `--fisher_cache`: directory where the matrix (GGN/Fisher) will be saved for reuse.
- `--load_fisher=0`: force recomputation; set to 1 to load from cache.
- `--dataset=seq-8vision`: uses the sequential 8-vision-datasets suite.

---

## 3) Create Task Vectors using the Fisher cache

Set `--fisher_cache` to the same path used in the previous step. Task vectors will be saved into the project's checkpoints folder.

```bash
python3 utils/main.py --clip_backbone=ViT-B/32 \
  --notes=vitb32_kfac --batch_size=32 --chunks=4 --model=clip_ft_fisher_v2 --fitting_mode=epochs --optimizer=adamw \
  --scheduler_ntk=cosine --load_fisher=1 --fisher_cache="/home/your_user/path/to/saved_GGN" \
  --lr=0.0003 --optim_wd=0.1 --alpha_merging=8.0 --clip_grad_norm=-1.0 --reg_lambda=100  \
  --seed=42 --save_task_vectors=1 --dataset=seq-8vision
```

Notes:
- `--load_fisher=1`: loads the GGN/Fisher from cache to generate task vectors.
- `--save_task_vectors=1`: saves task vectors under `checkpoints/...` (check the output for the exact path).

---

## 4) Evaluate saved Task Vectors

Set `--load_path` to the folder containing the saved task vectors (e.g., `checkpoints/<run>/load_fisher_path`).

```bash
python3 utils/main.py \
  --load_path="/home/your_user/checkpoints/load_fisher_path" \
  --clip_backbone=ViT-B/32 --tangent=1 \
  --alpha_merging=8.0 \
  --dataset=seq-8vision --model=clip_ft_ntk --eval_only=1 \
  --batch_size=128 --chunks=1 \
  --fitting_mode=iters --n_epochs=0 --n_iters=0 \
  --optimizer=adamw --scheduler_ntk=cosine --lr=0.0003 \
  --clip_grad_norm=-1.0 --optim_wd=0.1 --seed=42
```
