# Semantic Segmentation Fine-tuning

This directory contains the code and configurations for fine-tuning pre-trained models on semantic segmentation tasks. The fine-tuning process adapts the pre-trained models to perform semantic segmentation in specific test environments.

<mark>**Note:** This code base and the provided dataset are for demonstration purposes only. The full code, detailed instructions, and all datasets download links will be provided upon the official release.</mark>

## Fine-tuning Process

To fine-tune the pre-trained models for semantic segmentation, execute the following commands:

```bash
cd segmentation
OMP_NUM_THREADS=1 torchrun --nproc_per_node=8 run_finetuning_semseg.py \
    --config cfgs/<config>.yaml \ # path to the transfer config for each dataset
    --finetune <path/to/pretrained/weights.pth> \ # checkpoints from the pre-trained model
    --wandb_entity <wandb-entity-name> \
    --wandb_project <wandb-project-name> \
    --wandb_run_name <wandb-run-name> \ # if not specified the run name will be set automatically
    --output_dir <path-to-save-outputs> # directory where model checkpoints and logs will be saved
```

> **Note:** Adjust the `--nproc_per_node` parameter based on the number of GPUs available on your system. For example, if you have 4 GPUs, set it to 4.

For more detailed configuration options, you can modify the corresponding YAML file specified in the `--config` parameter. The YAML files contain various hyperparameters and training settings that you can customize according to your needs.

To only evaluate the fine-tuned model, you can run the following commands:

```bash
cd segmentation
OMP_NUM_THREADS=1 torchrun --nproc_per_node=8 run_finetuning_semseg.py \
    --eval \
    --config cfgs/<config>.yaml \ # path to the transfer config for each dataset
    --finetune <path/to/finetune/weights.pth> \ # path to the finetuned weights
    --wandb_entity <wandb-entity-name> \
    --wandb_project <wandb-project-name> \
    --wandb_run_name <wandb-run-name> \ # if not specified the run name will be set automatically
    --output_dir <path-to-save-outputs> # directory where evaluation results will be saved
```

For model configurations and finetuned weights, please refer to the [Resources](#resources) section.

## Resources

We provide transfer configurations for semantic segmentation fine-tuning for each test space:

| Test Space | Transfer Config |
| ------- | -------- |
| ProcTHOR | [Config](./cfgs/procthor_semseg_base.yaml) |