<h2 align="center">Semantic Alignment for Effective Feature Fusion in Real-Time Object Detection</h2>


This is the PyTorch implementation of "Semantic Alignment for Effective Feature Fusion in Real-Time Object Detection".

It is implemented as a plugin module on top of the [RT-DETR](https://github.com/lyuwenyu/RT-DETR) repository.


## Feature Interaction NEtwork (FINE)

The **FINE** module is integrated into the **hybrid encoder** of RT-DETR, improving multi-scale feature interaction and semantic alignment.  
Key modifications are localized to:

- [`src/zoo/rtdetr/fine.py`](src/zoo/rtdetr/fine.py): Implementation of the FINE module.
- [`src/zoo/rtdetr/hybrid_encoder.py`](src/zoo/rtdetr/hybrid_encoder.py): Integration of FINE into the hybrid encoder architecture.



## Environment Setup

We recommend using a dedicated conda environment for reproducibility.

<details>
<summary>Step 1. Create Conda Environment</summary>

```bash
conda create -n fine python=3.12
conda activate fine
```

</details>

<details>
<summary>Step 2. Install Dependencies</summary>

```bash
pip install -r requirements.txt
```

</details>

<details>
<summary>Step 3. Prepare the MS COCO Dataset</summary>

- Download and extract COCO 2017 train and val images.
```
path/to/coco/
  annotations/  # annotation json files
  train2017/    # train images
  val2017/      # val images
```
- Modify config [`img_folder`, `ann_file`](configs/dataset/coco_detection.yml)
</details>



## Training and Evaluation




<details>
<summary>Training</summary>

- Training on a Single GPU:

```shell
# training on single-gpu
export CUDA_VISIBLE_DEVICES=0
python train.py \
-c configs/rtdetr/rtdetr_r50vd_FINE_6x_coco.yml
```

- Training on Multiple GPUs:

```shell
# train on multi-gpu
export CUDA_VISIBLE_DEVICES=0,1,2,3
torchrun --nproc_per_node=4 train.py -c configs/rtdetr/rtdetr_r50vd_FINE_6x_coco.yml
```

</details>

<details>
<summary>Evaluation</summary>

- Evaluation on Multiple GPUs:

```shell
# val on multi-gpu
export CUDA_VISIBLE_DEVICES=0,1,2,3
torchrun --nproc_per_node=4 train.py -c configs/rtdetr/rtdetr_r50vd_FINE_6x_coco.yml -r path/to/checkpoint --test-only
```

</details>



## Training Logs

Training logs for the models evaluated in the experiments are available in the `output` directory.

- [RT-DETRv1 R18 + FINE Training Log](./output/rtdetr_r18vd_FINE_6x_coco/train.log)
- [RT-DETRv1 R50 + FINE Training Log](./output/rtdetr_r50vd_FINE_6x_coco/train.log)


