# DINOReg: Strong Point Cloud Registration with Vision Foundation Model

Code implementation of the paper:

DINOReg: Strong Point Cloud Registration with Vision Foundation Model

## Installation

Please use the following command for installation.

```bash
conda create -n dinoreg python==3.12.11
conda activate dinoreg

pip install -r requirements.txt
pip install -e .
```

Code has been tested with Ubuntu 22.04, Python 3.12.11, PyTorch 2.7.1 and CUDA 12.6

## Pre-trained Weights

Due to the file size limit, we provide the pre-trained weight on RGBD-3DMatch dataset in `weights` directory.

## RGBD-3DMatch

### Data preparation

Due to the file size limit, we cannot provide full RGBD-3DMatch & RGBD-3DLoMatch datasets. However, you can use 'datasets/prepare_rgbd_3dmatch.py' to generate the datasets from original 3DMatch scanning data. The data should be organized as follows:

```text
--RGBD_3DMatch--metadata
             |--data--train--7-scenes-chess--cloud_bin_0.png
                   |      |               |--cloud_bin_0.pth
                   |      |--...          |--...
                   |--test--7-scenes-redkitchen--cloud_bin_0.png
                         |                    |--cloud_bin_0.pth
                         |--...               |--...
```

After that, you need to change the 'data.dataset_root' in 'config.py' to the actual path of the dataset.

We provide data that are alreadly preprocced from the original 3DMatch dataset. If you are interested in the preprocessing of the data, please refer to 'datasets/prepare_rgbd_3dmatch.py'.

### Training

The code for RGBD_3DMatch is in `experiments/DINOReg`. Besides, the code of the super version of DINOreg is in `experiments/DINOReg_Super`, which uses DINOv2-base as the visual backbone. Use the following command for training.

```bash
CUDA_VISIBLE_DEVICES=0 python trainval.py
```

The pretrained model of DINOv2 will be automatically downloaded from huggingface.

### Testing

Use the following command for testing.

```bash
# RGBD-3DMatch
CUDA_VISIBLE_DEVICES=0 ./eval.sh EPOCH RGBD_3DMatch
# RGBD-3DLoMatch
CUDA_VISIBLE_DEVICES=0 ./eval.sh EPOCH RGBD_3DLoMatch
```

`EPOCH` is the epoch id.

We also provide pretrained weights in `weights`, use the following command to test the pretrained weights.

```bash
CUDA_VISIBLE_DEVICES=0 python test.py --snapshot=../../weights/dinoreg.pth.tar --benchmark=RGBD_3DMatch
CUDA_VISIBLE_DEVICES=0 python eval.py --benchmark=RGBD_3DMatch --method=lgr
```

Replace `RGBD_3DMatch` with `RGBD_3DLoMatch` to evaluate on RGBD_3DLoMatch.

## KITTI

### Data preparation

The data should be organized as follows:

```text
--KITTI--metadata
      |--data--00--image_2--000000.png
            |   |        |--...
            |   |--velodyne--000000.bin
            |             |--...
            |--01--image_2--000000.png
            |   |        |--...
            |   |--velodyne--000000.bin
            |             |--...
            |--...
```

After that, you need to change the 'data.dataset_root' in 'config.py' to the actual path of the dataset.

If you are interested in the preprocessing of the data, please refer to 'datasets/prepare_kitti.py'.

### Training

The code for KITTI is in `experiments/DINOReg_KITTI`. Use the following command for training.

```bash
CUDA_VISIBLE_DEVICES=0 python trainval.py
```

### Testing

Use the following command for testing.

```bash
CUDA_VISIBLE_DEVICES=0 ./eval.sh EPOCH
```

`EPOCH` is the epoch id.

We also provide pretrained weights in `weights`, use the following command to test the pretrained weights.

```bash
CUDA_VISIBLE_DEVICES=0 python test.py --snapshot=../../weights/dinoreg_kitti.pth.tar
CUDA_VISIBLE_DEVICES=0 python eval.py --method=lgr
```