# PA-LoFTR: Local Feature Matching with 3D Position-Aware Transformer

## Installation
```shell
# For full pytorch-lightning trainer features
conda env create -f environment.yaml
conda activate PAloftr

# For the PALoFTR matcher only
pip install torch einops yacs kornia
```

We provide the [download link](https://drive.google.com/drive/folders/1wBBwE97BtU8SdWGSfk5k5oapy2u_e0Kq?usp=sharing) to 
  - scannet data indices (training & validation & testing).
  - 3 pretrained models of indoor dataset. The models are for three implementations of PA-LoFTR. Each is about 60MB.
    * Full PA-LoFTR model.
    * PA-LoFTR model with only 3d pe generator and without depth predictor
    * PA-LoFTR with pose-refine structure


## Run PA-LoFTR demos
The three versions of PA-LoFTR are in the corresponding folders. Source codes and a simple demo are povided.

Note: You need to download the corresponding model weights and put it in the folder `weights/`.

Now follow the steps in `demo/demo_paloftr.ipynb`, you can see the inference results of three versions of PA-LoFTR. The requirements of inputs are a bit different for three versions of models:
* For the case camera extrinsic between images is available, `PA-LoFTR-3dpe-only` or `PA-LoFTR-full` can be used. Generally, they can give better matching qualities.
* For the case camera extrinsic is not available, `PA-LoFTR-pose-refine` can be used to generate matches.
* For all cases, camera intrinsic is needed to for the models. 

## Train PA-LoFTR
Detailed training process will be released soon. The main work is to prepare data in target format. We have provided a sample structure for ScanNet dataset, and the ScanNet scenes datasets are put in the folder `Scannet/scan/`.

To Train PA-LoFTR, you should prepare the config files. The config files we use are provided as `configs/data/scannet_100scene.py` and `configs/loftr/indoor/PAloftr_matcher.py`.

Run the command
```shell
# For training
python depthTrain.py

# For testing
python depthTest.py
```