# ParCon: Multi-module Parallel Connection \\ for Noise Robust Collaborative Perception

![architecture](./images/architecture.png)

## Requirements

We build the environment using [OpenCOOD](https://opencood.readthedocs.io/en/latest/md_files/installation.html), open source for collaborative perception.

```bash
# Create a conda environment
cd ParCon
conda env create -f environment.yml
conda activate parcon

# Install dependencies
python setup.py develop
pip install -r requirements.txt

pip install spconv-cu113
conda install pytorch-scatter -c pyg
pip3 install natten==0.14.6+torch1120cu113 -f https://shi-labs.com/natten/wheels
pip install --upgrade git+https://github.com/klintan/pypcd.git

pip install torch==1.12.0+cu113 torchvision==0.13.0+cu113 torchaudio==0.12.0 --extra-index-url https://download.pytorch.org/whl/cu113

# Bbx IOU cuda version compile
python opencood/utils/setup.py build_ext --inplace
```

## Datasets
We train and evaluate the model on V2XSet and OPV2V datasets.
The directory structure of saving datasets is as follows:
```sh
\root
 └─ dataset
      ├─ V2XSet_W
      |     ├─ train
      |     ├─ validate
      |     └─ test
      |
      └─ OPV2V
      |     ├─ train
      |     ├─ validate
      |     └─ test
      |
      └─ cooperative-vehicle-infrastructure
            ├─ cooperative  
            ├─ infrastructure-side 
            ├─ vehicle-side
            ├─ train.json 
            └─ val.json
```

## Training

To train the model in the paper, run this command:

```sh
# Training M2C-P on V2XSet or OPV2V
python opencood/tools/train.py --hypes_yaml opencood/hypes_yaml/point_pillar_parcon.yaml

# Training M2C-P on DAIR-V2X
python opencood/tools/train.py --hypes_yaml opencood/hypes_yaml/dair/dair_parcon.yaml
```

## Evaluation

To evaluate the model, run this command:

```sh
# Evaluating Model
python opencood/tools/inference.py --model_dir ${MODEL_DIR} --fusion_method ${Fusion_METHOD}
```

- model_dir: directory path of saved model
- fusion_method: fusion method of saved model [early, intermediate, late]

## Pre-trained Models

Our pretrained models can download at Google Drive :
[Link](https://drive.google.com/drive/folders/1qHTIvMuwp8XW7WlGZ_JwAQ3-CIWpI39T?usp=sharing)


## Results

Model performance on V2XSet, OPV2V, and DAIR-V2X.

![architecture](./images/detection_results.png)

Noise robustness comparison on V2XSet, OPV2V, and DAIR-V2X.

![architecture](./images/Harsh_Noise.png)

## Acknowledgement
- [OpenCOOD](https://github.com/DerrickXuNu/OpenCOOD)
- [OPV2V](https://mobility-lab.seas.ucla.edu/opv2v/)
- [DAIR-V2X](https://github.com/AIR-THU/DAIR-V2X)
- [CoBEVT](https://github.com/DerrickXuNu/CoBEVT)
- [V2XSet, V2X-ViT](https://github.com/DerrickXuNu/v2x-vit)
- [Where2comm](https://github.com/MediaBrain-SJTU/where2comm)
- [CoAlign](https://github.com/yifanlu0227/CoAlign)
