
## 1. Set up environment

**a. Create a conda virtual environment and activate it.**
```shell
conda create -n open-mmlab python=3.8 -y
conda activate open-mmlab
```

**b. Install PyTorch and torchvision following the [official instructions](https://pytorch.org/).**
```shell
pip install torch==1.9.1+cu111 torchvision==0.10.1+cu111 torchaudio==0.9.1 -f https://download.pytorch.org/whl/torch_stable.html
# Recommended torch>=1.9

```

**c. Install mmcv-full.**
```shell
pip install mmcv-full==1.4.0
#  pip install mmcv-full==1.4.0 -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.9.0/index.html
```

**d. Install mmdet and mmseg.**
```shell
pip install mmdet==2.14.0
pip install mmsegmentation==0.14.1
```

**e. Install mmdet3d from source code.**
```shell
git clone https://github.com/open-mmlab/mmdetection3d.git
cd mmdetection3d
git checkout v0.17.1 # Other versions may not be compatible.
pip install -v -e .
```

**f. Install timm.**
```shell
pip install timm
```



**h. Prepare pretrained resnet50 and resnet18 models.**
```shell
cd VoxFormer && mkdir ckpts && cd ckpts
```
Download the pretrained resnet50 and resnet18.



## 2. Prepare data
Download semantic-kitti dataset and KITTI-360 dataset. We use the label provided by [Voxformer](https://github.com/NVlabs/VoxFormer/blob/main/docs/prepare_dataset.md) and [OCCBench-KITTI360](https://github.com/ai4ce/OCCBench/tree/main/dataset/KITTI-360) for our experiment.
Symlink the dataset root to ./kitti and ./kitti360.
```
ln -s [SemanticKITTI root] ./kitti
ln -s [KITTI-360 root] ./kitti360
```
Copy the calibration files to dataset
```
cp -r data_odometry_calib/sequences kitti/dataset
```
The data is organized in the following format:

```
/kitti/dataset/
          └── sequences/
          │       ├── 00/
          │       │   ├── poses.txt
          │       │   ├── calib.txt
          │       │   ├── image_2/
          │       │   ├── image_3/
          │       |   ├── voxels/
          │       |         ├ 000000.bin
          │       |         ├ 000000.label
          │       |         ├ 000000.occluded
          │       |         ├ 000000.invalid
          │       |         ├ 000005.bin
          │       |         ├ 000005.label
          │       |         ├ 000005.occluded
          │       |         ├ 000005.invalid
          │       ├── 01/
          │       ├── 02/
          │       .
          │       └── 21/
          └── labels/
                  ├── 00/
                  │   ├── 000000_1_1.npy
                  │   ├── 000000_1_2.npy
                  │   ├── 000005_1_1.npy
                  │   ├── 000005_1_2.npy
                  ├── 01/
                  .
                  └── 10/

/kitti360/
|   |-- data_2d_raw
|   |   |-- 2013_05_28_drive_0000_sync # train:[0, 2, 3, 4, 5, 7, 10] + val:[6] + test:[9]
|   |   |   |-- image_00
|   |   |   |   |-- data_rect # RGB images for left camera
|   |   |   |   |   |-- 000000.png
|   |   |   |   |   |-- 000001.png
|   |   |   |   |   |-- ...
|   |   |   |   |-- timestamps.txt
|   |   |   |-- image_01
|   |   |   |   |-- data_rect # RGB images for right camera
|   |   |   |   |   |-- 000000.png
|   |   |   |   |   |-- 000001.png
|   |   |   |   |   |-- ...
|   |   |   |   |-- timestamps.txt
|   |   |   |-- voxels # voxelized point clouds
|   |   |   |   |-- 000000.bin # voxelized input
|   |   |   |   |-- 000000.invalid # voxelized invalid mask
|   |   |   |   |-- 000000.label  #voxelized label
|   |   |   |   |-- 000005.bin # calculate every 5 frames 
|   |   |   |   |-- 000005.invalid
|   |   |   |   |-- 000005.label
|   |   |   |   |-- ...
|   |   |   |-- cam0_to_world.txt
|   |   |   |-- pose.txt # car pose information
|   |   |-- ...
|   |   |-- 2013_05_28_drive_0010_sync 
|   |-- preprocess # preprocessed downsampled labels
|   |   |-- labels # not unified
|   |   |   |-- 2013_05_28_drive_0000_sync 
|   |   |   |   |-- 000000_1_1.npy # original labels
|   |   |   |   |-- 000000_1_8.npy # 8x downsampled labels
|   |   |   |   |-- 000005_1_1.npy
|   |   |   |   |-- 000005_1_8.npy
|   |   |   |   |-- ...
|   |   |   |-- ... 
|   |   |   |-- 2013_05_28_drive_0010_sync
|   |-- calibration # preprocessed downsampled labels
|   |   |-- calib_cam_to_pose.txt
|   |   |-- calib_cam_to_velo.txt
|   |   |-- calib_sick_to_velo.txt
|   |   |-- image_02.yaml
|   |   |-- image_03.yaml
|   |   |-- perspective.txt

```

## 3. Generate probabilistic voxel grid map
**Depth&&UQ Prediction**
We modifiled [MobileStereoNet3d](https://github.com/cogsys-tuebingen/mobilestereonet) to obtain the depth and uncertainty. 

**Requirements**
The code is tested on:
- Ubuntu 22.04
- Python 3.8 
- PyTorch 1.9.1 
- CUDA 11.3


**Depth&&UQ Prediction**

```shell
./UNCERTAINTY_AWARE_OCC/preprocess/image2depth.sh
```
**Depth&&UQ to single-variate Gaussian distribution**

```shell
./UNCERTAINTY_AWARE_OCC/preprocess/depth2lidar.sh
```
**Prepare ray intersection information**
```shell
python ./UNCERTAINTY_AWARE_OCC/preprocess/utils/ray_intersec.py $seq
```
**Probabilistic geometry projection**
```shell
./UNCERTAINTY_AWARE_OCC/preprocess/lidar2voxel_tem.sh
```
## 4. Train and Eval
**Train geometry completion model with 4 GPUs**
```
./tools/dist_train.sh ./projects/configs/voxformer/qpn_uq_q10t1s.py 4
./tools/dist_train.sh ./projects/configs/voxformer/qpn_uq_q10t1s_kitti360.py 4
```

**Eval geometry completion model with 4 GPUs**
```
./tools/dist_test.sh ./projects/configs/voxformer/qpn_uq_q10t1s.py ./path/to/ckpts.pth 4
./tools/dist_test.sh ./projects/configs/voxformer/qpn_uq_q10t1s_kitti360.py ./path/to/ckpts.pth 4
```
**Train Depth-UP OCC model with with 4 GPUs** 
```
./tools/dist_train.sh ./projects/configs/voxformer/voxformer-S_s1_uq_stage2_qpn_q10t1s.py 4
./tools/dist_train.sh ./projects/configs/voxformer/voxformer-UQ_kitti360.py 4
```

**Eval Depth-UP OCC model with 4 GPUs**
```
./tools/dist_test.sh ./projects/configs/voxformer/voxformer-S_s1_uq_stage2_qpn_q10t1s.py ./path/to/ckpts.pth 4
./tools/dist_test.sh ./projects/configs/voxformer/voxformer-UQ_kitti360.py ./path/to/ckpts.pth 4
```
## 5. HierarchicalConformal Prediction
**Geometric level**
```
python cp_1_ours.py
```

**Semantic level**
```
python sem_cp_1_ours.py
```