# Differentiable Rendering with Reparameterized Volume Sampling

Supplementary materials for the submission.

Note that the 3D experiments refer to color estimates as "Lebesgue" and "Riemann". These are the old names we used to distinguish between sums using random points on a ray $t$ ("horizontal", Riemann) and sums using opacity samples $y = F(t)$ ("vertical", Lebesgue) with $F$ acting as a measure.

## Install & Environment

You can clone our environment using conda and ```enviroment.yaml``` file. Core dependencies are listed here:

- torch >= 1.6
- pytorch-lightning >= 1.5
- pytorch3d >= 0.6
- wandb
- cv2

## 1D Experiments

In ```notebooks/1d_experiments.ipynb``` we illustrate our main contributions for density fields on a single 1D line. We implement a class for spline approximations and reparameterized sampling, showcase the expected color estimates and run a toy optimization task. Our 3D rendering experiments use a similar class for multiple rays in 3D space.

## 3D Experiments

### Training

To run training you need to select a configuration file from the ones available in ```conf/``` folder and run command:

```python train.py gpus=id, conf_path=conf/lebesgue_spline_bin16_8pts.yaml data_conf_path=data_configs/lego.yaml```

Training results will be stored by wandb logger in your own wandb space.

### Evaluation

Again, you need to specify a configuration file. You may also replace the path to checkpoint in wandb with yours by modifying the field: ```trained_ckpt```. Run the following command to evaluate the checkpoint: 

```python test.py gpus=id, conf_path=conf/lebesgue_spline_bin16_8pts.yaml data_conf_path=data_configs/lego.yaml```

### Pretrained models

Checkpoint performance. To reproduce the results run the training script with the corresponding configure files. For LPIPS evaluation please refer to the official repository [https://github.com/richzhang/PerceptualSimilarity](https://github.com/richzhang/PerceptualSimilarity). We used ```lpips_2dirs.py``` script to obtain the numbers in the submission.

| data |   conf   |  PSNR  |  SSIM | LPIPS  |
|------|----------|--------|-------|--------|
| lego | baseline | 27.247 | 0.904 | 0.113  |
| lego |   8pts   | 26.502 | 0.892 | 0.124  |
| lego |  32pts   | 27.286 | 0.904 | 0.122  |
