<!--
 * @Author: 
 * @Email: 
 * @Date: 2020-04-15 12:09:14
 * @LastEditTime: 2021-05-30 21:39:55
 * @Description: 
--> 

# Synthetic Scene Reconstruction

## Environment Requirements

We use Python 3.7 in Anaconda environment and use pip to install the following packages 
```
torch==1.8.1
matplotlib==3.1.3
numpy==1.18.1 
scikit-image==0.16.2
nltk==3.6.2
```

We use CUDA 10.2 as the nvcc compiler. After installing requied packages, compile and install the neural_renderer package with:
```angularjs
cd ./neural_renderer
python3 setup.py install
```


## Download Synthetic Datasets

Download the dataset from this [link](https://drive.google.com/drive/folders/1RetXggty0GPeT3KkKr-gGhfDA3GzlHyE?usp=sharing) and put them into the *./data* folder.

We synthesis 3 kinds of datasets for VAE, T-VAE and GVAE methods, each of which contains 2 files: random samples and oracle samples. To use the image renderer, you also need to download the .obj files and target image. 


## Train Generative Models

If you want to train the models from scratch, you need to run:
```angularjs
# train VAE
python3 main.py --type=0 --model=grid
# train GVAE
python3 main.py --type=0 --model=grammar
# train T-VAE
python3 main.py --type=0 --model=tree
```

We also provide the pre-trained models (VAE, GVAE, T-VAE) obtained from the stage 1 of SCG in this [link](https://drive.google.com/drive/folders/1cGdOc-JXXdjPveZ-amrAV_IyRkexksZr?usp=sharing) and put them into the *./models* folder.. Then, you can directly generate samples in stage 2.



## Get Samples and Good initalization Points
You can sample from the 3 models (VAE, GVAE, T-VAE) and also get the good initalization points for them.
```angularjs
# sample from VAE
python3 main.py --type=5 --model=grid
# sample from GVAE
python3 main.py --type=5 --model=grammar
# sample from T-VAE
python3 main.py --type=5 --model=tree
# get good initialization of VAE
python3 main.py --type=4 --model=grid
# get good initialization of GVAE
python3 main.py --type=4 --model=grammar
# get good initialization of T-VAE
python3 main.py --type=4 --model=tree
```


## Different Types of Searching

1. Direct search in the physical space:
```angularjs
# with a random initialization 
python3 main.py --type=2
# with a good initialization 
python3 main.py --type=2 --oracle
```

2. Search in the latent space of VAE:
```angularjs
# with a random initialization 
python3 main.py --type=1 --model=grid
# with a good initialization 
python3 main.py --type=1 --model=grid --oracle
```

3. Search in the latent space of GVAE:
```angularjs
# with a random initialization 
python3 main.py --type=1 --model=grammar
# with a good initialization 
python3 main.py --type=1 --model=grammar --oracle
```

4. Search in the latent space of T-VAE without knowledge integration:
```angularjs
# with a random initialization 
python3 main.py --type=1 --model=tree
# with a good initialization 
python3 main.py --type=1 --model=tree --oracle
```

5. Search in the latent space of T-VAE with knowledge integration:
```angularjs
python3 main.py --type=3
```
