# Joint Modeling of Visual Objects and Relations for Scene Graph Generation

## Introduction

This project is an implementation of ``Joint Modeling of Visual Objects and Relations for Scene Graph Generation'' in PyTorch.

### Prerequisites

We develop this project with `Python 3.6`, `PyTorch 1.1.0` and the packages listed in `requirements.txt`. In our project, these packages can be successfully installed and work together under `CUDA/9.0` and `cuDNN/7.0.5`.

### Dataset Preparation

In this work, we evaluate our approach on the [Visual Genome dataset](https://visualgenome.org/). You can follow [this previous project](https://github.com/danfeiX/scene-graph-TF-release/tree/master/data_tools) for data preparation.

### Compilation

Install all the python dependencies using pip:
```
pip install -r requirements.txt
```

Compile the cuda depended modules, e.g. NMS, ROI Pooling, ROI Align and ROI Crop, using following simple commands:

```
cd lib
python setup.py build develop
```

## Training & Evaluation

To train the JM-SGG model, simply run:
```
python train_JM_SGG.py --model_config $model_name$
```

To evaluate the JM-SGG model under the metric *Recall@k*, simply run:
```
python test_JM_SGG.py --model_config $model_name$ --checksession $session_id$ \
                      --checkepoch $epoch_id$ --checkpoint $iter_id$ --top_k $50/100$ \
                      --eval_mode $predcls/sgc/sgg$ --gc --use_mean_field 
```

To evaluate the JM-SGG model under the metric *Mean Recall@k*, simply run:
```
python test_JM_SGG.py --model_config $model_name$ --checksession $session_id$ \
                      --checkepoch $epoch_id$ --checkpoint $iter_id$ --top_k $50/100$ \
                      --eval_mode $predcls/sgc/sgg$ --gc --use_mean_field --mean_recall
```