# *Question-Answering based Re-ranking via Flow Imitation for Cross-modal Image Retrieval*
- Code of "Question-Answering based Re-ranking via Flow Imitation for Cross-modal Image Retrieval" for review

## Note
Our paper is constructed following:
- https://github.com/sgrvinod/a-PyTorch-Tutorial-to-Image-Captioning
- https://github.com/tbmoon/basic_vqa


## Dependencies

- Python 3.7
- Pytorch 1.7.0
- NumPy 1.19.1
- tqdm
- scipy 1.5.2
- scikit-learn 0.23.2

## How to use
**Preparation**
1. Download Dataset

```bash
$ cd VQA/utils
$ chmod +x download_and_unzip_datasets.csh
$ ./download_and_unzip_datasets.csh
$ python resize_images.py --input_dir='../Dataset/Images' --output_dir='../Dataset/Resized_Images'  
$ python make_vacabs_for_questions_answers.py --input_dir='../Dataset'
$ python build_vqa_inputs.py --input_dir='../Dataset' --output_dir='../Dataset'
```

2. VQG, VQA, discriminative modules pre-trainig

- VQA
```bash
$ cd VQA
$ python train.py
```

- VQG
```bash
$ cd VQG
$ python train.py
```

- discriminative module
```bash
$ cd question_discriminator
$ python train.py
```

3. Generating psuedo query from candidate images based on the trained image captionig model(https://github.com/sgrvinod/a-PyTorch-Tutorial-to-Image-Captioning)

4. Calculate initial simialrities based on the conveniotla cross-modal image retrieval methods (both psuedo and test query)

5. Calculate input features (both psuedo and test)
```bash
$ cd PM
$ python InputFeatCalc.py --sim_PATH /path/to/psuedo_initial_sim
$ python InputFeatCalcTest.py --sim_PATH /path/to/test_initial_sim
```

6. Set PATH
- initial similarity path (psuedo)
```
/PM/train.py:line 287 
```
- initial similarity path (test)
```
/PM/eval_PM.py:line 391 
```
- Candidate path
```
/PM/train.py:line 44
/PM/eval_PM.py:line 94
/PM/InputFeatCalc.py:line 80
```

**Training**
```bash
$ cd PM
$ python train.py
```

**Test**
```bash
$ cd PM
$ python eval_PM.py
```

