## Setup:

1. We recommand using virtual env or conda, then:

`pip3 install -r requirements.txt`

2. Create a `.env` file in the project directory, with the following keys:

```
HF_TOKEN
AZURE_API_KEY
AZURE_OPENAI_ENDPOINT
AZURE_INFERENCE_SDK_ENDPOINT
OPENAI_API_KEY (optional)
GEMINI_API_KEY
CLAUDE_API_KEY
GOOGLE_GEOCODING_API_KEY
```

## The VLM-GeoPrivacy Benchmark

Due to submission size limit, images can be access and downloaded at https://anonymous.4open.science/r/VLM-GeoPrivacy-ED33/benchmark/images/. Code is also available at https://anonymous.4open.science/r/VLM-GeoPrivacy-ED33/.

The 1200 images and corresponding metadata and annotations are in the `benchmark/` folder. Model inference results on all splits and settings are located in `benchmark/results/test<SPLIT>/<SETTING>`, where model responses are in `responses/` and the geocoded coordinates are in `coordinates`. To evaluate, run `source scripts/eval.sh`, with adjustable configurations in `eval.sh` (see comments there). API model inference can be done with `source scripts/gen_api.sh`, and open model inference is launched with `gen.sh`. For few-shot experiment described in the paper, run `contextual_aid.sh` and evaluate it with `eval_contextual_aid.sh`.
 