# From Link *Prediction* to *Forecasting*: Addressing Challenges in Batch-based Temporal Graph Learning

This is the supplement for the submission "From Link *Prediction* to *Forecasting*: Addressing Challenges in Batch-based Temporal Graph Learning" to TMLR. It includes the implementations of our time window-based evaluation method for link forecasting as well as instructions and code to reproduce our results.

## Method Implementations

We provide implementations for our evaluation procedure in commonly used PyTorch libraries to simplify the adoption of our approach. Specifically, we implement a new `DataLoader` called `SnapshotLoader` that replaces the widely used [`TemporalDataLoader`](https://pytorch-geometric.readthedocs.io/en/stable/modules/loader.html#torch_geometric.loader.TemporalDataLoader) in [PyTorch Geometric](https://pytorch-geometric.readthedocs.io/en/stable/). We extend [DyGLib](https://github.com/yule-BUAA/DyGLib) with a command line argument `horizon` that can be used in the evaluation pipeline. The latter was also used for the experiments in this work and can be used to reproduce our results.

### Snapshot-Loader in PyTorch Geometric

To ensure anonymity for a double-blind review, we provide the implmentation and unit-tests for PyTorch Geometric in the directory `PyG/`. We will make the code publicly available via a GitHub-repository and open a Pull Request to integrate our proposed method into PyTorch Geometric after acceptance of the paper.

You can include our changes as follows:
1. Clone the official PyG-repository:
```
git clone https://github.com/pyg-team/pytorch_geometric.git
```
2. Insert our implementations at the following locations:
```
- torch_geometric/loader/snapshot_loader.py
- test/loader/test_snapshot_dataloader.py
```
3. Install the local version by moving into the directory `torch_geometric/` and then executing:
```
pip install .
```

### Time Window-based Evaluation in DyGLib

We include the complete `DyGLib`-repository since it can also be used to reproduce our results. To use our evaluation method, you can add `--horizon XX`. An example with a horizon of 60s is shown below (for more information on how to use `DyGLib`, see `DyGLib/README.md`):

```sh
python evaluate_link_prediction.py --dataset_name "wikipedia" --model_name DyGFormer --negative_sample_strategy "historical" --horizon 60
```

## Reproducibility

### Setup

We include a `.devcontainer`-configuration that can be used to set up a [DevContainer](https://code.visualstudio.com/docs/devcontainers/containers) with VSCode. Another option is to install the necessary dependencies into your (virtual) Python environment: 
```
pip install -r DyGLib/requirements.txt
```

### Data

Only the `Myket` dataset is included as part of the supplement. The remaining datasets can be downloaded [here](https://zenodo.org/records/7213796#.Y1cO6y8r30o). We use the preprocessing features provided by `DyGLib`. For more information, see `DyGLib/README.md` section "Benchmark Datasets and Preprocessing". 

### Experiments

You can run all our experiments as follows:
```
cd DyGLib
bash run_experiments.sh
```

### Further Help

- **Execute Bash-Scripts on Windows:** We recommend Git-Bash for a simple way to run bash-scripts on Windows.
- **Problems with finding Module `utils`:** If you have an error message similar to the following, you might need to add the `DyGLib/utils/`-directory to your Python-Path.
    ```
    Traceback (most recent call last):
        File "/workspaces/supplement_neurips/DyGLib/train_link_prediction.py", line 13, in <module>
            from models.TGAT import TGAT
        File "/workspaces/supplement_neurips/DyGLib/models/TGAT.py", line 6, in <module>
            from utils.utils import NeighborSampler
    ModuleNotFoundError: No module named 'utils'
    ```
- **Error when setting up the DevContainer on Mac or another Machine without NVIDIA GPU:** If you get an error message similar to the following, Docker fails to find an NVIDIA GPU on your machine:
    ```
    Error response from daemon: could not select device driver ‘’ with capabilities: [[gpu]]
    ```
    You can fix this by removing the following lines (line 22 and 23) from `.devcontainer/devcontainer.json`:
    ```
        "--gpus",
		"all"
    ```