# Attending to Graph Transformers

This repository is based on [GraphGPS](https://github.com/rampasek/GraphGPS).

### Python environment setup with Conda

```bash
conda create -n graphgps python=3.10
conda activate graphgps

conda install pytorch=1.13 torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia
conda install pyg=2.3 -c pyg -c conda-forge
pip install pyg-lib -f https://data.pyg.org/whl/torch-1.13.0+cu117.html

# RDKit is required for OGB-LSC PCQM4Mv2 and datasets derived from it.  
conda install openbabel fsspec rdkit -c conda-forge

pip install pytorch-lightning yacs torchmetrics
pip install performer-pytorch
pip install tensorboardX
pip install ogb
pip install wandb

# DGL - install for your environment: https://www.dgl.ai/pages/start.html, e.g.,
conda install -c dglteam/label/cu117 dgl 

conda clean --all
```

### Running an experiment with GraphGPS
```bash
conda activate graphgps

# Running an arbitrary config file in the `configs` folder
python main.py --cfg configs/GPS/<config_file>.yaml  wandb.use False
```
We provide the config files necessary to reproduce our experiments under `configs/` (see more below).

### W&B logging
To use W&B logging, set `wandb.use True` and have a `gtransformers` entity set-up in your W&B account (or change it to whatever else you like by setting `wandb.entity`).

### Structural Awareness of GTs
We prepared config files to reproduce the structural awareness experiments under `configs/StructuralAwareness`.
The experimets are performed on three tasks, `Edges`, `Triangles`, `CSL`. In addition, the test set of the `Triangles` task contains both small and large graphs and we benchmark performance for them separately, resulting in `Triangles-small` and `Triangles-large` in the paper. The precise commands used to run these experiments can be found in `run/run_structure_awareness.sh`. To benchmark the `Triangles-small` and `Triangles-large` separately, first run `run/run_structure_awareness.sh` and then copy the folder generated for the `Triangles` runs under `results` into a new folder called `pretrained` and run `run/run_triangles_small_large_split.sh`.

### Reduced Over-smoothing in GTs?
Similar to the structural awareness experiments, we prepared config files to reproduce the experiments on heterophilic datasets under `configs/GPS` and `configs/Graphormer` for Transformer with positional/structural encodigns and optional message-passing and Graphormer, respectively. The precise commands used to run our experiments, including the commands for our hyper-parameter search, can be found in `run/run_heterophilic.sh`.

### Reduced Over-squashing in GTs?
To reproduce our results on the `NeighborsMatch` dataset, go to `bottleneck/`, a folder with a copy from the original repository in [Alon and Yahav, 2021](https://arxiv.org/abs/2006.05205), with only a few modifications to run a transformer.
