# Graphormer-SPIS
This repo is the official implementation of "On Structural Expressive Power of Graph Transformers". 


## Introduction
Graphormer-SPIS is a variant of Graphormer proposed in ["Do Transformers Really Perform Badly for Graph Representation?"](https://openreview.net/forum?id=OeWooOxFwDa) with a provably more powerful structural encoding SPIS added into the model.

Theoretically, SPIS has stronger capability in distinguishing non-isomorphic graph pairs than SPD, the structural encoding utilized by original Graphormer. Besides, in the real-world dataset experiments, the new variant of Graphormer with SPIS has shown better performance over graph representation learning tasks on four regression datasets and four classification tasks. The source code used in our real-world experiments have been provided here.


## Requirements and Installation

#### Setup with Conda
```
# create new conda environment
conda create --name graphormer_SPIS python=3.7
conda activate graphormer_SPIS

# install requirements
pip install rdkit-pypi cython
pip install ogb==1.3.1 pytorch-lightning==1.3.0
pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 -f https://download.pytorch.org/whl/torch_stable.html
pip install torch-geometric==1.6.3 ogb==1.3.1 pytorch-lightning==1.3.1 tqdm torch-sparse==0.6.9 torch-scatter==2.0.6 -f https://pytorch-geometric.com/whl/torch-1.7.0+cu110.html
```

## Example Usage

Training and validating Graphormer-SPIS on graph regression datasets including ESOL, QM8, QM9 and ZINC in a 8/1/1 split style:
```
# just take ZINC as an instance
sh zinc.sh
```
Training and validating Graphormer-SPIS on graph classification datasets including PTC-MR, MUTAG, COX2 and PROTEINS in a 10-fold cross validation style:
```
# just take MUTAG as an instance
sh mutag.sh
```
Note that all the files mentioned above can be found in the `/examples` directory. Recommendations of hyper parameters to reproduce our experimental results have been provided in our paper's Appendix.

All datasets except QM8 are obtained through the interface provided in torch-geometric package. The source .csv file of QM8 dataset and the code to transform the molecules in it into torch-geometric style are provided in `/qm8_generator` directory. 
