# Learnable Graph Convolutional Attention Networks

This code reproduces the experiments from the submission Learnable Graph Convolutional Attention Networks.



## GraphGym experiments

We rely on the GraphGym framework to run the experiments with the synthetic and real-world data.


### Requirements
Create conda environment and activate it:

```
conda create --name lcat python=3.9.7 --no-default-packages
conda activate lcat 
```

Pytorch 1.10.0
```
cd graphgym

conda install pip
pip install torch==1.10.0 torchvision==0.11.1 torchaudio==0.10.0
pip install torch-scatter torch-sparse torch-cluster torch-spline-conv torch-geometric -f https://data.pyg.org/whl/torch-1.10.0+cpu.html
pip install -r requirements.txt
pip install pandas
pip install seaborn
pip uninstall graphgym

python setup.py install
```

In order to test the installation concluded sucessfully, go to `gatv3-dev/graphgym/run` and run

```
export PYTHONPATH='..'
python main.py --cfg configs/gcnbasic.yaml --repeat 2
python main.py --cfg configs/csbm_24.yaml --repeat 2
```
### Synthetic


The following scripts create the submission files required to run the experiments for Section 6.1 and Appendix B. They should be from  `gatv3-dev/graphgym/run`. They will also create all the configuration files (i.e., yaml files)  needed to be run inside the folder `graphgym/run/configs2`.


```train
# Experiments Section 6.1 and Appendix B: fix parameters
# ./scripts/run_general.sh CONFIG_BUDGET GRID DEVICE SEEDS

./scripts/run_general_no_budget.sh csbm_24 csbm_ansatz_vary_q_v2 cpu 1
./scripts/run_general_no_budget.sh csbm_24 csbm_ansatz_vary_mu_v2 cpu 1
./scripts/run_general_no_budget.sh csbm_24 csbm_ansatz_vary_q_v2_folds cpu 1
./scripts/run_general_no_budget.sh csbm_24 csbm_ansatz_vary_mu_v2_folds cpu 1
```


```train
# Experiments Appendix B.1: learning C and lambda parameters
# ./scripts/run_general_no_budget.sh CONFIG_BUDGET GRID DEVICE SEEDS

./scripts/run_general_no_budget.sh csbm_24 csbm_vary_q_v2 cpu 4
./scripts/run_general_no_budget.sh csbm_24 csbm_vary_q_ind_v2 cpu 4
./scripts/run_general_no_budget.sh csbm_24 csbm_vary_mu_v2 cpu 4
./scripts/run_general_no_budget.sh csbm_24 csbm_vary_mu_ind_v2 cpu 4
```

### Real world data


The following scripts create the submission files required to run the experiments for Section 6.2 and Appendix D. They should be from  `gatv3-dev/graphgym/run`. They will also create all the configuration files (i.e., yaml files)  needed to be run inside the folder `graphgym/run/configs2`. 

The output configuration of hyperparameters will match the number of parameters in `configs/gcnbasic.yaml`. 

Running the following script will automatically download the datasets.

```train
./scripts/run_general.sh CONFIG_BUDGET GRID DEVICE

./scripts/run_general.sh gcnbasic node_1 cpu
./scripts/run_general.sh gcnbasic node_2 cpu
./scripts/run_general.sh gcnbasic node_3 cpu
./scripts/run_general.sh gcnbasic node_ind cpu
./scripts/run_general.sh gcnbasic node_mlp cpu
```


## OGB experiments

### Requirements

We provide the DockerFile used for our experiments in the root folder.

Alternatively, dependencies should be easily installed with the following commands (assuming cuda 11.3 is available):

```setup
pip install torchvision==0.11.1+cu113 -f https://download.pytorch.org/whl/torch_stable.html
pip install torch-scatter torch-sparse torch-cluster torch-spline-conv torch-geometric \
        -f https://data.pyg.org/whl/torch-1.10.0+cu113.html
pip install -r requirements.txt
pip install dgl-cu113 dglgo -f https://data.dgl.ai/wheels/repo.html
```

### OGB - MAG

To train the models on the mag dataset run the following:

```train
cd mag_products

# GCN
python mag_exp.py --type GCN --num_heads 1 --convolve \
            --share-weights-score --share-weights-value \
            --use_layer_norm --use_residual
            
# GAT options
## --type GAT GAT2
## --convolve : whether to convolve the attention score
## --lambda-policy learn12 : whether to use L-CAT
## --num_heads : number of heads to use

## For example, L-CAT with 8 heads
python mag_exp.py --type GAT --num_heads 8  --convolve --lambda-policy learn12 \
        --share-weights-score --share-weights-value --use_layer_norm --use_residual
```

### OGB - Products

To train the models on the products dataset run the following:

```train
cd mag_products

# GCN
python products_exp.py --type GCN --num_heads 1 --convolve \
            --share-weights-score --share-weights-value
            
# GAT options
## --type GAT GAT2
## --convolve : whether to convolve the attention score
## --lambda-policy learn12 : whether to use L-CAT
## --num_heads : number of heads to use

## For example, L-CAT with 8 heads
python products_exp.py --type GAT --num_heads 8 --convolve --lambda-policy learn12 \
          --share-weights-score --share-weights-value
```

### OGB - Proteins

To train the models on the proteins dataset run the following:

```train
cd proteins

# GCN
python proteins_exp.py --type GCN --n-heads 1 --convolve \
            --share-weights-score --share-weights-value

# GAT options
## --type GAT GAT2
## --convolve : whether to convolve the attention score
## --lambda-policy learn12 : whether to use L-CAT
## --n-heads : number of heads to use

## For example, L-CAT with 8 heads
python proteins_exp.py  --type GAT --n-heads 8 --convolve --lambda-policy learn12 \
            --share-weights-score --share-weights-value
```

### OGB - ArXiv

To train the models on the arxiv dataset run the following:

```train
cd arxiv

# GCN
python gnn.py --model gatv2 --convolve --gcn-mode --runs 5 --epochs 1500

# GAT options
## --model gatv1 gatv2
## --convolve : whether to convolve the attention score
## --lambda-policy learn12 : whether to use L-CAT
## --heads : number of heads to use

## For example, L-CATv2 with 1 head
python gnn.py --model gatv2  --heads 8 --convolve --lambda-policy learn12 --runs 5 --epochs 1500
```

### OGB - ArXiv - robustness to noise

Same arguments as in arxiv (although we used 500 epochs). 
The noise level can be controlled using ``--noise-level 0.5`` where 0.5 is the value for sigma.

### OGB - proteins - robustness to initialization

Same arguments as in proteins above. 
There is no argument to change the initialization, one only needs to comment/uncomment the code 
in the `reset_parameters` function on the `gatv3/dgl` folder. 

---

- Code for the real data section was adapted from https://github.com/snap-stanford/GraphGym
- Code for arxiv was adapted from https://github.com/snap-stanford/ogb/
- Code for mag, products, and proteins was adapted from https://github.com/tech-srl/how_attentive_are_gats
