#  VERIFYING GNNS WITH READOUT IS INTRACTABLE

This repository contains code, data, and notebooks for the article 'VERIFYING GNNS WITH READOUT IS INTRACTABLE''.
It includes experiments, result analysis, and visualizations aimed at understanding how quantization affects model performance and efficiency.

## Requirements

To run this code successfully, please ensure that:

- You have the correct version of **Python**:  
   Required: **Python 3.11.9**

- All required libraries are installed with compatible versions. You can do this by running:


`pip install -r requirements.txt`

## Project Files Overview --- Notebooks
### requirements.txt
Requirements to run a code with right versions of the libraries. Make sure you are using the correct Python version to run this code with PyTorch. You need **Python 3.11.9**, as not all PyTorch versions support all Python versions.

### experiments-with-tensors.ipynb
This notebook provides experiments with tensors using symmetric and asymmetric quantization schemes. It illustrates also the differences between INT8 and QINT8 formats.

### activation-functions-synthetic-data.ipynb
This notebook presents a comprehensive analysis of quantization techniques applied to Aggregate Combined Graph Neural Networks with Global Readout (ACRGNNs) for the synthetic data.

Key components of this notebook include:
- Pre-Trained models: Evaluation of pre-trained GNN models.
- Post-Training Quantization (PTQ): Evaluation of dynamic quantization on pre-trained GNN models.
- Accuracy Metrics: Comparison of model performance before and after quantization.
- Visualization: Interactive plots illustrating quantization effects on accuracy and weight distributions.

The data for these experiments are presented on kaggle.zip. More detailed path: kaggle\input\results_synthetic

### activation-functions-ppi-data.ipynb
This notebook presents a comprehensive analysis of quantization techniques applied to Aggregate Combined Graph Neural Networks with Global Readout (ACRGNNs) for the Protein-Protein Interactions (PPI) benchmark.

Key components of this notebook include:
- Pre-Trained models: Evaluation of pre-trained GNN models.
- Post-Training Quantization (PTQ): Evaluation of dynamic quantization on pre-trained GNN models.
- Accuracy Metrics: Comparison of model performance before and after quantization.
- Visualization: Interactive plots illustrating quantization effects on accuracy and weight distributions.

The data for these experiments are presented on kaggle.zip. More detailed path: kaggle\input\results_ppi


### kaggle.zip
This .zip contains all `.csv` files for the analysis that can be done with `.ipynb`.

More detailed information about `.cvs` files can be found in the file `README_for_analysis.md`.

## Project Files Overview --- Code.py
This work builds on the GNN-Logic framework introduced by Barceló et al. [1], and follows the theoretical foundations from their ICLR 2020 paper on the logical expressiveness of GNNs [2].

### dataset.zip
Here is stored the datasets for the FOC_2 folmulas. Please unzip them on the folder `Supplement_materials\Code\src\data\datasets`

### run_ppi.py
In this file, we refer to the code of the Barcelò et al. [1,2]. Authors described this file in the next way: The results will be printed to console and logged in `src/logging/results_ppi`. A single file will collect the last epoch for each GNN combination.
A file with no extension will be created with the mean of 10 runs for each configuration and the standard deviation.

We add the time measurements of the training time. The results will be printed in the console as `Time taken for {l} layers: {running_time} seconds`. A table with all the results of the running time can be formed after the file `ppi_collect_training_time.py`.

We also provided the `README.md` file from the original code [1] inside `src` folder.

### run_ppi_dptq.py
This Python script takes the trained models generated by `run_ppi.py`, applies the dynamic Post-Traing Quantization (PTQ) [3,4], and collects the data for the Appendix.  
The models' results provided: Accuracy and Loss for the Train, Validation, and Test splits.  

All measurements follow the same procedure:
1. Load the pre-trained model.
2. Run train, validation, and test data through the model.
3. Record accuracy, loss, model size, and inference time.

### main.py
This script is based on and extends the original code by Barcelò et al. [1,2], with our own modifications to support dynamic quantization and batch evaluation of ACR-GNN models on synthetic datasets.

It is designed to generate models for datasets p1, p2, and p3, evaluate their performance, and optionally apply Post-Training Dynamic Quantization (PTQ) using PyTorch.

Key Features
-Loads pre-trained ACR-GNN models (FP32) from disk.
-Evaluates each model on Train, Test1, and Test2 splits.
-Optionally trains models from scratch if enabled.
-Applies PTQ to convert models to quantized versions (QINT8).
-Supports both ReLU and truncated ReLU activation functions.
-Uses torch.quantization.quantize_dynamic for quantization of nn.Linear layers.
-Evaluation results and quantization stats are printed to the console and saved to disk.
-Logs performance metrics and evaluation summaries.


## Technical specification of the laptop
The experiments presented in this project were conducted on two different platforms:

- **Local Laptop**:  For `.py` a Samsung Galaxy Book4 laptop with an Intel Core i7-150U processor, 16 GB RAM, and 1 TB SSD storage. 
- **Kaggle Cloud Platform**.  For `.ipynb` notebooks executed on Kaggle: NVIDIA Tesla P100 GPU (16 GB RAM). Hosted runtime with pre-configured environment


## References
[1] Pablo Barceló, Egor V. Kostylev, Mikaël Monet, Jorge Pérez, Juan L. Reutter, and Juan Pablo Silva.  
**GNN-logic**, GitHub repository, 2021.  
Available at: [https://github.com/juanpablos/GNN-logic](https://github.com/juanpablos/GNN-logic)

[2] Pablo Barceló, Egor V. Kostylev, Mikaël Monet, Jorge Pérez, Juan L. Reutter, and Juan Pablo Silva.  
**The Logical Expressiveness of Graph Neural Networks**, 8th International Conference on Learning Representations (ICLR), 2020.  
Available at: [https://openreview.net/forum?id=r1lZ7AEKvB](https://openreview.net/forum?id=r1lZ7AEKvB)

[3] PyTorch Team.  
  **Dynamic Quantization Recipe**.  
  [https://docs.pytorch.org/tutorials/recipes/recipes/dynamic_quantization.html](https://docs.pytorch.org/tutorials/recipes/recipes/dynamic_quantization.html)

[4] PyTorch Documentation.  
  **Post-Training Dynamic Quantization**.  
  [https://pytorch.org/docs/stable/quantization.html#post-training-dynamic-quantization](https://pytorch.org/docs/stable/quantization.html#post-training-dynamic-quantization)
## License

This project is licensed under the [MIT License](LICENSE).