# Repeatability

- Transactions of Machine Learning Research
- Paper: Formal Verification of Graph Convolutional Networks with Uncertain Node Features and Uncertain Graph Structure

## Installation
This folder contains the code as well as a docker file to run the code in one click (see below).

However, you might need to provide a MATLAB licence.
You can specify the license server (preferred) or a license file:

### Option 1: License server
- Configure the license server in `run.sh` using `-e MLM_LICENSE_FILE=<port>@<url>`.

### Option 2: License file
Download  a Licence file `license.lic` to run the code:
- Create a MATLAB License file: 
	For the docker container to run MATLAB, one has to create a new license file for the container.
	Log in with your MATLAB account at https://www.mathworks.com/licensecenter/licenses/
	Click on your license, and then navigate to
	1. "Install and Activate"
    1. "View activated computers"
	1. "Activate a Computer"
	(...may differ depending on how your licensing is set up).
- Choose:
	- Release: `R2023b`
	- Operating System: `Linux`
	- Host ID: `0242AC11000a` (= Default MAC-Adress of Docker Container) | your host MAC Adress
	- Computer Login Name: `matlab`
	- Activation Label: `<any name>`
- When prompted if the software is already installed, choose "Yes".
- Download the file and place it next to `Dockerfile`
- In `Dockerfile`, uncomment the line about copying the license file into docker
- In `run.sh`, comment option 1 and uncomment option 2.

## Run the code

You can run all results in one click in a docker container using the `run.sh` script.

	./run.sh

If this results in obscure error messages, it might be due to different line breaks in `run.sh` using windows/linux. 
You can fix it using

    sed -i 's/\r$//' run.sh
	
The results will be stored to `./results`.


Alternatively, open this directory in MATLAB, add everything the MATLAB path, and run:

	gnn_main()
	
Note that all required toolboxes for CORA have to be installed first (see <a href="https://cora.in.tum.de/manual">CORA Manual</a>).

### Important Notes

- The evaluation takes several days to complete, mainly while enumerating all graphs.
  You can adjust the settings in `./scripts/gnn_main` in lines 121-154,
  e.g., disable the first experiment or reduce the number of repetitions using `NUM_CHECKS`.
- When running the evaluation in docker, docker might randomly stop if not enough memory is available.
  We provide you with a reference run.



