--------------------------------------------------------
Repeatability Evaluation
Paper: Formal Verification of Graph Neural Networks with Uncertain Node Features and Uncertain Graph Structure
Transactions of Machine Learning Research
Date: 12-Nov-2024 12:21:03
CORA v2024.1.5 (dev-nn)
--------------------------------------------------------
 
Running 5 scripts.. 
 
--------------------------------------------------------
Running 'example_quadMap' ...
 
'example_quadMap' was run successfully!
Saving plots to '/results/240416-122102/plots'..
 
--------------------------------------------------------
Running 'example_invsqrtroot' ...
 
'example_invsqrtroot' was run successfully!
Saving plots to '/results/240416-122102/plots'..
 
--------------------------------------------------------
Running 'example_2_message_passing' ...
Elapsed time is 0.541097 seconds.
Plotting..
 
'example_2_message_passing' was run successfully!
Saving plots to '/results/240416-122102/plots'..
 
--------------------------------------------------------
Running 'evaluation' ...
---
pert_edge: 0.000, delta=0.001, model=models/Enzymes_34035, do_enumeration=1
Seed: 491
Reading network and data in: /code/experiments/models/Enzymes_34035
Successful: 0/20. Skipped 0. Failed: 0; 	 Running data 1/84 (80) with seed=768.. 
Successful: 1/20. Skipped 0. Failed: 0; 	 Running data 2/84 (47) with seed=116.. 
Successful: 2/20. Skipped 0. Failed: 0; 	 Running data 3/84 (3) with seed=484.. 
Successful: 3/20. Skipped 0. Failed: 0; 	 Running data 4/84 (27) with seed=520.. 
Successful: 4/20. Skipped 0. Failed: 0; 	 Running data 5/84 (9) with seed=619.. 
Successful: 5/20. Skipped 0. Failed: 0; 	 Running data 6/84 (6) with seed=239.. 
Successful: 6/20. Skipped 0. Failed: 0; 	 Running data 7/84 (29) with seed=350.. 
Successful: 7/20. Skipped 0. Failed: 0; 	 Running data 8/84 (22) with seed=384.. 
Successful: 8/20. Skipped 0. Failed: 0; 	 Running data 9/84 (54) with seed=871.. 
Successful: 9/20. Skipped 0. Failed: 0; 	 Running data 10/84 (57) with seed=290.. 
Successful: 10/20. Skipped 0. Failed: 0; 	 Running data 11/84 (17) with seed=452.. 
Successful: 11/20. Skipped 0. Failed: 0; 	 Running data 12/84 (68) with seed=353.. 
Successful: 12/20. Skipped 0. Failed: 0; 	 Running data 13/84 (19) with seed=872.. 
Successful: 13/20. Skipped 0. Failed: 0; 	 Running data 14/84 (45) with seed=752.. 
Successful: 14/20. Skipped 0. Failed: 0; 	 Running data 15/84 (15) with seed=74.. 
Successful: 15/20. Skipped 0. Failed: 0; 	 Running data 16/84 (12) with seed=764.. 
Successful: 16/20. Skipped 0. Failed: 0; 	 Running data 17/84 (62) with seed=917.. 
Successful: 17/20. Skipped 0. Failed: 0; 	 Running data 18/84 (61) with seed=53.. 
Successful: 18/20. Skipped 0. Failed: 0; 	 Running data 19/84 (28) with seed=764.. 
Successful: 19/20. Skipped 0. Failed: 0; 	 Running data 20/84 (72) with seed=203.. 
 
Sanity check: 1.00
Successful: 20/20. Skipped 0. Failed: 0.
Verified: 0.75 - Violated: 0.00
 
---
pert_edge: 1.000, delta=0.001, model=models/Enzymes_34035, do_enumeration=1
Seed: 848
Reading network and data in: /code/experiments/models/Enzymes_34035
Successful: 0/20. Skipped 0. Failed: 0; 	 Running data 1/84 (72) with seed=819.. 
Successful: 1/20. Skipped 0. Failed: 0; 	 Running data 2/84 (21) with seed=818.. 
Successful: 2/20. Skipped 0. Failed: 0; 	 Running data 3/84 (23) with seed=269.. 
Successful: 3/20. Skipped 0. Failed: 0; 	 Running data 4/84 (66) with seed=785.. 
Successful: 4/20. Skipped 0. Failed: 0; 	 Running data 5/84 (18) with seed=65.. 
Successful: 5/20. Skipped 0. Failed: 0; 	 Running data 6/84 (15) with seed=528.. 
Successful: 6/20. Skipped 0. Failed: 0; 	 Running data 7/84 (48) with seed=144.. 
Successful: 7/20. Skipped 0. Failed: 0; 	 Running data 8/84 (63) with seed=23.. 
Successful: 8/20. Skipped 0. Failed: 0; 	 Running data 9/84 (55) with seed=533.. 
Successful: 9/20. Skipped 0. Failed: 0; 	 Running data 10/84 (79) with seed=830.. 
Successful: 10/20. Skipped 0. Failed: 0; 	 Running data 11/84 (46) with seed=380.. 
Successful: 11/20. Skipped 0. Failed: 0; 	 Running data 12/84 (7) with seed=957.. 
Successful: 12/20. Skipped 0. Failed: 0; 	 Running data 13/84 (82) with seed=998.. 
Successful: 13/20. Skipped 0. Failed: 0; 	 Running data 14/84 (17) with seed=891.. 
Successful: 14/20. Skipped 0. Failed: 0; 	 Running data 15/84 (49) with seed=273.. 
Successful: 15/20. Skipped 0. Failed: 0; 	 Running data 16/84 (11) with seed=200.. 
Successful: 16/20. Skipped 0. Failed: 0; 	 Running data 17/84 (25) with seed=177.. 
Successful: 17/20. Skipped 0. Failed: 0; 	 Running data 18/84 (32) with seed=639.. 
Successful: 18/20. Skipped 0. Failed: 0; 	 Running data 19/84 (30) with seed=64.. 
Successful: 19/20. Skipped 0. Failed: 0; 	 Running data 20/84 (39) with seed=736.. 
 
Sanity check: 1.00
Successful: 20/20. Skipped 0. Failed: 0.
Verified: 0.65 - Violated: 0.00
 
---
pert_edge: 2.000, delta=0.001, model=models/Enzymes_34035, do_enumeration=1
Seed: 125
Reading network and data in: /code/experiments/models/Enzymes_34035
Successful: 0/20. Skipped 0. Failed: 0; 	 Running data 1/84 (35) with seed=636.. 
Successful: 1/20. Skipped 0. Failed: 0; 	 Running data 2/84 (32) with seed=284.. 
Successful: 2/20. Skipped 0. Failed: 0; 	 Running data 3/84 (30) with seed=822.. 
Successful: 3/20. Skipped 0. Failed: 0; 	 Running data 4/84 (71) with seed=875.. 
Successful: 4/20. Skipped 0. Failed: 0; 	 Running data 5/84 (2) with seed=455.. 
Successful: 5/20. Skipped 0. Failed: 0; 	 Running data 6/84 (14) with seed=623.. 
Successful: 6/20. Skipped 0. Failed: 0; 	 Running data 7/84 (52) with seed=268.. 
Successful: 7/20. Skipped 0. Failed: 0; 	 Running data 8/84 (41) with seed=727.. 
Successful: 8/20. Skipped 0. Failed: 0; 	 Running data 9/84 (36) with seed=380.. 
Successful: 9/20. Skipped 0. Failed: 0; 	 Running data 10/84 (66) with seed=346.. 
Successful: 10/20. Skipped 0. Failed: 0; 	 Running data 11/84 (5) with seed=695.. 
Successful: 11/20. Skipped 0. Failed: 0; 	 Running data 12/84 (4) with seed=586.. 
Successful: 12/20. Skipped 0. Failed: 0; 	 Running data 13/84 (12) with seed=845.. 
Successful: 13/20. Skipped 0. Failed: 0; 	 Running data 14/84 (48) with seed=663.. 
Successful: 14/20. Skipped 0. Failed: 0; 	 Running data 15/84 (19) with seed=591.. 
Successful: 15/20. Skipped 0. Failed: 0; 	 Running data 16/84 (50) with seed=613.. 
Successful: 16/20. Skipped 0. Failed: 0; 	 Running data 17/84 (59) with seed=259.. 
Successful: 17/20. Skipped 0. Failed: 0; 	 Running data 18/84 (56) with seed=414.. 
Successful: 18/20. Skipped 0. Failed: 0; 	 Running data 19/84 (33) with seed=798.. 
Successful: 19/20. Skipped 0. Failed: 0; 	 Running data 20/84 (57) with seed=432.. 
 
Sanity check: 1.00
Successful: 20/20. Skipped 0. Failed: 0.
Verified: 0.55 - Violated: 0.00
 
---
pert_edge: 3.000, delta=0.001, model=models/Enzymes_34035, do_enumeration=1
Seed: 852
Reading network and data in: /code/experiments/models/Enzymes_34035
Successful: 0/20. Skipped 0. Failed: 0; 	 Running data 1/84 (76) with seed=588.. 
Successful: 1/20. Skipped 0. Failed: 0; 	 Running data 2/84 (45) with seed=547.. 
Successful: 2/20. Skipped 0. Failed: 0; 	 Running data 3/84 (68) with seed=338.. 
Successful: 3/20. Skipped 0. Failed: 0; 	 Running data 4/84 (77) with seed=512.. 
Successful: 4/20. Skipped 0. Failed: 0; 	 Running data 5/84 (69) with seed=343.. 
Successful: 5/20. Skipped 0. Failed: 0; 	 Running data 6/84 (18) with seed=394.. 
Successful: 6/20. Skipped 0. Failed: 0; 	 Running data 7/84 (74) with seed=723.. 
Successful: 7/20. Skipped 0. Failed: 0; 	 Running data 8/84 (58) with seed=274.. 
Successful: 8/20. Skipped 0. Failed: 0; 	 Running data 9/84 (70) with seed=786.. 
Successful: 9/20. Skipped 0. Failed: 0; 	 Running data 10/84 (31) with seed=935.. 
Successful: 10/20. Skipped 0. Failed: 0; 	 Running data 11/84 (55) with seed=493.. 
Successful: 11/20. Skipped 0. Failed: 0; 	 Running data 12/84 (50) with seed=933.. 
Successful: 12/20. Skipped 0. Failed: 0; 	 Running data 13/84 (84) with seed=701.. 
Successful: 13/20. Skipped 0. Failed: 0; 	 Running data 14/84 (3) with seed=422.. 
Successful: 14/20. Skipped 0. Failed: 0; 	 Running data 15/84 (14) with seed=353.. 
Successful: 15/20. Skipped 0. Failed: 0; 	 Running data 16/84 (52) with seed=864.. 
Successful: 16/20. Skipped 0. Failed: 0; 	 Running data 17/84 (32) with seed=174.. 
Successful: 17/20. Skipped 0. Failed: 0; 	 Running data 18/84 (59) with seed=359.. 
Successful: 18/20. Skipped 0. Failed: 0; 	 Running data 19/84 (56) with seed=63.. 
Successful: 19/20. Skipped 0. Failed: 0; 	 Running data 20/84 (22) with seed=52.. 
 
Sanity check: 1.00
Successful: 20/20. Skipped 0. Failed: 0.
Verified: 0.45 - Violated: 0.00
 
---
pert_edge: 4.000, delta=0.001, model=models/Enzymes_34035, do_enumeration=1
Seed: 54
Reading network and data in: /code/experiments/models/Enzymes_34035
Successful: 0/20. Skipped 0. Failed: 0; 	 Running data 1/84 (5) with seed=202.. 
Successful: 1/20. Skipped 0. Failed: 0; 	 Running data 2/84 (65) with seed=129.. 
Successful: 2/20. Skipped 0. Failed: 0; 	 Running data 3/84 (10) with seed=220.. 
Successful: 3/20. Skipped 0. Failed: 0; 	 Running data 4/84 (81) with seed=3.. 
Successful: 4/20. Skipped 0. Failed: 0; 	 Running data 5/84 (57) with seed=203.. 
Successful: 5/20. Skipped 0. Failed: 0; 	 Running data 6/84 (68) with seed=696.. 
Successful: 6/20. Skipped 0. Failed: 0; 	 Running data 7/84 (31) with seed=587.. 
Successful: 7/20. Skipped 0. Failed: 0; 	 Running data 8/84 (69) with seed=992.. 
Successful: 8/20. Skipped 0. Failed: 0; 	 Running data 9/84 (52) with seed=268.. 
Successful: 9/20. Skipped 0. Failed: 0; 	 Running data 10/84 (50) with seed=394.. 
Successful: 10/20. Skipped 0. Failed: 0; 	 Running data 11/84 (44) with seed=786.. 
Successful: 11/20. Skipped 0. Failed: 0; 	 Running data 12/84 (42) with seed=716.. 
Successful: 12/20. Skipped 0. Failed: 0; 	 Running data 13/84 (24) with seed=233.. 
Successful: 13/20. Skipped 0. Failed: 0; 	 Running data 14/84 (64) with seed=285.. 
Successful: 14/20. Skipped 0. Failed: 0; 	 Running data 15/84 (22) with seed=275.. 
Successful: 15/20. Skipped 0. Failed: 0; 	 Running data 16/84 (23) with seed=131.. 
Successful: 16/20. Skipped 0. Failed: 0; 	 Running data 17/84 (76) with seed=542.. 
Successful: 17/20. Skipped 0. Failed: 0; 	 Running data 18/84 (3) with seed=960.. 
Successful: 18/20. Skipped 0. Failed: 0; 	 Running data 19/84 (13) with seed=509.. 
Successful: 19/20. Skipped 0. Failed: 0; 	 Running data 20/84 (33) with seed=92.. 
 
Sanity check: 1.00
Successful: 20/20. Skipped 0. Failed: 0.
Verified: 0.25 - Violated: 0.00
 
---
pert_edge: 5.000, delta=0.001, model=models/Enzymes_34035, do_enumeration=1
Seed: 442
Reading network and data in: /code/experiments/models/Enzymes_34035
Successful: 0/20. Skipped 0. Failed: 0; 	 Running data 1/84 (13) with seed=421.. 
Successful: 1/20. Skipped 0. Failed: 0; 	 Running data 2/84 (46) with seed=7.. 
Successful: 2/20. Skipped 0. Failed: 0; 	 Running data 3/84 (80) with seed=24.. 
Successful: 3/20. Skipped 0. Failed: 0; 	 Running data 4/84 (19) with seed=470.. 
Successful: 4/20. Skipped 0. Failed: 0; 	 Running data 5/84 (22) with seed=869.. 
Successful: 5/20. Skipped 0. Failed: 0; 	 Running data 6/84 (47) with seed=864.. 
Successful: 6/20. Skipped 0. Failed: 0; 	 Running data 7/84 (24) with seed=918.. 
Successful: 7/20. Skipped 0. Failed: 0; 	 Running data 8/84 (65) with seed=351.. 
Successful: 8/20. Skipped 0. Failed: 0; 	 Running data 9/84 (30) with seed=658.. 
Successful: 9/20. Skipped 0. Failed: 0; 	 Running data 10/84 (68) with seed=490.. 
Successful: 10/20. Skipped 0. Failed: 0; 	 Running data 11/84 (60) with seed=98.. 
Successful: 11/20. Skipped 0. Failed: 0; 	 Running data 12/84 (63) with seed=268.. 
Successful: 12/20. Skipped 0. Failed: 0; 	 Running data 13/84 (54) with seed=299.. 
Successful: 13/20. Skipped 0. Failed: 0; 	 Running data 14/84 (25) with seed=68.. 
Successful: 14/20. Skipped 0. Failed: 0; 	 Running data 15/84 (2) with seed=726.. 
Successful: 15/20. Skipped 0. Failed: 0; 	 Running data 16/84 (72) with seed=532.. 
Successful: 16/20. Skipped 0. Failed: 0; 	 Running data 17/84 (9) with seed=687.. 
Successful: 17/20. Skipped 0. Failed: 0; 	 Running data 18/84 (42) with seed=825.. 
Successful: 18/20. Skipped 0. Failed: 0; 	 Running data 19/84 (49) with seed=214.. 
Successful: 19/20. Skipped 0. Failed: 0; 	 Running data 20/84 (3) with seed=759.. 
 
Sanity check: 1.00
Successful: 20/20. Skipped 0. Failed: 0.
Verified: 0.15 - Violated: 0.00
 
---
pert_edge: 6.000, delta=0.001, model=models/Enzymes_34035, do_enumeration=1
Seed: 708
Reading network and data in: /code/experiments/models/Enzymes_34035
Successful: 0/20. Skipped 0. Failed: 0; 	 Running data 1/84 (1) with seed=651.. 
Successful: 1/20. Skipped 0. Failed: 0; 	 Running data 2/84 (47) with seed=670.. 
Successful: 2/20. Skipped 0. Failed: 0; 	 Running data 3/84 (75) with seed=265.. 
Successful: 3/20. Skipped 0. Failed: 0; 	 Running data 4/84 (78) with seed=207.. 
Successful: 4/20. Skipped 0. Failed: 0; 	 Running data 5/84 (66) with seed=988.. 
Successful: 5/20. Skipped 0. Failed: 0; 	 Running data 6/84 (76) with seed=219.. 
Successful: 6/20. Skipped 0. Failed: 0; 	 Running data 7/84 (81) with seed=649.. 
Successful: 7/20. Skipped 0. Failed: 0; 	 Running data 8/84 (31) with seed=794.. 
Successful: 8/20. Skipped 0. Failed: 0; 	 Running data 9/84 (19) with seed=759.. 
Successful: 9/20. Skipped 0. Failed: 0; 	 Running data 10/84 (42) with seed=384.. 
Successful: 10/20. Skipped 0. Failed: 0; 	 Running data 11/84 (67) with seed=710.. 
Successful: 11/20. Skipped 0. Failed: 0; 	 Running data 12/84 (48) with seed=426.. 
Successful: 12/20. Skipped 0. Failed: 0; 	 Running data 13/84 (27) with seed=763.. 
Successful: 13/20. Skipped 0. Failed: 0; 	 Running data 14/84 (80) with seed=9.. 
Successful: 14/20. Skipped 0. Failed: 0; 	 Running data 15/84 (17) with seed=291.. 
Successful: 15/20. Skipped 0. Failed: 0; 	 Running data 16/84 (40) with seed=368.. 
Successful: 16/20. Skipped 0. Failed: 0; 	 Running data 17/84 (20) with seed=505.. 
Successful: 17/20. Skipped 0. Failed: 0; 	 Running data 18/84 (4) with seed=611.. 
Successful: 18/20. Skipped 0. Failed: 0; 	 Running data 19/84 (26) with seed=803.. 
Successful: 19/20. Skipped 0. Failed: 0; 	 Running data 20/84 (10) with seed=481.. 
 
Sanity check: 1.00
Successful: 20/20. Skipped 0. Failed: 0.
Verified: 0.05 - Violated: 0.00
 
---
pert_edge: 7.000, delta=0.001, model=models/Enzymes_34035, do_enumeration=1
Seed: 141
Reading network and data in: /code/experiments/models/Enzymes_34035
Successful: 0/20. Skipped 0. Failed: 0; 	 Running data 1/84 (75) with seed=493.. 
Successful: 1/20. Skipped 0. Failed: 0; 	 Running data 2/84 (78) with seed=695.. 
Successful: 2/20. Skipped 0. Failed: 0; 	 Running data 3/84 (11) with seed=619.. 
Successful: 3/20. Skipped 0. Failed: 0; 	 Running data 4/84 (79) with seed=474.. 
Successful: 4/20. Skipped 0. Failed: 0; 	 Running data 5/84 (3) with seed=112.. 
Successful: 5/20. Skipped 0. Failed: 0; 	 Running data 6/84 (21) with seed=223.. 
Successful: 6/20. Skipped 0. Failed: 0; 	 Running data 7/84 (7) with seed=364.. 
Successful: 7/20. Skipped 0. Failed: 0; 	 Running data 8/84 (28) with seed=70.. 
Successful: 8/20. Skipped 0. Failed: 0; 	 Running data 9/84 (84) with seed=107.. 
Successful: 9/20. Skipped 0. Failed: 0; 	 Running data 10/84 (23) with seed=888.. 
Successful: 10/20. Skipped 0. Failed: 0; 	 Running data 11/84 (76) with seed=668.. 
Successful: 11/20. Skipped 0. Failed: 0; 	 Running data 12/84 (53) with seed=632.. 
Successful: 12/20. Skipped 0. Failed: 0; 	 Running data 13/84 (35) with seed=572.. 
Successful: 13/20. Skipped 0. Failed: 0; 	 Running data 14/84 (82) with seed=801.. 
Successful: 14/20. Skipped 0. Failed: 0; 	 Running data 15/84 (6) with seed=83.. 
Successful: 15/20. Skipped 0. Failed: 0; 	 Running data 16/84 (10) with seed=919.. 
Successful: 16/20. Skipped 0. Failed: 0; 	 Running data 17/84 (55) with seed=184.. 
Successful: 17/20. Skipped 0. Failed: 0; 	 Running data 18/84 (27) with seed=900.. 
Successful: 18/20. Skipped 0. Failed: 0; 	 Running data 19/84 (34) with seed=539.. 
Successful: 19/20. Skipped 0. Failed: 0; 	 Running data 20/84 (14) with seed=413.. 
 
Sanity check: 1.00
Successful: 20/20. Skipped 0. Failed: 0.
Verified: 0.05 - Violated: 0.00
 
---
pert_edge: 8.000, delta=0.001, model=models/Enzymes_34035, do_enumeration=1
Seed: 847
Reading network and data in: /code/experiments/models/Enzymes_34035
Successful: 0/20. Skipped 0. Failed: 0; 	 Running data 1/84 (8) with seed=126.. 
Successful: 1/20. Skipped 0. Failed: 0; 	 Running data 2/84 (40) with seed=57.. 
Successful: 2/20. Skipped 0. Failed: 0; 	 Running data 3/84 (79) with seed=414.. 
Successful: 3/20. Skipped 0. Failed: 0; 	 Running data 4/84 (76) with seed=983.. 
Successful: 4/20. Skipped 0. Failed: 0; 	 Running data 5/84 (38) with seed=181.. 
Successful: 5/20. Skipped 0. Failed: 0; 	 Running data 6/84 (36) with seed=134.. 
Successful: 6/20. Skipped 0. Failed: 0; 	 Running data 7/84 (61) with seed=101.. 
Successful: 7/20. Skipped 0. Failed: 0; 	 Running data 8/84 (13) with seed=926.. 
Successful: 8/20. Skipped 0. Failed: 0; 	 Running data 9/84 (53) with seed=85.. 
Successful: 9/20. Skipped 0. Failed: 0; 	 Running data 10/84 (26) with seed=662.. 
Successful: 10/20. Skipped 0. Failed: 0; 	 Running data 11/84 (48) with seed=911.. 
Successful: 11/20. Skipped 0. Failed: 0; 	 Running data 12/84 (27) with seed=416.. 
Successful: 12/20. Skipped 0. Failed: 0; 	 Running data 13/84 (64) with seed=463.. 
Successful: 13/20. Skipped 0. Failed: 0; 	 Running data 14/84 (71) with seed=114.. 
Successful: 14/20. Skipped 0. Failed: 0; 	 Running data 15/84 (28) with seed=257.. 
Successful: 15/20. Skipped 0. Failed: 0; 	 Running data 16/84 (45) with seed=783.. 
Successful: 16/20. Skipped 0. Failed: 0; 	 Running data 17/84 (35) with seed=397.. 
Successful: 17/20. Skipped 0. Failed: 0; 	 Running data 18/84 (77) with seed=756.. 
Successful: 18/20. Skipped 0. Failed: 0; 	 Running data 19/84 (20) with seed=175.. 
Successful: 19/20. Skipped 0. Failed: 0; 	 Running data 20/84 (33) with seed=588.. 
 
Sanity check: 1.00
Successful: 20/20. Skipped 0. Failed: 0.
Verified: 0.05 - Violated: 0.00
 
---
pert_edge: 9.000, delta=0.001, model=models/Enzymes_34035, do_enumeration=1
Seed: 475
Reading network and data in: /code/experiments/models/Enzymes_34035
Successful: 0/20. Skipped 0. Failed: 0; 	 Running data 1/84 (53) with seed=28.. 
Successful: 1/20. Skipped 0. Failed: 0; 	 Running data 2/84 (60) with seed=930.. 
Successful: 2/20. Skipped 0. Failed: 0; 	 Running data 3/84 (1) with seed=176.. 
Successful: 3/20. Skipped 0. Failed: 0; 	 Running data 4/84 (51) with seed=521.. 
Successful: 4/20. Skipped 0. Failed: 0; 	 Running data 5/84 (15) with seed=392.. 
Successful: 5/20. Skipped 0. Failed: 0; 	 Running data 6/84 (79) with seed=996.. 
Successful: 6/20. Skipped 0. Failed: 0; 	 Running data 7/84 (40) with seed=64.. 
Successful: 7/20. Skipped 0. Failed: 0; 	 Running data 8/84 (11) with seed=93.. 
Successful: 8/20. Skipped 0. Failed: 0; 	 Running data 9/84 (36) with seed=406.. 
Successful: 9/20. Skipped 0. Failed: 0; 	 Running data 10/84 (66) with seed=14.. 
Successful: 10/20. Skipped 0. Failed: 0; 	 Running data 11/84 (10) with seed=793.. 
Successful: 11/20. Skipped 0. Failed: 0; 	 Running data 12/84 (69) with seed=438.. 
Successful: 12/20. Skipped 0. Failed: 0; 	 Running data 13/84 (7) with seed=827.. 
Successful: 13/20. Skipped 0. Failed: 0; 	 Running data 14/84 (25) with seed=110.. 
Successful: 14/20. Skipped 0. Failed: 0; 	 Running data 15/84 (31) with seed=352.. 
Successful: 15/20. Skipped 0. Failed: 0; 	 Running data 16/84 (21) with seed=554.. 
Successful: 16/20. Skipped 0. Failed: 0; 	 Running data 17/84 (16) with seed=883.. 
Successful: 17/20. Skipped 0. Failed: 0; 	 Running data 18/84 (48) with seed=595.. 
Successful: 18/20. Skipped 0. Failed: 0; 	 Running data 19/84 (32) with seed=988.. 
Successful: 19/20. Skipped 0. Failed: 0; 	 Running data 20/84 (37) with seed=230.. 
 
Sanity check: 1.00
Successful: 20/20. Skipped 0. Failed: 0.
Verified: 0.00 - Violated: 0.00
 
---
pert_edge: 0.000, delta=0.001, model=models/PROTEINS_314496, do_enumeration=1
Seed: 209
Reading network and data in: /code/experiments/models/PROTEINS_314496
Successful: 0/20. Skipped 0. Failed: 0; 	 Running data 1/171 (40) with seed=126.. 
Successful: 1/20. Skipped 0. Failed: 0; 	 Running data 2/171 (96) with seed=656.. 
Successful: 2/20. Skipped 0. Failed: 0; 	 Running data 3/171 (141) with seed=175.. 
Successful: 3/20. Skipped 0. Failed: 0; 	 Running data 4/171 (93) with seed=4.. 
Successful: 4/20. Skipped 0. Failed: 0; 	 Running data 5/171 (14) with seed=353.. 
Successful: 5/20. Skipped 0. Failed: 0; 	 Running data 6/171 (125) with seed=722.. 
Successful: 6/20. Skipped 0. Failed: 0; 	 Running data 7/171 (46) with seed=223.. 
Successful: 7/20. Skipped 0. Failed: 0; 	 Running data 8/171 (148) with seed=848.. 
Successful: 8/20. Skipped 0. Failed: 0; 	 Running data 9/171 (105) with seed=1000.. 
Successful: 9/20. Skipped 0. Failed: 0; 	 Running data 10/171 (24) with seed=886.. 
Successful: 10/20. Skipped 0. Failed: 0; 	 Running data 11/171 (136) with seed=83.. 
Successful: 11/20. Skipped 0. Failed: 0; 	 Running data 12/171 (8) with seed=404.. 
Successful: 12/20. Skipped 0. Failed: 0; 	 Running data 13/171 (17) with seed=266.. 
Successful: 13/20. Skipped 0. Failed: 0; 	 Running data 14/171 (98) with seed=763.. 
Successful: 14/20. Skipped 0. Failed: 0; 	 Running data 15/171 (34) with seed=458.. 
Successful: 15/20. Skipped 0. Failed: 0; 	 Running data 16/171 (27) with seed=743.. 
Successful: 16/20. Skipped 0. Failed: 0; 	 Running data 17/171 (171) with seed=541.. 
Successful: 17/20. Skipped 0. Failed: 0; 	 Running data 18/171 (94) with seed=487.. 
Successful: 18/20. Skipped 0. Failed: 0; 	 Running data 19/171 (153) with seed=135.. 
Successful: 19/20. Skipped 0. Failed: 0; 	 Running data 20/171 (38) with seed=561.. 
 
Sanity check: 1.00
Successful: 20/20. Skipped 0. Failed: 0.
Verified: 1.00 - Violated: 0.00
 
---
pert_edge: 1.000, delta=0.001, model=models/PROTEINS_314496, do_enumeration=1
Seed: 956
Reading network and data in: /code/experiments/models/PROTEINS_314496
Successful: 0/20. Skipped 0. Failed: 0; 	 Running data 1/171 (152) with seed=427.. 
Successful: 1/20. Skipped 0. Failed: 0; 	 Running data 2/171 (75) with seed=751.. 
Successful: 2/20. Skipped 0. Failed: 0; 	 Running data 3/171 (80) with seed=444.. 
Successful: 3/20. Skipped 0. Failed: 0; 	 Running data 4/171 (113) with seed=712.. 
Successful: 4/20. Skipped 0. Failed: 0; 	 Running data 5/171 (128) with seed=743.. 
Successful: 5/20. Skipped 0. Failed: 0; 	 Running data 6/171 (70) with seed=860.. 
Successful: 6/20. Skipped 0. Failed: 0; 	 Running data 7/171 (162) with seed=689.. 
Successful: 7/20. Skipped 0. Failed: 0; 	 Running data 8/171 (77) with seed=499.. 
Successful: 8/20. Skipped 0. Failed: 0; 	 Running data 9/171 (166) with seed=596.. 
Successful: 9/20. Skipped 0. Failed: 0; 	 Running data 10/171 (45) with seed=296.. 
Successful: 10/20. Skipped 0. Failed: 0; 	 Running data 11/171 (159) with seed=52.. 
Successful: 11/20. Skipped 0. Failed: 0; 	 Running data 12/171 (109) with seed=517.. 
Successful: 12/20. Skipped 0. Failed: 0; 	 Running data 13/171 (99) with seed=982.. 
Successful: 13/20. Skipped 0. Failed: 0; 	 Running data 14/171 (73) with seed=142.. 
Successful: 14/20. Skipped 0. Failed: 0; 	 Running data 15/171 (82) with seed=600.. 
Successful: 15/20. Skipped 0. Failed: 0; 	 Running data 16/171 (36) with seed=781.. 
Successful: 16/20. Skipped 0. Failed: 0; 	 Running data 17/171 (116) with seed=322.. 
Successful: 17/20. Skipped 0. Failed: 0; 	 Running data 18/171 (71) with seed=308.. 
Successful: 18/20. Skipped 0. Failed: 0; 	 Running data 19/171 (122) with seed=160.. 
Successful: 19/20. Skipped 0. Failed: 0; 	 Running data 20/171 (5) with seed=448.. 
 
Sanity check: 1.00
Successful: 20/20. Skipped 0. Failed: 0.
Verified: 1.00 - Violated: 0.00
 
---
pert_edge: 2.000, delta=0.001, model=models/PROTEINS_314496, do_enumeration=1
Seed: 933
Reading network and data in: /code/experiments/models/PROTEINS_314496
Successful: 0/20. Skipped 0. Failed: 0; 	 Running data 1/171 (156) with seed=596.. 
Successful: 1/20. Skipped 0. Failed: 0; 	 Running data 2/171 (13) with seed=746.. 
Successful: 2/20. Skipped 0. Failed: 0; 	 Running data 3/171 (83) with seed=786.. 
Successful: 3/20. Skipped 0. Failed: 0; 	 Running data 4/171 (107) with seed=973.. 
Successful: 4/20. Skipped 0. Failed: 0; 	 Running data 5/171 (140) with seed=998.. 
Successful: 5/20. Skipped 0. Failed: 0; 	 Running data 6/171 (61) with seed=850.. 
Successful: 6/20. Skipped 0. Failed: 0; 	 Running data 7/171 (82) with seed=97.. 
Successful: 7/20. Skipped 0. Failed: 0; 	 Running data 8/171 (58) with seed=110.. 
Successful: 8/20. Skipped 0. Failed: 0; 	 Running data 9/171 (147) with seed=500.. 
Successful: 9/20. Skipped 0. Failed: 0; 	 Running data 10/171 (119) with seed=929.. 
Successful: 10/20. Skipped 0. Failed: 0; 	 Running data 11/171 (126) with seed=813.. 
Successful: 11/20. Skipped 0. Failed: 0; 	 Running data 12/171 (143) with seed=372.. 
Successful: 12/20. Skipped 0. Failed: 0; 	 Running data 13/171 (121) with seed=849.. 
Successful: 13/20. Skipped 0. Failed: 0; 	 Running data 14/171 (101) with seed=363.. 
Successful: 14/20. Skipped 0. Failed: 0; 	 Running data 15/171 (46) with seed=730.. 
Successful: 15/20. Skipped 0. Failed: 0; 	 Running data 16/171 (47) with seed=127.. 
Successful: 16/20. Skipped 0. Failed: 0; 	 Running data 17/171 (135) with seed=98.. 
Successful: 17/20. Skipped 0. Failed: 0; 	 Running data 18/171 (122) with seed=13.. 
Successful: 18/20. Skipped 0. Failed: 0; 	 Running data 19/171 (108) with seed=246.. 
Successful: 19/20. Skipped 0. Failed: 0; 	 Running data 20/171 (109) with seed=992.. 
 
Sanity check: 1.00
Successful: 20/20. Skipped 0. Failed: 0.
Verified: 0.95 - Violated: 0.00
 
---
pert_edge: 3.000, delta=0.001, model=models/PROTEINS_314496, do_enumeration=1
Seed: 832
Reading network and data in: /code/experiments/models/PROTEINS_314496
Successful: 0/20. Skipped 0. Failed: 0; 	 Running data 1/171 (106) with seed=202.. 
Successful: 1/20. Skipped 0. Failed: 0; 	 Running data 2/171 (151) with seed=323.. 
Successful: 2/20. Skipped 0. Failed: 0; 	 Running data 3/171 (43) with seed=121.. 
Successful: 3/20. Skipped 0. Failed: 0; 	 Running data 4/171 (99) with seed=955.. 
Successful: 4/20. Skipped 0. Failed: 0; 	 Running data 5/171 (167) with seed=735.. 
Successful: 5/20. Skipped 0. Failed: 0; 	 Running data 6/171 (168) with seed=398.. 
Successful: 6/20. Skipped 0. Failed: 0; 	 Running data 7/171 (146) with seed=538.. 
Successful: 7/20. Skipped 0. Failed: 0; 	 Running data 8/171 (148) with seed=735.. 
Successful: 8/20. Skipped 0. Failed: 0; 	 Running data 9/171 (79) with seed=297.. 
Successful: 9/20. Skipped 0. Failed: 0; 	 Running data 10/171 (143) with seed=716.. 
Successful: 10/20. Skipped 0. Failed: 0; 	 Running data 11/171 (103) with seed=226.. 
Successful: 11/20. Skipped 0. Failed: 0; 	 Running data 12/171 (115) with seed=279.. 
Successful: 12/20. Skipped 0. Failed: 0; 	 Running data 13/171 (164) with seed=642.. 
Successful: 13/20. Skipped 0. Failed: 0; 	 Running data 14/171 (85) with seed=309.. 
Successful: 14/20. Skipped 0. Failed: 0; 	 Running data 15/171 (165) with seed=96.. 
Successful: 15/20. Skipped 0. Failed: 0; 	 Running data 16/171 (159) with seed=153.. 
Successful: 16/20. Skipped 0. Failed: 0; 	 Running data 17/171 (36) with seed=421.. 
Successful: 17/20. Skipped 0. Failed: 0; 	 Running data 18/171 (22) with seed=140.. 
Successful: 18/20. Skipped 0. Failed: 0; 	 Running data 19/171 (94) with seed=621.. 
Successful: 19/20. Skipped 0. Failed: 0; 	 Running data 20/171 (156) with seed=861.. 
 
Sanity check: 1.00
Successful: 20/20. Skipped 0. Failed: 0.
Verified: 0.80 - Violated: 0.00
 
---
pert_edge: 4.000, delta=0.001, model=models/PROTEINS_314496, do_enumeration=1
Seed: 112
Reading network and data in: /code/experiments/models/PROTEINS_314496
Successful: 0/20. Skipped 0. Failed: 0; 	 Running data 1/171 (132) with seed=655.. 
Successful: 1/20. Skipped 0. Failed: 0; 	 Running data 2/171 (11) with seed=431.. 
Successful: 2/20. Skipped 0. Failed: 0; 	 Running data 3/171 (16) with seed=61.. 
Successful: 3/20. Skipped 0. Failed: 0; 	 Running data 4/171 (91) with seed=747.. 
Successful: 4/20. Skipped 0. Failed: 0; 	 Running data 5/171 (154) with seed=906.. 
Successful: 5/20. Skipped 0. Failed: 0; 	 Running data 6/171 (80) with seed=582.. 
Successful: 6/20. Skipped 0. Failed: 0; 	 Running data 7/171 (78) with seed=257.. 
Successful: 7/20. Skipped 0. Failed: 0; 	 Running data 8/171 (68) with seed=407.. 
Successful: 8/20. Skipped 0. Failed: 0; 	 Running data 9/171 (84) with seed=557.. 
Successful: 9/20. Skipped 0. Failed: 0; 	 Running data 10/171 (136) with seed=203.. 
Successful: 10/20. Skipped 0. Failed: 0; 	 Running data 11/171 (27) with seed=700.. 
Successful: 11/20. Skipped 0. Failed: 0; 	 Running data 12/171 (145) with seed=605.. 
Successful: 12/20. Skipped 0. Failed: 0; 	 Running data 13/171 (85) with seed=742.. 
Successful: 13/20. Skipped 0. Failed: 0; 	 Running data 14/171 (121) with seed=317.. 
Successful: 14/20. Skipped 0. Failed: 0; 	 Running data 15/171 (7) with seed=913.. 
Successful: 15/20. Skipped 0. Failed: 0; 	 Running data 16/171 (146) with seed=642.. 
Successful: 15/20. Skipped 1. Failed: 0; 	 Running data 17/171 (89) with seed=959.. 
Successful: 16/20. Skipped 1. Failed: 0; 	 Running data 18/171 (56) with seed=83.. 
Successful: 17/20. Skipped 1. Failed: 0; 	 Running data 19/171 (81) with seed=957.. 
Successful: 18/20. Skipped 1. Failed: 0; 	 Running data 20/171 (41) with seed=897.. 
Successful: 19/20. Skipped 1. Failed: 0; 	 Running data 21/171 (4) with seed=667.. 
 
Sanity check: 1.00
Successful: 20/20. Skipped 1. Failed: 0.
Verified: 0.40 - Violated: 0.00
 
---
pert_edge: 5.000, delta=0.001, model=models/PROTEINS_314496, do_enumeration=1
Seed: 286
Reading network and data in: /code/experiments/models/PROTEINS_314496
Successful: 0/20. Skipped 0. Failed: 0; 	 Running data 1/171 (10) with seed=966.. 
Successful: 1/20. Skipped 0. Failed: 0; 	 Running data 2/171 (91) with seed=47.. 
Successful: 2/20. Skipped 0. Failed: 0; 	 Running data 3/171 (20) with seed=282.. 
Successful: 3/20. Skipped 0. Failed: 0; 	 Running data 4/171 (8) with seed=285.. 
Successful: 4/20. Skipped 0. Failed: 0; 	 Running data 5/171 (19) with seed=691.. 
Successful: 5/20. Skipped 0. Failed: 0; 	 Running data 6/171 (48) with seed=966.. 
Successful: 6/20. Skipped 0. Failed: 0; 	 Running data 7/171 (80) with seed=208.. 
Successful: 7/20. Skipped 0. Failed: 0; 	 Running data 8/171 (41) with seed=842.. 
Successful: 8/20. Skipped 0. Failed: 0; 	 Running data 9/171 (169) with seed=779.. 
Successful: 9/20. Skipped 0. Failed: 0; 	 Running data 10/171 (12) with seed=48.. 
Successful: 10/20. Skipped 0. Failed: 0; 	 Running data 11/171 (103) with seed=624.. 
Successful: 11/20. Skipped 0. Failed: 0; 	 Running data 12/171 (69) with seed=960.. 
Successful: 12/20. Skipped 0. Failed: 0; 	 Running data 13/171 (163) with seed=436.. 
Successful: 12/20. Skipped 1. Failed: 0; 	 Running data 14/171 (74) with seed=132.. 
Successful: 13/20. Skipped 1. Failed: 0; 	 Running data 15/171 (27) with seed=719.. 
Successful: 14/20. Skipped 1. Failed: 0; 	 Running data 16/171 (156) with seed=398.. 
Successful: 15/20. Skipped 1. Failed: 0; 	 Running data 17/171 (94) with seed=361.. 
Successful: 16/20. Skipped 1. Failed: 0; 	 Running data 18/171 (78) with seed=237.. 
Successful: 17/20. Skipped 1. Failed: 0; 	 Running data 19/171 (47) with seed=224.. 
Successful: 18/20. Skipped 1. Failed: 0; 	 Running data 20/171 (34) with seed=894.. 
Successful: 19/20. Skipped 1. Failed: 0; 	 Running data 21/171 (85) with seed=373.. 
 
Sanity check: 1.00
Successful: 20/20. Skipped 1. Failed: 0.
Verified: 0.50 - Violated: 0.00
 
---
pert_edge: 6.000, delta=0.001, model=models/PROTEINS_314496, do_enumeration=1
Seed: 134
Reading network and data in: /code/experiments/models/PROTEINS_314496
Successful: 0/20. Skipped 0. Failed: 0; 	 Running data 1/171 (74) with seed=226.. 
Successful: 1/20. Skipped 0. Failed: 0; 	 Running data 2/171 (96) with seed=206.. 
Successful: 2/20. Skipped 0. Failed: 0; 	 Running data 3/171 (105) with seed=168.. 
Successful: 3/20. Skipped 0. Failed: 0; 	 Running data 4/171 (77) with seed=644.. 
Successful: 4/20. Skipped 0. Failed: 0; 	 Running data 5/171 (128) with seed=922.. 
Successful: 5/20. Skipped 0. Failed: 0; 	 Running data 6/171 (10) with seed=411.. 
Successful: 6/20. Skipped 0. Failed: 0; 	 Running data 7/171 (72) with seed=219.. 
Successful: 7/20. Skipped 0. Failed: 0; 	 Running data 8/171 (33) with seed=876.. 
Successful: 8/20. Skipped 0. Failed: 0; 	 Running data 9/171 (125) with seed=374.. 
Successful: 9/20. Skipped 0. Failed: 0; 	 Running data 10/171 (85) with seed=499.. 
Successful: 10/20. Skipped 0. Failed: 0; 	 Running data 11/171 (41) with seed=59.. 
Successful: 11/20. Skipped 0. Failed: 0; 	 Running data 12/171 (152) with seed=701.. 
Successful: 12/20. Skipped 0. Failed: 0; 	 Running data 13/171 (93) with seed=621.. 
Successful: 13/20. Skipped 0. Failed: 0; 	 Running data 14/171 (133) with seed=358.. 
Successful: 14/20. Skipped 0. Failed: 0; 	 Running data 15/171 (39) with seed=701.. 
Successful: 15/20. Skipped 0. Failed: 0; 	 Running data 16/171 (47) with seed=535.. 
Successful: 16/20. Skipped 0. Failed: 0; 	 Running data 17/171 (9) with seed=945.. 
Successful: 17/20. Skipped 0. Failed: 0; 	 Running data 18/171 (65) with seed=986.. 
Successful: 18/20. Skipped 0. Failed: 0; 	 Running data 19/171 (64) with seed=48.. 
Successful: 19/20. Skipped 0. Failed: 0; 	 Running data 20/171 (63) with seed=151.. 
 
Sanity check: 1.00
Successful: 20/20. Skipped 0. Failed: 0.
Verified: 0.40 - Violated: 0.00
 
---
pert_edge: 7.000, delta=0.001, model=models/PROTEINS_314496, do_enumeration=1
Seed: 422
Reading network and data in: /code/experiments/models/PROTEINS_314496
Successful: 0/20. Skipped 0. Failed: 0; 	 Running data 1/171 (89) with seed=525.. 
Successful: 1/20. Skipped 0. Failed: 0; 	 Running data 2/171 (44) with seed=690.. 
Successful: 1/20. Skipped 1. Failed: 0; 	 Running data 3/171 (46) with seed=147.. 
Successful: 2/20. Skipped 1. Failed: 0; 	 Running data 4/171 (18) with seed=129.. 
Successful: 2/20. Skipped 1. Failed: 1; 	 Running data 5/171 (125) with seed=358.. 
Successful: 3/20. Skipped 1. Failed: 1; 	 Running data 6/171 (120) with seed=577.. 
Successful: 4/20. Skipped 1. Failed: 1; 	 Running data 7/171 (12) with seed=990.. 
Successful: 5/20. Skipped 1. Failed: 1; 	 Running data 8/171 (52) with seed=884.. 
Successful: 6/20. Skipped 1. Failed: 1; 	 Running data 9/171 (25) with seed=362.. 
Successful: 7/20. Skipped 1. Failed: 1; 	 Running data 10/171 (67) with seed=94.. 
Successful: 8/20. Skipped 1. Failed: 1; 	 Running data 11/171 (168) with seed=141.. 
Successful: 9/20. Skipped 1. Failed: 1; 	 Running data 12/171 (100) with seed=294.. 
Successful: 10/20. Skipped 1. Failed: 1; 	 Running data 13/171 (160) with seed=811.. 
Successful: 11/20. Skipped 1. Failed: 1; 	 Running data 14/171 (169) with seed=701.. 
Successful: 12/20. Skipped 1. Failed: 1; 	 Running data 15/171 (64) with seed=147.. 
Successful: 13/20. Skipped 1. Failed: 1; 	 Running data 16/171 (21) with seed=783.. 
Successful: 14/20. Skipped 1. Failed: 1; 	 Running data 17/171 (84) with seed=699.. 
Successful: 15/20. Skipped 1. Failed: 1; 	 Running data 18/171 (146) with seed=572.. 
Successful: 15/20. Skipped 2. Failed: 1; 	 Running data 19/171 (3) with seed=370.. 
Successful: 16/20. Skipped 2. Failed: 1; 	 Running data 20/171 (20) with seed=466.. 
Successful: 17/20. Skipped 2. Failed: 1; 	 Running data 21/171 (144) with seed=473.. 
Successful: 18/20. Skipped 2. Failed: 1; 	 Running data 22/171 (117) with seed=823.. 
Successful: 19/20. Skipped 2. Failed: 1; 	 Running data 23/171 (66) with seed=22.. 
 
Sanity check: 1.00
Successful: 20/20. Skipped 2. Failed: 1.
Verified: 0.60 - Violated: 0.00
 
---
pert_edge: 8.000, delta=0.001, model=models/PROTEINS_314496, do_enumeration=1
Seed: 388
Reading network and data in: /code/experiments/models/PROTEINS_314496
Successful: 0/20. Skipped 0. Failed: 0; 	 Running data 1/171 (98) with seed=510.. 
Successful: 1/20. Skipped 0. Failed: 0; 	 Running data 2/171 (119) with seed=340.. 
Successful: 2/20. Skipped 0. Failed: 0; 	 Running data 3/171 (83) with seed=163.. 
Successful: 3/20. Skipped 0. Failed: 0; 	 Running data 4/171 (145) with seed=782.. 
Successful: 4/20. Skipped 0. Failed: 0; 	 Running data 5/171 (22) with seed=75.. 
Successful: 5/20. Skipped 0. Failed: 0; 	 Running data 6/171 (152) with seed=662.. 
Successful: 6/20. Skipped 0. Failed: 0; 	 Running data 7/171 (42) with seed=289.. 
Successful: 7/20. Skipped 0. Failed: 0; 	 Running data 8/171 (64) with seed=532.. 
Successful: 8/20. Skipped 0. Failed: 0; 	 Running data 9/171 (32) with seed=58.. 
Successful: 9/20. Skipped 0. Failed: 0; 	 Running data 10/171 (30) with seed=693.. 
Successful: 9/20. Skipped 1. Failed: 0; 	 Running data 11/171 (84) with seed=657.. 
Successful: 10/20. Skipped 1. Failed: 0; 	 Running data 12/171 (59) with seed=911.. 
Successful: 11/20. Skipped 1. Failed: 0; 	 Running data 13/171 (12) with seed=263.. 
Successful: 12/20. Skipped 1. Failed: 0; 	 Running data 14/171 (8) with seed=321.. 
Successful: 13/20. Skipped 1. Failed: 0; 	 Running data 15/171 (150) with seed=20.. 
Successful: 14/20. Skipped 1. Failed: 0; 	 Running data 16/171 (55) with seed=989.. 
Successful: 15/20. Skipped 1. Failed: 0; 	 Running data 17/171 (155) with seed=776.. 
Successful: 16/20. Skipped 1. Failed: 0; 	 Running data 18/171 (38) with seed=847.. 
Successful: 17/20. Skipped 1. Failed: 0; 	 Running data 19/171 (86) with seed=383.. 
Successful: 18/20. Skipped 1. Failed: 0; 	 Running data 20/171 (78) with seed=549.. 
Successful: 19/20. Skipped 1. Failed: 0; 	 Running data 21/171 (72) with seed=806.. 
 
Sanity check: 1.00
Successful: 20/20. Skipped 1. Failed: 0.
Verified: 0.45 - Violated: 0.00
 
---
pert_edge: 9.000, delta=0.001, model=models/PROTEINS_314496, do_enumeration=1
Seed: 86
Reading network and data in: /code/experiments/models/PROTEINS_314496
Successful: 0/20. Skipped 0. Failed: 0; 	 Running data 1/171 (129) with seed=534.. 
Successful: 1/20. Skipped 0. Failed: 0; 	 Running data 2/171 (99) with seed=419.. 
Successful: 2/20. Skipped 0. Failed: 0; 	 Running data 3/171 (122) with seed=57.. 
Successful: 3/20. Skipped 0. Failed: 0; 	 Running data 4/171 (30) with seed=249.. 
Successful: 3/20. Skipped 1. Failed: 0; 	 Running data 5/171 (121) with seed=912.. 
Successful: 4/20. Skipped 1. Failed: 0; 	 Running data 6/171 (138) with seed=191.. 
Successful: 5/20. Skipped 1. Failed: 0; 	 Running data 7/171 (16) with seed=72.. 
Successful: 6/20. Skipped 1. Failed: 0; 	 Running data 8/171 (102) with seed=873.. 
Successful: 7/20. Skipped 1. Failed: 0; 	 Running data 9/171 (64) with seed=195.. 
Successful: 8/20. Skipped 1. Failed: 0; 	 Running data 10/171 (54) with seed=585.. 
Successful: 9/20. Skipped 1. Failed: 0; 	 Running data 11/171 (84) with seed=392.. 
Successful: 10/20. Skipped 1. Failed: 0; 	 Running data 12/171 (127) with seed=676.. 
Successful: 11/20. Skipped 1. Failed: 0; 	 Running data 13/171 (104) with seed=904.. 
Successful: 12/20. Skipped 1. Failed: 0; 	 Running data 14/171 (109) with seed=877.. 
Successful: 13/20. Skipped 1. Failed: 0; 	 Running data 15/171 (103) with seed=487.. 
Successful: 14/20. Skipped 1. Failed: 0; 	 Running data 16/171 (61) with seed=227.. 
Successful: 15/20. Skipped 1. Failed: 0; 	 Running data 17/171 (100) with seed=356.. 
Successful: 16/20. Skipped 1. Failed: 0; 	 Running data 18/171 (98) with seed=982.. 
Successful: 17/20. Skipped 1. Failed: 0; 	 Running data 19/171 (81) with seed=492.. 
Successful: 18/20. Skipped 1. Failed: 0; 	 Running data 20/171 (83) with seed=804.. 
Successful: 19/20. Skipped 1. Failed: 0; 	 Running data 21/171 (34) with seed=103.. 
Successful: 19/20. Skipped 2. Failed: 0; 	 Running data 22/171 (89) with seed=322.. 
 
Sanity check: 1.00
Successful: 20/20. Skipped 2. Failed: 0.
Verified: 0.30 - Violated: 0.00
 
---
pert_edge: 0.000, delta=0.001, model=models/Enzymes_34035, do_enumeration=0
Seed: 950
Reading network and data in: /code/experiments/models/Enzymes_34035
Successful: 0/50. Skipped 0. Failed: 0; 	 Running data 1/84 (10) with seed=745.. 
Successful: 1/50. Skipped 0. Failed: 0; 	 Running data 2/84 (7) with seed=421.. 
Successful: 2/50. Skipped 0. Failed: 0; 	 Running data 3/84 (47) with seed=583.. 
Successful: 3/50. Skipped 0. Failed: 0; 	 Running data 4/84 (31) with seed=933.. 
Successful: 4/50. Skipped 0. Failed: 0; 	 Running data 5/84 (3) with seed=97.. 
Successful: 5/50. Skipped 0. Failed: 0; 	 Running data 6/84 (20) with seed=152.. 
Successful: 6/50. Skipped 0. Failed: 0; 	 Running data 7/84 (60) with seed=114.. 
Successful: 7/50. Skipped 0. Failed: 0; 	 Running data 8/84 (61) with seed=584.. 
Successful: 8/50. Skipped 0. Failed: 0; 	 Running data 9/84 (2) with seed=254.. 
Successful: 9/50. Skipped 0. Failed: 0; 	 Running data 10/84 (78) with seed=144.. 
Successful: 10/50. Skipped 0. Failed: 0; 	 Running data 11/84 (38) with seed=751.. 
Successful: 11/50. Skipped 0. Failed: 0; 	 Running data 12/84 (77) with seed=493.. 
Successful: 12/50. Skipped 0. Failed: 0; 	 Running data 13/84 (43) with seed=535.. 
Successful: 13/50. Skipped 0. Failed: 0; 	 Running data 14/84 (64) with seed=774.. 
Successful: 14/50. Skipped 0. Failed: 0; 	 Running data 15/84 (45) with seed=350.. 
Successful: 15/50. Skipped 0. Failed: 0; 	 Running data 16/84 (19) with seed=358.. 
Successful: 16/50. Skipped 0. Failed: 0; 	 Running data 17/84 (12) with seed=444.. 
Successful: 17/50. Skipped 0. Failed: 0; 	 Running data 18/84 (9) with seed=342.. 
Successful: 18/50. Skipped 0. Failed: 0; 	 Running data 19/84 (18) with seed=381.. 
Successful: 19/50. Skipped 0. Failed: 0; 	 Running data 20/84 (63) with seed=180.. 
Successful: 20/50. Skipped 0. Failed: 0; 	 Running data 21/84 (21) with seed=576.. 
Successful: 21/50. Skipped 0. Failed: 0; 	 Running data 22/84 (62) with seed=29.. 
Successful: 22/50. Skipped 0. Failed: 0; 	 Running data 23/84 (22) with seed=702.. 
Successful: 23/50. Skipped 0. Failed: 0; 	 Running data 24/84 (53) with seed=56.. 
Successful: 24/50. Skipped 0. Failed: 0; 	 Running data 25/84 (37) with seed=649.. 
Successful: 25/50. Skipped 0. Failed: 0; 	 Running data 26/84 (66) with seed=46.. 
Successful: 26/50. Skipped 0. Failed: 0; 	 Running data 27/84 (73) with seed=956.. 
Successful: 27/50. Skipped 0. Failed: 0; 	 Running data 28/84 (5) with seed=337.. 
Successful: 28/50. Skipped 0. Failed: 0; 	 Running data 29/84 (40) with seed=302.. 
Successful: 29/50. Skipped 0. Failed: 0; 	 Running data 30/84 (14) with seed=750.. 
Successful: 30/50. Skipped 0. Failed: 0; 	 Running data 31/84 (33) with seed=775.. 
Successful: 31/50. Skipped 0. Failed: 0; 	 Running data 32/84 (70) with seed=516.. 
Successful: 32/50. Skipped 0. Failed: 0; 	 Running data 33/84 (74) with seed=315.. 
Successful: 33/50. Skipped 0. Failed: 0; 	 Running data 34/84 (16) with seed=976.. 
Successful: 34/50. Skipped 0. Failed: 0; 	 Running data 35/84 (6) with seed=195.. 
Successful: 35/50. Skipped 0. Failed: 0; 	 Running data 36/84 (58) with seed=431.. 
Successful: 36/50. Skipped 0. Failed: 0; 	 Running data 37/84 (68) with seed=268.. 
Successful: 37/50. Skipped 0. Failed: 0; 	 Running data 38/84 (82) with seed=29.. 
Successful: 38/50. Skipped 0. Failed: 0; 	 Running data 39/84 (81) with seed=403.. 
Successful: 39/50. Skipped 0. Failed: 0; 	 Running data 40/84 (15) with seed=80.. 
Successful: 40/50. Skipped 0. Failed: 0; 	 Running data 41/84 (80) with seed=486.. 
Successful: 41/50. Skipped 0. Failed: 0; 	 Running data 42/84 (39) with seed=658.. 
Successful: 42/50. Skipped 0. Failed: 0; 	 Running data 43/84 (25) with seed=926.. 
Successful: 43/50. Skipped 0. Failed: 0; 	 Running data 44/84 (51) with seed=456.. 
Successful: 44/50. Skipped 0. Failed: 0; 	 Running data 45/84 (57) with seed=715.. 
Successful: 45/50. Skipped 0. Failed: 0; 	 Running data 46/84 (28) with seed=108.. 
Successful: 46/50. Skipped 0. Failed: 0; 	 Running data 47/84 (24) with seed=609.. 
Successful: 47/50. Skipped 0. Failed: 0; 	 Running data 48/84 (65) with seed=642.. 
Successful: 48/50. Skipped 0. Failed: 0; 	 Running data 49/84 (83) with seed=237.. 
Successful: 49/50. Skipped 0. Failed: 0; 	 Running data 50/84 (50) with seed=36.. 
 
Sanity check: 1.00
Successful: 50/50. Skipped 0. Failed: 0.
Verified: 0.86 - Violated: 0.00
 
---
pert_edge: 0.001, delta=0.001, model=models/Enzymes_34035, do_enumeration=0
Seed: 729
Reading network and data in: /code/experiments/models/Enzymes_34035
Successful: 0/50. Skipped 0. Failed: 0; 	 Running data 1/84 (52) with seed=323.. 
Successful: 1/50. Skipped 0. Failed: 0; 	 Running data 2/84 (36) with seed=178.. 
Successful: 2/50. Skipped 0. Failed: 0; 	 Running data 3/84 (44) with seed=929.. 
Successful: 3/50. Skipped 0. Failed: 0; 	 Running data 4/84 (1) with seed=174.. 
Successful: 4/50. Skipped 0. Failed: 0; 	 Running data 5/84 (14) with seed=58.. 
Successful: 5/50. Skipped 0. Failed: 0; 	 Running data 6/84 (68) with seed=628.. 
Successful: 6/50. Skipped 0. Failed: 0; 	 Running data 7/84 (64) with seed=539.. 
Successful: 7/50. Skipped 0. Failed: 0; 	 Running data 8/84 (28) with seed=921.. 
Successful: 8/50. Skipped 0. Failed: 0; 	 Running data 9/84 (62) with seed=932.. 
Successful: 9/50. Skipped 0. Failed: 0; 	 Running data 10/84 (19) with seed=357.. 
Successful: 10/50. Skipped 0. Failed: 0; 	 Running data 11/84 (35) with seed=426.. 
Successful: 11/50. Skipped 0. Failed: 0; 	 Running data 12/84 (49) with seed=478.. 
Successful: 12/50. Skipped 0. Failed: 0; 	 Running data 13/84 (67) with seed=919.. 
Successful: 13/50. Skipped 0. Failed: 0; 	 Running data 14/84 (34) with seed=203.. 
Successful: 14/50. Skipped 0. Failed: 0; 	 Running data 15/84 (71) with seed=668.. 
Successful: 15/50. Skipped 0. Failed: 0; 	 Running data 16/84 (82) with seed=269.. 
Successful: 16/50. Skipped 0. Failed: 0; 	 Running data 17/84 (60) with seed=359.. 
Successful: 17/50. Skipped 0. Failed: 0; 	 Running data 18/84 (15) with seed=245.. 
Successful: 18/50. Skipped 0. Failed: 0; 	 Running data 19/84 (42) with seed=449.. 
Successful: 19/50. Skipped 0. Failed: 0; 	 Running data 20/84 (41) with seed=881.. 
Successful: 20/50. Skipped 0. Failed: 0; 	 Running data 21/84 (45) with seed=289.. 
Successful: 21/50. Skipped 0. Failed: 0; 	 Running data 22/84 (25) with seed=371.. 
Successful: 22/50. Skipped 0. Failed: 0; 	 Running data 23/84 (58) with seed=603.. 
Successful: 23/50. Skipped 0. Failed: 0; 	 Running data 24/84 (65) with seed=230.. 
Successful: 24/50. Skipped 0. Failed: 0; 	 Running data 25/84 (27) with seed=170.. 
Successful: 25/50. Skipped 0. Failed: 0; 	 Running data 26/84 (9) with seed=623.. 
Successful: 26/50. Skipped 0. Failed: 0; 	 Running data 27/84 (39) with seed=456.. 
Successful: 27/50. Skipped 0. Failed: 0; 	 Running data 28/84 (81) with seed=820.. 
Successful: 28/50. Skipped 0. Failed: 0; 	 Running data 29/84 (73) with seed=755.. 
Successful: 29/50. Skipped 0. Failed: 0; 	 Running data 30/84 (38) with seed=512.. 
Successful: 30/50. Skipped 0. Failed: 0; 	 Running data 31/84 (33) with seed=395.. 
Successful: 31/50. Skipped 0. Failed: 0; 	 Running data 32/84 (12) with seed=14.. 
Successful: 32/50. Skipped 0. Failed: 0; 	 Running data 33/84 (70) with seed=528.. 
Successful: 33/50. Skipped 0. Failed: 0; 	 Running data 34/84 (13) with seed=667.. 
Successful: 34/50. Skipped 0. Failed: 0; 	 Running data 35/84 (66) with seed=256.. 
Successful: 35/50. Skipped 0. Failed: 0; 	 Running data 36/84 (18) with seed=235.. 
Successful: 36/50. Skipped 0. Failed: 0; 	 Running data 37/84 (26) with seed=266.. 
Successful: 37/50. Skipped 0. Failed: 0; 	 Running data 38/84 (37) with seed=636.. 
Successful: 38/50. Skipped 0. Failed: 0; 	 Running data 39/84 (61) with seed=916.. 
Successful: 39/50. Skipped 0. Failed: 0; 	 Running data 40/84 (57) with seed=398.. 
Successful: 40/50. Skipped 0. Failed: 0; 	 Running data 41/84 (76) with seed=929.. 
Successful: 41/50. Skipped 0. Failed: 0; 	 Running data 42/84 (22) with seed=366.. 
Successful: 42/50. Skipped 0. Failed: 0; 	 Running data 43/84 (48) with seed=600.. 
Successful: 43/50. Skipped 0. Failed: 0; 	 Running data 44/84 (54) with seed=531.. 
Successful: 44/50. Skipped 0. Failed: 0; 	 Running data 45/84 (43) with seed=135.. 
Successful: 45/50. Skipped 0. Failed: 0; 	 Running data 46/84 (74) with seed=674.. 
Successful: 46/50. Skipped 0. Failed: 0; 	 Running data 47/84 (83) with seed=951.. 
Successful: 47/50. Skipped 0. Failed: 0; 	 Running data 48/84 (40) with seed=783.. 
Successful: 48/50. Skipped 0. Failed: 0; 	 Running data 49/84 (77) with seed=108.. 
Successful: 49/50. Skipped 0. Failed: 0; 	 Running data 50/84 (20) with seed=895.. 
 
Sanity check: 1.00
Successful: 50/50. Skipped 0. Failed: 0.
Verified: 0.80 - Violated: 0.00
 
---
pert_edge: 0.005, delta=0.001, model=models/Enzymes_34035, do_enumeration=0
Seed: 186
Reading network and data in: /code/experiments/models/Enzymes_34035
Successful: 0/50. Skipped 0. Failed: 0; 	 Running data 1/84 (36) with seed=7.. 
Successful: 1/50. Skipped 0. Failed: 0; 	 Running data 2/84 (77) with seed=389.. 
Successful: 2/50. Skipped 0. Failed: 0; 	 Running data 3/84 (76) with seed=55.. 
Successful: 3/50. Skipped 0. Failed: 0; 	 Running data 4/84 (62) with seed=638.. 
Successful: 4/50. Skipped 0. Failed: 0; 	 Running data 5/84 (65) with seed=205.. 
Successful: 5/50. Skipped 0. Failed: 0; 	 Running data 6/84 (75) with seed=527.. 
Successful: 6/50. Skipped 0. Failed: 0; 	 Running data 7/84 (13) with seed=735.. 
Successful: 7/50. Skipped 0. Failed: 0; 	 Running data 8/84 (41) with seed=782.. 
Successful: 8/50. Skipped 0. Failed: 0; 	 Running data 9/84 (21) with seed=368.. 
Successful: 9/50. Skipped 0. Failed: 0; 	 Running data 10/84 (82) with seed=428.. 
Successful: 10/50. Skipped 0. Failed: 0; 	 Running data 11/84 (43) with seed=67.. 
Successful: 11/50. Skipped 0. Failed: 0; 	 Running data 12/84 (29) with seed=372.. 
Successful: 12/50. Skipped 0. Failed: 0; 	 Running data 13/84 (78) with seed=969.. 
Successful: 13/50. Skipped 0. Failed: 0; 	 Running data 14/84 (33) with seed=287.. 
Successful: 14/50. Skipped 0. Failed: 0; 	 Running data 15/84 (56) with seed=91.. 
Successful: 15/50. Skipped 0. Failed: 0; 	 Running data 16/84 (7) with seed=343.. 
Successful: 16/50. Skipped 0. Failed: 0; 	 Running data 17/84 (68) with seed=564.. 
Successful: 17/50. Skipped 0. Failed: 0; 	 Running data 18/84 (35) with seed=267.. 
Successful: 18/50. Skipped 0. Failed: 0; 	 Running data 19/84 (73) with seed=436.. 
Successful: 19/50. Skipped 0. Failed: 0; 	 Running data 20/84 (8) with seed=575.. 
Successful: 20/50. Skipped 0. Failed: 0; 	 Running data 21/84 (19) with seed=885.. 
Successful: 21/50. Skipped 0. Failed: 0; 	 Running data 22/84 (17) with seed=675.. 
Successful: 22/50. Skipped 0. Failed: 0; 	 Running data 23/84 (38) with seed=568.. 
Successful: 23/50. Skipped 0. Failed: 0; 	 Running data 24/84 (16) with seed=517.. 
Successful: 24/50. Skipped 0. Failed: 0; 	 Running data 25/84 (84) with seed=236.. 
Successful: 25/50. Skipped 0. Failed: 0; 	 Running data 26/84 (74) with seed=74.. 
Successful: 26/50. Skipped 0. Failed: 0; 	 Running data 27/84 (81) with seed=828.. 
Successful: 27/50. Skipped 0. Failed: 0; 	 Running data 28/84 (18) with seed=203.. 
Successful: 28/50. Skipped 0. Failed: 0; 	 Running data 29/84 (79) with seed=352.. 
Successful: 29/50. Skipped 0. Failed: 0; 	 Running data 30/84 (52) with seed=754.. 
Successful: 30/50. Skipped 0. Failed: 0; 	 Running data 31/84 (46) with seed=115.. 
Successful: 31/50. Skipped 0. Failed: 0; 	 Running data 32/84 (44) with seed=107.. 
Successful: 32/50. Skipped 0. Failed: 0; 	 Running data 33/84 (58) with seed=27.. 
Successful: 33/50. Skipped 0. Failed: 0; 	 Running data 34/84 (27) with seed=227.. 
Successful: 34/50. Skipped 0. Failed: 0; 	 Running data 35/84 (49) with seed=896.. 
Successful: 35/50. Skipped 0. Failed: 0; 	 Running data 36/84 (32) with seed=690.. 
Successful: 36/50. Skipped 0. Failed: 0; 	 Running data 37/84 (66) with seed=956.. 
Successful: 37/50. Skipped 0. Failed: 0; 	 Running data 38/84 (26) with seed=527.. 
Successful: 38/50. Skipped 0. Failed: 0; 	 Running data 39/84 (24) with seed=355.. 
Successful: 39/50. Skipped 0. Failed: 0; 	 Running data 40/84 (3) with seed=906.. 
Successful: 40/50. Skipped 0. Failed: 0; 	 Running data 41/84 (1) with seed=875.. 
Successful: 41/50. Skipped 0. Failed: 0; 	 Running data 42/84 (50) with seed=66.. 
Successful: 42/50. Skipped 0. Failed: 0; 	 Running data 43/84 (80) with seed=333.. 
Successful: 43/50. Skipped 0. Failed: 0; 	 Running data 44/84 (83) with seed=98.. 
Successful: 44/50. Skipped 0. Failed: 0; 	 Running data 45/84 (34) with seed=239.. 
Successful: 45/50. Skipped 0. Failed: 0; 	 Running data 46/84 (37) with seed=392.. 
Successful: 46/50. Skipped 0. Failed: 0; 	 Running data 47/84 (40) with seed=499.. 
Successful: 47/50. Skipped 0. Failed: 0; 	 Running data 48/84 (45) with seed=465.. 
Successful: 48/50. Skipped 0. Failed: 0; 	 Running data 49/84 (30) with seed=529.. 
Successful: 49/50. Skipped 0. Failed: 0; 	 Running data 50/84 (70) with seed=370.. 
 
Sanity check: 1.00
Successful: 50/50. Skipped 0. Failed: 0.
Verified: 0.72 - Violated: 0.00
 
---
pert_edge: 0.010, delta=0.001, model=models/Enzymes_34035, do_enumeration=0
Seed: 868
Reading network and data in: /code/experiments/models/Enzymes_34035
Successful: 0/50. Skipped 0. Failed: 0; 	 Running data 1/84 (73) with seed=166.. 
Successful: 1/50. Skipped 0. Failed: 0; 	 Running data 2/84 (3) with seed=156.. 
Successful: 2/50. Skipped 0. Failed: 0; 	 Running data 3/84 (24) with seed=15.. 
Successful: 3/50. Skipped 0. Failed: 0; 	 Running data 4/84 (44) with seed=210.. 
Successful: 4/50. Skipped 0. Failed: 0; 	 Running data 5/84 (47) with seed=174.. 
Successful: 5/50. Skipped 0. Failed: 0; 	 Running data 6/84 (82) with seed=89.. 
Successful: 6/50. Skipped 0. Failed: 0; 	 Running data 7/84 (35) with seed=378.. 
Successful: 7/50. Skipped 0. Failed: 0; 	 Running data 8/84 (40) with seed=839.. 
Successful: 8/50. Skipped 0. Failed: 0; 	 Running data 9/84 (22) with seed=681.. 
Successful: 9/50. Skipped 0. Failed: 0; 	 Running data 10/84 (17) with seed=685.. 
Successful: 10/50. Skipped 0. Failed: 0; 	 Running data 11/84 (29) with seed=3.. 
Successful: 11/50. Skipped 0. Failed: 0; 	 Running data 12/84 (53) with seed=87.. 
Successful: 12/50. Skipped 0. Failed: 0; 	 Running data 13/84 (76) with seed=721.. 
Successful: 13/50. Skipped 0. Failed: 0; 	 Running data 14/84 (65) with seed=478.. 
Successful: 14/50. Skipped 0. Failed: 0; 	 Running data 15/84 (33) with seed=800.. 
Successful: 15/50. Skipped 0. Failed: 0; 	 Running data 16/84 (64) with seed=968.. 
Successful: 16/50. Skipped 0. Failed: 0; 	 Running data 17/84 (8) with seed=5.. 
Successful: 17/50. Skipped 0. Failed: 0; 	 Running data 18/84 (83) with seed=90.. 
Successful: 18/50. Skipped 0. Failed: 0; 	 Running data 19/84 (1) with seed=205.. 
Successful: 19/50. Skipped 0. Failed: 0; 	 Running data 20/84 (36) with seed=378.. 
Successful: 20/50. Skipped 0. Failed: 0; 	 Running data 21/84 (63) with seed=420.. 
Successful: 21/50. Skipped 0. Failed: 0; 	 Running data 22/84 (9) with seed=548.. 
Successful: 22/50. Skipped 0. Failed: 0; 	 Running data 23/84 (74) with seed=772.. 
Successful: 23/50. Skipped 0. Failed: 0; 	 Running data 24/84 (69) with seed=829.. 
Successful: 24/50. Skipped 0. Failed: 0; 	 Running data 25/84 (13) with seed=87.. 
Successful: 25/50. Skipped 0. Failed: 0; 	 Running data 26/84 (10) with seed=108.. 
Successful: 26/50. Skipped 0. Failed: 0; 	 Running data 27/84 (11) with seed=337.. 
Successful: 27/50. Skipped 0. Failed: 0; 	 Running data 28/84 (6) with seed=672.. 
Successful: 28/50. Skipped 0. Failed: 0; 	 Running data 29/84 (45) with seed=52.. 
Successful: 29/50. Skipped 0. Failed: 0; 	 Running data 30/84 (30) with seed=159.. 
Successful: 30/50. Skipped 0. Failed: 0; 	 Running data 31/84 (2) with seed=908.. 
Successful: 31/50. Skipped 0. Failed: 0; 	 Running data 32/84 (43) with seed=544.. 
Successful: 32/50. Skipped 0. Failed: 0; 	 Running data 33/84 (12) with seed=78.. 
Successful: 33/50. Skipped 0. Failed: 0; 	 Running data 34/84 (78) with seed=458.. 
Successful: 34/50. Skipped 0. Failed: 0; 	 Running data 35/84 (54) with seed=923.. 
Successful: 35/50. Skipped 0. Failed: 0; 	 Running data 36/84 (61) with seed=149.. 
Successful: 36/50. Skipped 0. Failed: 0; 	 Running data 37/84 (42) with seed=682.. 
Successful: 37/50. Skipped 0. Failed: 0; 	 Running data 38/84 (79) with seed=423.. 
Successful: 38/50. Skipped 0. Failed: 0; 	 Running data 39/84 (62) with seed=349.. 
Successful: 39/50. Skipped 0. Failed: 0; 	 Running data 40/84 (60) with seed=440.. 
Successful: 40/50. Skipped 0. Failed: 0; 	 Running data 41/84 (37) with seed=523.. 
Successful: 41/50. Skipped 0. Failed: 0; 	 Running data 42/84 (7) with seed=364.. 
Successful: 42/50. Skipped 0. Failed: 0; 	 Running data 43/84 (55) with seed=986.. 
Successful: 43/50. Skipped 0. Failed: 0; 	 Running data 44/84 (58) with seed=34.. 
Successful: 44/50. Skipped 0. Failed: 0; 	 Running data 45/84 (39) with seed=829.. 
Successful: 45/50. Skipped 0. Failed: 0; 	 Running data 46/84 (77) with seed=191.. 
Successful: 46/50. Skipped 0. Failed: 0; 	 Running data 47/84 (31) with seed=768.. 
Successful: 47/50. Skipped 0. Failed: 0; 	 Running data 48/84 (25) with seed=294.. 
Successful: 48/50. Skipped 0. Failed: 0; 	 Running data 49/84 (84) with seed=145.. 
Successful: 49/50. Skipped 0. Failed: 0; 	 Running data 50/84 (51) with seed=511.. 
 
Sanity check: 1.00
Successful: 50/50. Skipped 0. Failed: 0.
Verified: 0.60 - Violated: 0.00
 
---
pert_edge: 0.050, delta=0.001, model=models/Enzymes_34035, do_enumeration=0
Seed: 239
Reading network and data in: /code/experiments/models/Enzymes_34035
Successful: 0/50. Skipped 0. Failed: 0; 	 Running data 1/84 (15) with seed=923.. 
Successful: 1/50. Skipped 0. Failed: 0; 	 Running data 2/84 (78) with seed=39.. 
Successful: 2/50. Skipped 0. Failed: 0; 	 Running data 3/84 (82) with seed=395.. 
Successful: 3/50. Skipped 0. Failed: 0; 	 Running data 4/84 (67) with seed=835.. 
Successful: 4/50. Skipped 0. Failed: 0; 	 Running data 5/84 (54) with seed=93.. 
Successful: 5/50. Skipped 0. Failed: 0; 	 Running data 6/84 (44) with seed=603.. 
Successful: 6/50. Skipped 0. Failed: 0; 	 Running data 7/84 (60) with seed=821.. 
Successful: 7/50. Skipped 0. Failed: 0; 	 Running data 8/84 (2) with seed=823.. 
Successful: 8/50. Skipped 0. Failed: 0; 	 Running data 9/84 (9) with seed=393.. 
Successful: 9/50. Skipped 0. Failed: 0; 	 Running data 10/84 (28) with seed=853.. 
Successful: 10/50. Skipped 0. Failed: 0; 	 Running data 11/84 (52) with seed=808.. 
Successful: 11/50. Skipped 0. Failed: 0; 	 Running data 12/84 (40) with seed=834.. 
Successful: 12/50. Skipped 0. Failed: 0; 	 Running data 13/84 (80) with seed=282.. 
Successful: 13/50. Skipped 0. Failed: 0; 	 Running data 14/84 (16) with seed=462.. 
Successful: 14/50. Skipped 0. Failed: 0; 	 Running data 15/84 (24) with seed=184.. 
Successful: 15/50. Skipped 0. Failed: 0; 	 Running data 16/84 (1) with seed=500.. 
Successful: 16/50. Skipped 0. Failed: 0; 	 Running data 17/84 (34) with seed=740.. 
Successful: 17/50. Skipped 0. Failed: 0; 	 Running data 18/84 (59) with seed=639.. 
Successful: 18/50. Skipped 0. Failed: 0; 	 Running data 19/84 (73) with seed=741.. 
Successful: 19/50. Skipped 0. Failed: 0; 	 Running data 20/84 (3) with seed=209.. 
Successful: 20/50. Skipped 0. Failed: 0; 	 Running data 21/84 (63) with seed=602.. 
Successful: 21/50. Skipped 0. Failed: 0; 	 Running data 22/84 (56) with seed=224.. 
Successful: 22/50. Skipped 0. Failed: 0; 	 Running data 23/84 (12) with seed=752.. 
Successful: 23/50. Skipped 0. Failed: 0; 	 Running data 24/84 (36) with seed=816.. 
Successful: 24/50. Skipped 0. Failed: 0; 	 Running data 25/84 (79) with seed=732.. 
Successful: 25/50. Skipped 0. Failed: 0; 	 Running data 26/84 (17) with seed=639.. 
Successful: 26/50. Skipped 0. Failed: 0; 	 Running data 27/84 (33) with seed=825.. 
Successful: 27/50. Skipped 0. Failed: 0; 	 Running data 28/84 (8) with seed=821.. 
Successful: 28/50. Skipped 0. Failed: 0; 	 Running data 29/84 (57) with seed=273.. 
Successful: 29/50. Skipped 0. Failed: 0; 	 Running data 30/84 (13) with seed=387.. 
Successful: 30/50. Skipped 0. Failed: 0; 	 Running data 31/84 (46) with seed=69.. 
Successful: 31/50. Skipped 0. Failed: 0; 	 Running data 32/84 (71) with seed=706.. 
Successful: 32/50. Skipped 0. Failed: 0; 	 Running data 33/84 (74) with seed=644.. 
Successful: 33/50. Skipped 0. Failed: 0; 	 Running data 34/84 (6) with seed=949.. 
Successful: 34/50. Skipped 0. Failed: 0; 	 Running data 35/84 (42) with seed=779.. 
Successful: 35/50. Skipped 0. Failed: 0; 	 Running data 36/84 (7) with seed=567.. 
Successful: 36/50. Skipped 0. Failed: 0; 	 Running data 37/84 (49) with seed=135.. 
Successful: 37/50. Skipped 0. Failed: 0; 	 Running data 38/84 (55) with seed=261.. 
Successful: 38/50. Skipped 0. Failed: 0; 	 Running data 39/84 (23) with seed=810.. 
Successful: 39/50. Skipped 0. Failed: 0; 	 Running data 40/84 (61) with seed=935.. 
Successful: 40/50. Skipped 0. Failed: 0; 	 Running data 41/84 (69) with seed=536.. 
Successful: 41/50. Skipped 0. Failed: 0; 	 Running data 42/84 (64) with seed=202.. 
Successful: 42/50. Skipped 0. Failed: 0; 	 Running data 43/84 (30) with seed=961.. 
Successful: 43/50. Skipped 0. Failed: 0; 	 Running data 44/84 (53) with seed=443.. 
Successful: 44/50. Skipped 0. Failed: 0; 	 Running data 45/84 (10) with seed=615.. 
Successful: 45/50. Skipped 0. Failed: 0; 	 Running data 46/84 (26) with seed=418.. 
Successful: 46/50. Skipped 0. Failed: 0; 	 Running data 47/84 (5) with seed=837.. 
Successful: 47/50. Skipped 0. Failed: 0; 	 Running data 48/84 (65) with seed=712.. 
Successful: 48/50. Skipped 0. Failed: 0; 	 Running data 49/84 (48) with seed=335.. 
Successful: 49/50. Skipped 0. Failed: 0; 	 Running data 50/84 (27) with seed=213.. 
 
Sanity check: 1.00
Successful: 50/50. Skipped 0. Failed: 0.
Verified: 0.22 - Violated: 0.00
 
---
pert_edge: 0.000, delta=0.001, model=models/PROTEINS_314496, do_enumeration=0
Seed: 842
Reading network and data in: /code/experiments/models/PROTEINS_314496
Successful: 0/50. Skipped 0. Failed: 0; 	 Running data 1/171 (59) with seed=918.. 
Successful: 1/50. Skipped 0. Failed: 0; 	 Running data 2/171 (125) with seed=601.. 
Successful: 2/50. Skipped 0. Failed: 0; 	 Running data 3/171 (128) with seed=811.. 
Successful: 3/50. Skipped 0. Failed: 0; 	 Running data 4/171 (136) with seed=402.. 
Successful: 4/50. Skipped 0. Failed: 0; 	 Running data 5/171 (63) with seed=860.. 
Successful: 5/50. Skipped 0. Failed: 0; 	 Running data 6/171 (168) with seed=86.. 
Successful: 6/50. Skipped 0. Failed: 0; 	 Running data 7/171 (122) with seed=224.. 
Successful: 7/50. Skipped 0. Failed: 0; 	 Running data 8/171 (119) with seed=525.. 
Successful: 8/50. Skipped 0. Failed: 0; 	 Running data 9/171 (152) with seed=714.. 
Successful: 9/50. Skipped 0. Failed: 0; 	 Running data 10/171 (52) with seed=399.. 
Successful: 10/50. Skipped 0. Failed: 0; 	 Running data 11/171 (67) with seed=345.. 
Successful: 11/50. Skipped 0. Failed: 0; 	 Running data 12/171 (157) with seed=462.. 
Successful: 12/50. Skipped 0. Failed: 0; 	 Running data 13/171 (147) with seed=309.. 
Successful: 13/50. Skipped 0. Failed: 0; 	 Running data 14/171 (76) with seed=385.. 
Successful: 14/50. Skipped 0. Failed: 0; 	 Running data 15/171 (78) with seed=848.. 
Successful: 15/50. Skipped 0. Failed: 0; 	 Running data 16/171 (137) with seed=428.. 
Successful: 16/50. Skipped 0. Failed: 0; 	 Running data 17/171 (70) with seed=950.. 
Successful: 17/50. Skipped 0. Failed: 0; 	 Running data 18/171 (73) with seed=913.. 
Successful: 18/50. Skipped 0. Failed: 0; 	 Running data 19/171 (79) with seed=758.. 
Successful: 19/50. Skipped 0. Failed: 0; 	 Running data 20/171 (21) with seed=945.. 
Successful: 20/50. Skipped 0. Failed: 0; 	 Running data 21/171 (144) with seed=808.. 
Successful: 21/50. Skipped 0. Failed: 0; 	 Running data 22/171 (30) with seed=580.. 
Successful: 22/50. Skipped 0. Failed: 0; 	 Running data 23/171 (35) with seed=714.. 
Successful: 23/50. Skipped 0. Failed: 0; 	 Running data 24/171 (61) with seed=281.. 
Successful: 24/50. Skipped 0. Failed: 0; 	 Running data 25/171 (9) with seed=411.. 
Successful: 25/50. Skipped 0. Failed: 0; 	 Running data 26/171 (124) with seed=146.. 
Successful: 26/50. Skipped 0. Failed: 0; 	 Running data 27/171 (159) with seed=8.. 
Successful: 27/50. Skipped 0. Failed: 0; 	 Running data 28/171 (69) with seed=209.. 
Successful: 28/50. Skipped 0. Failed: 0; 	 Running data 29/171 (41) with seed=250.. 
Successful: 29/50. Skipped 0. Failed: 0; 	 Running data 30/171 (44) with seed=407.. 
Successful: 30/50. Skipped 0. Failed: 0; 	 Running data 31/171 (121) with seed=59.. 
Successful: 31/50. Skipped 0. Failed: 0; 	 Running data 32/171 (126) with seed=326.. 
Successful: 32/50. Skipped 0. Failed: 0; 	 Running data 33/171 (82) with seed=940.. 
Successful: 33/50. Skipped 0. Failed: 0; 	 Running data 34/171 (98) with seed=245.. 
Successful: 34/50. Skipped 0. Failed: 0; 	 Running data 35/171 (97) with seed=669.. 
Successful: 35/50. Skipped 0. Failed: 0; 	 Running data 36/171 (120) with seed=895.. 
Successful: 36/50. Skipped 0. Failed: 0; 	 Running data 37/171 (57) with seed=629.. 
Successful: 37/50. Skipped 0. Failed: 0; 	 Running data 38/171 (26) with seed=718.. 
Successful: 38/50. Skipped 0. Failed: 0; 	 Running data 39/171 (13) with seed=752.. 
Successful: 39/50. Skipped 0. Failed: 0; 	 Running data 40/171 (110) with seed=407.. 
Successful: 40/50. Skipped 0. Failed: 0; 	 Running data 41/171 (18) with seed=503.. 
Successful: 40/50. Skipped 0. Failed: 1; 	 Running data 42/171 (55) with seed=511.. 
Successful: 41/50. Skipped 0. Failed: 1; 	 Running data 43/171 (22) with seed=20.. 
Successful: 42/50. Skipped 0. Failed: 1; 	 Running data 44/171 (31) with seed=106.. 
Successful: 43/50. Skipped 0. Failed: 1; 	 Running data 45/171 (16) with seed=291.. 
Successful: 44/50. Skipped 0. Failed: 1; 	 Running data 46/171 (49) with seed=390.. 
Successful: 45/50. Skipped 0. Failed: 1; 	 Running data 47/171 (170) with seed=876.. 
Successful: 46/50. Skipped 0. Failed: 1; 	 Running data 48/171 (54) with seed=992.. 
Successful: 47/50. Skipped 0. Failed: 1; 	 Running data 49/171 (43) with seed=865.. 
Successful: 48/50. Skipped 0. Failed: 1; 	 Running data 50/171 (167) with seed=701.. 
Successful: 49/50. Skipped 0. Failed: 1; 	 Running data 51/171 (106) with seed=672.. 
 
Sanity check: 1.00
Successful: 50/50. Skipped 0. Failed: 1.
Verified: 1.00 - Violated: 0.00
 
---
pert_edge: 0.001, delta=0.001, model=models/PROTEINS_314496, do_enumeration=0
Seed: 830
Reading network and data in: /code/experiments/models/PROTEINS_314496
Successful: 0/50. Skipped 0. Failed: 0; 	 Running data 1/171 (163) with seed=378.. 
Successful: 1/50. Skipped 0. Failed: 0; 	 Running data 2/171 (91) with seed=619.. 
Successful: 2/50. Skipped 0. Failed: 0; 	 Running data 3/171 (145) with seed=417.. 
Successful: 3/50. Skipped 0. Failed: 0; 	 Running data 4/171 (117) with seed=364.. 
Successful: 4/50. Skipped 0. Failed: 0; 	 Running data 5/171 (135) with seed=817.. 
Successful: 5/50. Skipped 0. Failed: 0; 	 Running data 6/171 (104) with seed=306.. 
Successful: 6/50. Skipped 0. Failed: 0; 	 Running data 7/171 (138) with seed=883.. 
Successful: 7/50. Skipped 0. Failed: 0; 	 Running data 8/171 (10) with seed=745.. 
Successful: 8/50. Skipped 0. Failed: 0; 	 Running data 9/171 (154) with seed=36.. 
Successful: 9/50. Skipped 0. Failed: 0; 	 Running data 10/171 (166) with seed=201.. 
Successful: 10/50. Skipped 0. Failed: 0; 	 Running data 11/171 (112) with seed=958.. 
Successful: 11/50. Skipped 0. Failed: 0; 	 Running data 12/171 (141) with seed=168.. 
Successful: 12/50. Skipped 0. Failed: 0; 	 Running data 13/171 (99) with seed=953.. 
Successful: 13/50. Skipped 0. Failed: 0; 	 Running data 14/171 (39) with seed=119.. 
Successful: 14/50. Skipped 0. Failed: 0; 	 Running data 15/171 (40) with seed=311.. 
Successful: 15/50. Skipped 0. Failed: 0; 	 Running data 16/171 (47) with seed=370.. 
Successful: 16/50. Skipped 0. Failed: 0; 	 Running data 17/171 (13) with seed=275.. 
Successful: 17/50. Skipped 0. Failed: 0; 	 Running data 18/171 (54) with seed=492.. 
Successful: 18/50. Skipped 0. Failed: 0; 	 Running data 19/171 (136) with seed=492.. 
Successful: 19/50. Skipped 0. Failed: 0; 	 Running data 20/171 (98) with seed=74.. 
Successful: 20/50. Skipped 0. Failed: 0; 	 Running data 21/171 (162) with seed=614.. 
Successful: 21/50. Skipped 0. Failed: 0; 	 Running data 22/171 (152) with seed=72.. 
Successful: 22/50. Skipped 0. Failed: 0; 	 Running data 23/171 (108) with seed=481.. 
Successful: 23/50. Skipped 0. Failed: 0; 	 Running data 24/171 (83) with seed=557.. 
Successful: 24/50. Skipped 0. Failed: 0; 	 Running data 25/171 (103) with seed=68.. 
Successful: 25/50. Skipped 0. Failed: 0; 	 Running data 26/171 (110) with seed=681.. 
Successful: 26/50. Skipped 0. Failed: 0; 	 Running data 27/171 (35) with seed=919.. 
Successful: 27/50. Skipped 0. Failed: 0; 	 Running data 28/171 (86) with seed=985.. 
Successful: 28/50. Skipped 0. Failed: 0; 	 Running data 29/171 (14) with seed=64.. 
Successful: 29/50. Skipped 0. Failed: 0; 	 Running data 30/171 (12) with seed=340.. 
Successful: 30/50. Skipped 0. Failed: 0; 	 Running data 31/171 (18) with seed=271.. 
Successful: 30/50. Skipped 0. Failed: 1; 	 Running data 32/171 (78) with seed=639.. 
Successful: 31/50. Skipped 0. Failed: 1; 	 Running data 33/171 (90) with seed=974.. 
Successful: 32/50. Skipped 0. Failed: 1; 	 Running data 34/171 (30) with seed=789.. 
Successful: 33/50. Skipped 0. Failed: 1; 	 Running data 35/171 (80) with seed=641.. 
Successful: 34/50. Skipped 0. Failed: 1; 	 Running data 36/171 (52) with seed=620.. 
Successful: 35/50. Skipped 0. Failed: 1; 	 Running data 37/171 (68) with seed=385.. 
Successful: 36/50. Skipped 0. Failed: 1; 	 Running data 38/171 (79) with seed=996.. 
Successful: 37/50. Skipped 0. Failed: 1; 	 Running data 39/171 (148) with seed=495.. 
Successful: 38/50. Skipped 0. Failed: 1; 	 Running data 40/171 (114) with seed=900.. 
Successful: 39/50. Skipped 0. Failed: 1; 	 Running data 41/171 (64) with seed=983.. 
Successful: 40/50. Skipped 0. Failed: 1; 	 Running data 42/171 (142) with seed=842.. 
Successful: 41/50. Skipped 0. Failed: 1; 	 Running data 43/171 (58) with seed=414.. 
Successful: 42/50. Skipped 0. Failed: 1; 	 Running data 44/171 (170) with seed=833.. 
Successful: 43/50. Skipped 0. Failed: 1; 	 Running data 45/171 (144) with seed=154.. 
Successful: 44/50. Skipped 0. Failed: 1; 	 Running data 46/171 (87) with seed=237.. 
Successful: 45/50. Skipped 0. Failed: 1; 	 Running data 47/171 (37) with seed=393.. 
Successful: 46/50. Skipped 0. Failed: 1; 	 Running data 48/171 (19) with seed=446.. 
Successful: 47/50. Skipped 0. Failed: 1; 	 Running data 49/171 (17) with seed=758.. 
Successful: 48/50. Skipped 0. Failed: 1; 	 Running data 50/171 (156) with seed=129.. 
Successful: 49/50. Skipped 0. Failed: 1; 	 Running data 51/171 (33) with seed=839.. 
 
Sanity check: 1.00
Successful: 50/50. Skipped 0. Failed: 1.
Verified: 0.94 - Violated: 0.00
 
---
pert_edge: 0.005, delta=0.001, model=models/PROTEINS_314496, do_enumeration=0
Seed: 825
Reading network and data in: /code/experiments/models/PROTEINS_314496
Successful: 0/50. Skipped 0. Failed: 0; 	 Running data 1/171 (13) with seed=889.. 
Successful: 1/50. Skipped 0. Failed: 0; 	 Running data 2/171 (101) with seed=936.. 
Successful: 2/50. Skipped 0. Failed: 0; 	 Running data 3/171 (39) with seed=487.. 
Successful: 3/50. Skipped 0. Failed: 0; 	 Running data 4/171 (166) with seed=532.. 
Successful: 4/50. Skipped 0. Failed: 0; 	 Running data 5/171 (27) with seed=527.. 
Successful: 5/50. Skipped 0. Failed: 0; 	 Running data 6/171 (45) with seed=857.. 
Successful: 6/50. Skipped 0. Failed: 0; 	 Running data 7/171 (40) with seed=541.. 
Successful: 7/50. Skipped 0. Failed: 0; 	 Running data 8/171 (34) with seed=762.. 
Successful: 8/50. Skipped 0. Failed: 0; 	 Running data 9/171 (32) with seed=345.. 
Successful: 9/50. Skipped 0. Failed: 0; 	 Running data 10/171 (104) with seed=586.. 
Successful: 10/50. Skipped 0. Failed: 0; 	 Running data 11/171 (65) with seed=860.. 
Successful: 11/50. Skipped 0. Failed: 0; 	 Running data 12/171 (43) with seed=228.. 
Successful: 12/50. Skipped 0. Failed: 0; 	 Running data 13/171 (131) with seed=211.. 
Successful: 13/50. Skipped 0. Failed: 0; 	 Running data 14/171 (146) with seed=969.. 
Successful: 14/50. Skipped 0. Failed: 0; 	 Running data 15/171 (9) with seed=944.. 
Successful: 15/50. Skipped 0. Failed: 0; 	 Running data 16/171 (148) with seed=478.. 
Successful: 16/50. Skipped 0. Failed: 0; 	 Running data 17/171 (83) with seed=655.. 
Successful: 17/50. Skipped 0. Failed: 0; 	 Running data 18/171 (25) with seed=300.. 
Successful: 18/50. Skipped 0. Failed: 0; 	 Running data 19/171 (41) with seed=118.. 
Successful: 19/50. Skipped 0. Failed: 0; 	 Running data 20/171 (77) with seed=178.. 
Successful: 20/50. Skipped 0. Failed: 0; 	 Running data 21/171 (61) with seed=880.. 
Successful: 21/50. Skipped 0. Failed: 0; 	 Running data 22/171 (23) with seed=133.. 
Successful: 22/50. Skipped 0. Failed: 0; 	 Running data 23/171 (59) with seed=138.. 
Successful: 23/50. Skipped 0. Failed: 0; 	 Running data 24/171 (42) with seed=461.. 
Successful: 24/50. Skipped 0. Failed: 0; 	 Running data 25/171 (158) with seed=604.. 
Successful: 25/50. Skipped 0. Failed: 0; 	 Running data 26/171 (151) with seed=270.. 
Successful: 26/50. Skipped 0. Failed: 0; 	 Running data 27/171 (92) with seed=869.. 
Successful: 27/50. Skipped 0. Failed: 0; 	 Running data 28/171 (116) with seed=636.. 
Successful: 28/50. Skipped 0. Failed: 0; 	 Running data 29/171 (75) with seed=731.. 
Successful: 29/50. Skipped 0. Failed: 0; 	 Running data 30/171 (117) with seed=602.. 
Successful: 30/50. Skipped 0. Failed: 0; 	 Running data 31/171 (91) with seed=99.. 
Successful: 31/50. Skipped 0. Failed: 0; 	 Running data 32/171 (122) with seed=123.. 
Successful: 32/50. Skipped 0. Failed: 0; 	 Running data 33/171 (168) with seed=353.. 
Successful: 33/50. Skipped 0. Failed: 0; 	 Running data 34/171 (49) with seed=14.. 
Successful: 34/50. Skipped 0. Failed: 0; 	 Running data 35/171 (6) with seed=591.. 
Successful: 35/50. Skipped 0. Failed: 0; 	 Running data 36/171 (170) with seed=762.. 
Successful: 36/50. Skipped 0. Failed: 0; 	 Running data 37/171 (129) with seed=897.. 
Successful: 37/50. Skipped 0. Failed: 0; 	 Running data 38/171 (100) with seed=337.. 
Successful: 38/50. Skipped 0. Failed: 0; 	 Running data 39/171 (53) with seed=729.. 
Successful: 39/50. Skipped 0. Failed: 0; 	 Running data 40/171 (68) with seed=454.. 
Successful: 40/50. Skipped 0. Failed: 0; 	 Running data 41/171 (84) with seed=693.. 
Successful: 41/50. Skipped 0. Failed: 0; 	 Running data 42/171 (50) with seed=761.. 
Successful: 42/50. Skipped 0. Failed: 0; 	 Running data 43/171 (72) with seed=322.. 
Successful: 43/50. Skipped 0. Failed: 0; 	 Running data 44/171 (132) with seed=775.. 
Successful: 44/50. Skipped 0. Failed: 0; 	 Running data 45/171 (46) with seed=616.. 
Successful: 45/50. Skipped 0. Failed: 0; 	 Running data 46/171 (18) with seed=905.. 
Successful: 45/50. Skipped 0. Failed: 1; 	 Running data 47/171 (17) with seed=60.. 
Successful: 46/50. Skipped 0. Failed: 1; 	 Running data 48/171 (31) with seed=799.. 
Successful: 47/50. Skipped 0. Failed: 1; 	 Running data 49/171 (28) with seed=494.. 
Successful: 48/50. Skipped 0. Failed: 1; 	 Running data 50/171 (110) with seed=814.. 
Successful: 49/50. Skipped 0. Failed: 1; 	 Running data 51/171 (67) with seed=941.. 
 
Sanity check: 1.00
Successful: 50/50. Skipped 0. Failed: 1.
Verified: 0.90 - Violated: 0.00
 
---
pert_edge: 0.010, delta=0.001, model=models/PROTEINS_314496, do_enumeration=0
Seed: 865
Reading network and data in: /code/experiments/models/PROTEINS_314496
Successful: 0/50. Skipped 0. Failed: 0; 	 Running data 1/171 (97) with seed=107.. 
Successful: 1/50. Skipped 0. Failed: 0; 	 Running data 2/171 (153) with seed=115.. 
Successful: 2/50. Skipped 0. Failed: 0; 	 Running data 3/171 (35) with seed=722.. 
Successful: 3/50. Skipped 0. Failed: 0; 	 Running data 4/171 (122) with seed=265.. 
Successful: 4/50. Skipped 0. Failed: 0; 	 Running data 5/171 (110) with seed=236.. 
Successful: 5/50. Skipped 0. Failed: 0; 	 Running data 6/171 (121) with seed=842.. 
Successful: 6/50. Skipped 0. Failed: 0; 	 Running data 7/171 (124) with seed=655.. 
Successful: 7/50. Skipped 0. Failed: 0; 	 Running data 8/171 (80) with seed=148.. 
Successful: 8/50. Skipped 0. Failed: 0; 	 Running data 9/171 (102) with seed=451.. 
Successful: 9/50. Skipped 0. Failed: 0; 	 Running data 10/171 (146) with seed=515.. 
Successful: 10/50. Skipped 0. Failed: 0; 	 Running data 11/171 (119) with seed=159.. 
Successful: 11/50. Skipped 0. Failed: 0; 	 Running data 12/171 (19) with seed=825.. 
Successful: 12/50. Skipped 0. Failed: 0; 	 Running data 13/171 (27) with seed=844.. 
Successful: 13/50. Skipped 0. Failed: 0; 	 Running data 14/171 (49) with seed=855.. 
Successful: 14/50. Skipped 0. Failed: 0; 	 Running data 15/171 (118) with seed=896.. 
Successful: 15/50. Skipped 0. Failed: 0; 	 Running data 16/171 (138) with seed=176.. 
Successful: 16/50. Skipped 0. Failed: 0; 	 Running data 17/171 (57) with seed=638.. 
Successful: 17/50. Skipped 0. Failed: 0; 	 Running data 18/171 (66) with seed=805.. 
Successful: 18/50. Skipped 0. Failed: 0; 	 Running data 19/171 (1) with seed=654.. 
Successful: 19/50. Skipped 0. Failed: 0; 	 Running data 20/171 (99) with seed=184.. 
Successful: 20/50. Skipped 0. Failed: 0; 	 Running data 21/171 (13) with seed=753.. 
Successful: 21/50. Skipped 0. Failed: 0; 	 Running data 22/171 (87) with seed=391.. 
Successful: 22/50. Skipped 0. Failed: 0; 	 Running data 23/171 (74) with seed=29.. 
Successful: 23/50. Skipped 0. Failed: 0; 	 Running data 24/171 (48) with seed=165.. 
Successful: 24/50. Skipped 0. Failed: 0; 	 Running data 25/171 (8) with seed=837.. 
Successful: 25/50. Skipped 0. Failed: 0; 	 Running data 26/171 (73) with seed=368.. 
Successful: 26/50. Skipped 0. Failed: 0; 	 Running data 27/171 (77) with seed=358.. 
Successful: 27/50. Skipped 0. Failed: 0; 	 Running data 28/171 (55) with seed=311.. 
Successful: 28/50. Skipped 0. Failed: 0; 	 Running data 29/171 (62) with seed=530.. 
Successful: 29/50. Skipped 0. Failed: 0; 	 Running data 30/171 (108) with seed=5.. 
Successful: 30/50. Skipped 0. Failed: 0; 	 Running data 31/171 (43) with seed=446.. 
Successful: 31/50. Skipped 0. Failed: 0; 	 Running data 32/171 (100) with seed=262.. 
Successful: 32/50. Skipped 0. Failed: 0; 	 Running data 33/171 (31) with seed=65.. 
Successful: 33/50. Skipped 0. Failed: 0; 	 Running data 34/171 (115) with seed=781.. 
Successful: 34/50. Skipped 0. Failed: 0; 	 Running data 35/171 (70) with seed=772.. 
Successful: 35/50. Skipped 0. Failed: 0; 	 Running data 36/171 (112) with seed=312.. 
Successful: 36/50. Skipped 0. Failed: 0; 	 Running data 37/171 (93) with seed=596.. 
Successful: 37/50. Skipped 0. Failed: 0; 	 Running data 38/171 (17) with seed=742.. 
Successful: 38/50. Skipped 0. Failed: 0; 	 Running data 39/171 (129) with seed=145.. 
Successful: 39/50. Skipped 0. Failed: 0; 	 Running data 40/171 (46) with seed=24.. 
Successful: 40/50. Skipped 0. Failed: 0; 	 Running data 41/171 (21) with seed=852.. 
Successful: 41/50. Skipped 0. Failed: 0; 	 Running data 42/171 (81) with seed=964.. 
Successful: 42/50. Skipped 0. Failed: 0; 	 Running data 43/171 (67) with seed=209.. 
Successful: 43/50. Skipped 0. Failed: 0; 	 Running data 44/171 (9) with seed=285.. 
Successful: 44/50. Skipped 0. Failed: 0; 	 Running data 45/171 (111) with seed=171.. 
Successful: 45/50. Skipped 0. Failed: 0; 	 Running data 46/171 (7) with seed=530.. 
Successful: 46/50. Skipped 0. Failed: 0; 	 Running data 47/171 (11) with seed=614.. 
Successful: 47/50. Skipped 0. Failed: 0; 	 Running data 48/171 (135) with seed=298.. 
Successful: 48/50. Skipped 0. Failed: 0; 	 Running data 49/171 (137) with seed=860.. 
Successful: 49/50. Skipped 0. Failed: 0; 	 Running data 50/171 (53) with seed=897.. 
 
Sanity check: 1.00
Successful: 50/50. Skipped 0. Failed: 0.
Verified: 0.94 - Violated: 0.00
 
---
pert_edge: 0.050, delta=0.001, model=models/PROTEINS_314496, do_enumeration=0
Seed: 889
Reading network and data in: /code/experiments/models/PROTEINS_314496
Successful: 0/50. Skipped 0. Failed: 0; 	 Running data 1/171 (113) with seed=732.. 
Successful: 1/50. Skipped 0. Failed: 0; 	 Running data 2/171 (159) with seed=79.. 
Successful: 2/50. Skipped 0. Failed: 0; 	 Running data 3/171 (91) with seed=778.. 
Successful: 3/50. Skipped 0. Failed: 0; 	 Running data 4/171 (158) with seed=368.. 
Successful: 4/50. Skipped 0. Failed: 0; 	 Running data 5/171 (141) with seed=588.. 
Successful: 5/50. Skipped 0. Failed: 0; 	 Running data 6/171 (65) with seed=848.. 
Successful: 6/50. Skipped 0. Failed: 0; 	 Running data 7/171 (30) with seed=867.. 
Successful: 7/50. Skipped 0. Failed: 0; 	 Running data 8/171 (95) with seed=815.. 
Successful: 8/50. Skipped 0. Failed: 0; 	 Running data 9/171 (96) with seed=258.. 
Successful: 9/50. Skipped 0. Failed: 0; 	 Running data 10/171 (77) with seed=214.. 
Successful: 10/50. Skipped 0. Failed: 0; 	 Running data 11/171 (48) with seed=4.. 
Successful: 11/50. Skipped 0. Failed: 0; 	 Running data 12/171 (90) with seed=8.. 
Successful: 12/50. Skipped 0. Failed: 0; 	 Running data 13/171 (54) with seed=738.. 
Successful: 13/50. Skipped 0. Failed: 0; 	 Running data 14/171 (125) with seed=528.. 
Successful: 14/50. Skipped 0. Failed: 0; 	 Running data 15/171 (13) with seed=259.. 
Successful: 15/50. Skipped 0. Failed: 0; 	 Running data 16/171 (132) with seed=378.. 
Successful: 16/50. Skipped 0. Failed: 0; 	 Running data 17/171 (66) with seed=815.. 
Successful: 17/50. Skipped 0. Failed: 0; 	 Running data 18/171 (142) with seed=509.. 
Successful: 18/50. Skipped 0. Failed: 0; 	 Running data 19/171 (137) with seed=208.. 
Successful: 19/50. Skipped 0. Failed: 0; 	 Running data 20/171 (139) with seed=830.. 
Successful: 20/50. Skipped 0. Failed: 0; 	 Running data 21/171 (38) with seed=798.. 
Successful: 21/50. Skipped 0. Failed: 0; 	 Running data 22/171 (170) with seed=608.. 
Successful: 22/50. Skipped 0. Failed: 0; 	 Running data 23/171 (152) with seed=469.. 
Successful: 23/50. Skipped 0. Failed: 0; 	 Running data 24/171 (98) with seed=371.. 
Successful: 24/50. Skipped 0. Failed: 0; 	 Running data 25/171 (50) with seed=998.. 
Successful: 25/50. Skipped 0. Failed: 0; 	 Running data 26/171 (124) with seed=363.. 
Successful: 26/50. Skipped 0. Failed: 0; 	 Running data 27/171 (147) with seed=228.. 
Successful: 27/50. Skipped 0. Failed: 0; 	 Running data 28/171 (34) with seed=886.. 
Successful: 28/50. Skipped 0. Failed: 0; 	 Running data 29/171 (41) with seed=344.. 
Successful: 29/50. Skipped 0. Failed: 0; 	 Running data 30/171 (26) with seed=546.. 
Successful: 30/50. Skipped 0. Failed: 0; 	 Running data 31/171 (43) with seed=816.. 
Successful: 31/50. Skipped 0. Failed: 0; 	 Running data 32/171 (145) with seed=92.. 
Successful: 32/50. Skipped 0. Failed: 0; 	 Running data 33/171 (31) with seed=857.. 
Successful: 33/50. Skipped 0. Failed: 0; 	 Running data 34/171 (94) with seed=160.. 
Successful: 34/50. Skipped 0. Failed: 0; 	 Running data 35/171 (136) with seed=536.. 
Successful: 35/50. Skipped 0. Failed: 0; 	 Running data 36/171 (17) with seed=290.. 
