Running 1 scripts.. 
 
------------------------------------------------------------------
 
Running 'trainModels' ...
GPU available: 1
Running evaluation for "cnn3-bn (relu)" on "mnist" [2025-Mar-31-12-21-15]
Saving results in ./results/mnist-ablation/evaluation
Loading training configuration "mnist"... done
Loading dataset "mnist"... done
Found trained networks...
Extracting training times, run (1/5)...
--- config "Set (tau=0.00)" (1/8)... done
--- config "Set (tau=0.01)" (2/8)... done
--- config "Set (tau=0.10)" (3/8)... done
--- config "Set (tau=0.20)" (4/8)... done
--- config "Set (tau=0.30)" (5/8)... done
--- config "Set (Zonotope+ours)" (6/8)... done
--- config "Set (Zonotope+Singh)" (7/8)... done
--- config "Set (IBP)" (8/8)... done
Extracting training times, run (2/5)...
--- config "Set (tau=0.00)" (1/8)... done
--- config "Set (tau=0.01)" (2/8)... done
--- config "Set (tau=0.10)" (3/8)... done
--- config "Set (tau=0.20)" (4/8)... done
--- config "Set (tau=0.30)" (5/8)... done
--- config "Set (Zonotope+ours)" (6/8)... done
--- config "Set (Zonotope+Singh)" (7/8)... done
--- config "Set (IBP)" (8/8)... done
Extracting training times, run (3/5)...
--- config "Set (tau=0.00)" (1/8)... done
--- config "Set (tau=0.01)" (2/8)... done
--- config "Set (tau=0.10)" (3/8)... done
--- config "Set (tau=0.20)" (4/8)... done
--- config "Set (tau=0.30)" (5/8)... done
--- config "Set (Zonotope+ours)" (6/8)... done
--- config "Set (Zonotope+Singh)" (7/8)... done
--- config "Set (IBP)" (8/8)... done
Extracting training times, run (4/5)...
--- config "Set (tau=0.00)" (1/8)... done
--- config "Set (tau=0.01)" (2/8)... done
--- config "Set (tau=0.10)" (3/8)... done
--- config "Set (tau=0.20)" (4/8)... done
--- config "Set (tau=0.30)" (5/8)... done
--- config "Set (Zonotope+ours)" (6/8)... done
--- config "Set (Zonotope+Singh)" (7/8)... done
--- config "Set (IBP)" (8/8)... done
Extracting training times, run (5/5)...
--- config "Set (tau=0.00)" (1/8)... done
--- config "Set (tau=0.01)" (2/8)... done
--- config "Set (tau=0.10)" (3/8)... done
--- config "Set (tau=0.20)" (4/8)... done
--- config "Set (tau=0.30)" (5/8)... done
--- config "Set (Zonotope+ours)" (6/8)... done
--- config "Set (Zonotope+Singh)" (7/8)... done
--- config "Set (IBP)" (8/8)... done
Generating results table...
_____________________________________________________________________________________________________
        Method        |   Accuracy   | falsified Acc. (eps=0.1) | fast-verif. Acc. (eps=0.1) | [max] 
-----------------------------------------------------------------------------------------------------
 Set (tau=0.00)       | 98.87 ± 0.01 |       96.48 ± 0.18       |        89.40 ± 0.58        | 90.04 
 Set (tau=0.01)       | 98.74 ± 0.07 |       96.77 ± 0.29       |        93.45 ± 0.47        | 93.73 
 Set (tau=0.10)       | 97.57 ± 0.09 |       95.60 ± 0.17       |        94.03 ± 0.04        | 94.06 
 Set (tau=0.20)       | 97.17 ± 0.05 |       95.07 ± 0.12       |        93.70 ± 0.12        | 93.82 
 Set (tau=0.30)       | 96.59 ± 0.07 |       93.46 ± 1.20       |        88.91 ± 4.36        | 93.95 
 Set (Zonotope+ours)  | 97.58 ± 0.03 |       95.31 ± 0.05       |        93.83 ± 0.06        | 93.89 
 Set (Zonotope+Singh) | 97.60 ± 0.07 |       95.21 ± 0.13       |        93.53 ± 0.40        | 93.99 
 Set (IBP)            | 93.16 ± 0.16 |       74.90 ± 1.04       |         0.15 ± 0.17        |  0.34 
_____________________________________________________________________________________________________
 
 done
Generating training time table...
_____________________________________________________
        Method        | (min) Train Time [s / Epoch] 
-----------------------------------------------------
 Set (tau=0.00)       |                         10.5 
 Set (tau=0.01)       |                         10.5 
 Set (tau=0.10)       |                         10.2 
 Set (tau=0.20)       |                         10.1 
 Set (tau=0.30)       |                         10.1 
 Set (Zonotope+ours)  |                         33.1 
 Set (Zonotope+Singh) |                         33.2 
 Set (IBP)            |                         30.5 
_____________________________________________________
 
 done
 
'trainModels' was run successfully!
Saving plots to './results/mnist-ablation/plots'..
 
------------------------------------------------------------------
 
Completed!
Date: 31-Mar-2025 12:21:17
