2025-05-16 01:57:10,404 [INFO ] [logger.py:dumpArgs:73]	 Starting with arguments
{"name": "cifar10", "dataset": {"path": "../../datasets/cifar10", "image_size": 224, "sensor_size": 56}, "cache_directory": null, "output": "./results/mvce-cifar10/", "classifier": "./models/cifar10/cifar10-20250515-150106.pklz", "classifier_feature": null, "mask": null, "drop": {"name": "block-dropout"}, "threads": 4, "generator_samples": [], "decision_samples": 1000, "cuda_index": 0, "zero_imputation": true, "zero_variance": true, "beta_variance_scales": [0.5, 0.99], "action_spaces": [{"name": "zero-one"}], "class_count": 1, "batch_size": 100, "batch_mean_imputation": [], "buckets": 10, "trials": 4, "seed": 1337, "verbose": 1}
2025-05-16 01:57:10,435 [INFO ] [util.py:selectDevice:87]	 Using cuda:0 for tensor calculations
2025-05-16 01:57:10,590 [INFO ] [mvce_dataset.py:<module>:90]	 Running on cuda:0
2025-05-16 01:57:10,877 [INFO ] [loader.py:getDatasetSplits:20]	 Loading CIFAR10 dataset from ../../datasets/cifar10
2025-05-16 01:57:10,877 [INFO ] [baselines.py:createCIFAR10Dataset:113]	 Using 10 CIFAR10 targets: ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
2025-05-16 01:57:10,948 [INFO ] [baselines.py:getTransform:62]	 Resizing images from 32 to 224
2025-05-16 01:57:10,948 [INFO ] [baselines.py:getTransform:75]	 Applying CIFAR10 standard normalization
2025-05-16 01:57:16,138 [INFO ] [baselines.py:splitTrainingValidation:47]	 Split training from 50000 into 35000 training images and 15000 validation images. Found 10000 testing images.
2025-05-16 01:57:16,138 [INFO ] [mvce_dataset.py:<module>:102]	 Using dataset cifar10 with 10000 test samples
2025-05-16 01:57:16,138 [INFO ] [mvce_dataset.py:<module>:131]	 Setting classifier activation function to softmax for multiclass
2025-05-16 01:57:16,138 [INFO ] [mvce_dataset.py:<module>:156]	 Including baseline imputators with zero imputation.
2025-05-16 01:57:16,138 [INFO ] [mvce_dataset.py:<module>:166]	 Running all baseline imputators zero variance.
2025-05-16 01:57:16,138 [INFO ] [mvce_dataset.py:<module>:168]	 Running all baseline imputators with [0.5, 0.99] scaled beta max variance.
2025-05-16 01:57:16,138 [INFO ] [mvce_dataset.py:<module>:184]	 Adding 3 methods.
2025-05-16 01:57:16,139 [INFO ] [mvce_dataset.py:<module>:196]	 Using randon dropping with arguments {'name': 'block-dropout'}
2025-05-16 01:57:16,139 [INFO ] [mvce_dataset.py:<module>:208]	 Considering action space zero-one
2025-05-16 01:57:16,139 [INFO ] [threading_utils.py:distributeTasks:91]	 Running 3 tasks each on their own thread
2025-05-16 01:57:16,140 [INFO ] [calibration.py:__call__:223]	 Started running Basic Imputation - Zero Imputation missing block-dropout in zero-one
2025-05-16 01:57:16,140 [INFO ] [calibration.py:__call__:223]	 Started running Beta Mean * 0.5 - Zero Imputation missing block-dropout in zero-one
2025-05-16 01:57:16,140 [INFO ] [calibration.py:__call__:223]	 Started running Beta Mean * 0.99 - Zero Imputation missing block-dropout in zero-one
2025-05-16 01:59:25,079 [INFO ] [calibration.py:computeMVCE:154]	 
        Computed MVCE 0.42048192024230957 for Beta Mean * 0.99 - Zero Imputation in 128.93249538820237 seconds with 10 buckets:
        * Non-zero buckets: tensor([3, 4, 5, 6, 7, 8, 9])
        * Final bucket sizes: tensor([  79, 3422, 2211,  947,  819,  857, 1665], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([0.3934, 0.4498, 0.5450, 0.6466, 0.7494, 0.8512, 0.9626])
        * Final bucket consistencies: tensor([0.0380, 0.1029, 0.3247, 0.1309, 0.1844, 0.2205, 0.3556])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([1298,  941,  797,  733,  893,  900, 1363,  815, 1300,  960],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([8708,  199,  140,  141,  123,  265,   45,  139,   84,  156],
       dtype=torch.int32)
        * Average consistency: 0.21289999783039093
    
2025-05-16 01:59:25,169 [INFO ] [calibration.py:computeMVCE:154]	 
        Computed MVCE 0.6545000076293945 for Basic Imputation - Zero Imputation in 129.0241113146767 seconds with 10 buckets:
        * Non-zero buckets: 9
        * Final bucket sizes: tensor([10000], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([1.])
        * Final bucket consistencies: tensor([0.3455])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([1298,  941,  797,  733,  893,  900, 1363,  815, 1300,  960],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([3856,  472,  915, 1533,  645, 1086,  183,  428,  375,  507],
       dtype=torch.int32)
        * Average consistency: 0.34549999237060547
    
2025-05-16 01:59:25,256 [INFO ] [calibration.py:computeMVCE:154]	 
        Computed MVCE 0.3267455995082855 for Beta Mean * 0.5 - Zero Imputation in 129.1107664294541 seconds with 10 buckets:
        * Non-zero buckets: tensor([1, 2, 3, 4, 5, 6, 7, 8, 9])
        * Final bucket sizes: tensor([   4,  239,  961, 1458, 1479, 1284, 1272, 1257, 2046],
       dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([0.1935, 0.2723, 0.3557, 0.4507, 0.5470, 0.6483, 0.7488, 0.8498, 0.9614])
        * Final bucket consistencies: tensor([0.2500, 0.1464, 0.1842, 0.2387, 0.2989, 0.3255, 0.3451, 0.3914, 0.5254])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([1298,  941,  797,  733,  893,  900, 1363,  815, 1300,  960],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([3983,  436,  914, 1578,  651, 1016,  187,  427,  325,  483],
       dtype=torch.int32)
        * Average consistency: 0.3427000045776367
    
2025-05-16 02:01:29,335 [INFO ] [calibration.py:computeMVCE:154]	 
        Computed MVCE 0.41807857155799866 for Beta Mean * 0.99 - Zero Imputation in 124.25353200919926 seconds with 10 buckets:
        * Non-zero buckets: tensor([3, 4, 5, 6, 7, 8, 9])
        * Final bucket sizes: tensor([  80, 3466, 2209,  950,  831,  851, 1613], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([0.3930, 0.4500, 0.5439, 0.6477, 0.7494, 0.8521, 0.9627])
        * Final bucket consistencies: tensor([0.0125, 0.1062, 0.3214, 0.1632, 0.1456, 0.2021, 0.3726])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([1298,  941,  797,  733,  893,  900, 1363,  815, 1300,  960],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([8694,  155,  158,  167,  124,  236,   39,  144,   90,  193],
       dtype=torch.int32)
        * Average consistency: 0.21279999613761902
    
2025-05-16 02:01:29,345 [INFO ] [calibration.py:computeMVCE:154]	 
        Computed MVCE 0.31712430715560913 for Beta Mean * 0.5 - Zero Imputation in 124.08744826819748 seconds with 10 buckets:
        * Non-zero buckets: tensor([1, 2, 3, 4, 5, 6, 7, 8, 9])
        * Final bucket sizes: tensor([   4,  263,  973, 1455, 1494, 1311, 1180, 1267, 2053],
       dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([0.1855, 0.2679, 0.3553, 0.4513, 0.5471, 0.6480, 0.7471, 0.8507, 0.9624])
        * Final bucket consistencies: tensor([0.2500, 0.1939, 0.2138, 0.2543, 0.2985, 0.3272, 0.3534, 0.4025, 0.5226])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([1298,  941,  797,  733,  893,  900, 1363,  815, 1300,  960],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([3911,  461,  893, 1612,  629, 1014,  159,  436,  373,  512],
       dtype=torch.int32)
        * Average consistency: 0.3504999876022339
    
2025-05-16 02:01:30,496 [INFO ] [calibration.py:computeMVCE:154]	 
        Computed MVCE 0.6578999757766724 for Basic Imputation - Zero Imputation in 125.31664744578302 seconds with 10 buckets:
        * Non-zero buckets: 9
        * Final bucket sizes: tensor([10000], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([1.])
        * Final bucket consistencies: tensor([0.3421])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([1298,  941,  797,  733,  893,  900, 1363,  815, 1300,  960],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([3958,  427,  908, 1573,  650, 1035,  172,  428,  368,  481],
       dtype=torch.int32)
        * Average consistency: 0.34209999442100525
    
2025-05-16 02:04:01,425 [INFO ] [calibration.py:computeMVCE:154]	 
        Computed MVCE 0.6502000093460083 for Basic Imputation - Zero Imputation in 150.92638353444636 seconds with 10 buckets:
        * Non-zero buckets: 9
        * Final bucket sizes: tensor([10000], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([1.])
        * Final bucket consistencies: tensor([0.3498])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([1298,  941,  797,  733,  893,  900, 1363,  815, 1300,  960],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([3962,  465,  910, 1531,  671,  997,  161,  409,  382,  512],
       dtype=torch.int32)
        * Average consistency: 0.3497999906539917
    
2025-05-16 02:04:04,616 [INFO ] [calibration.py:computeMVCE:154]	 
        Computed MVCE 0.42279160022735596 for Beta Mean * 0.99 - Zero Imputation in 155.27574333082885 seconds with 10 buckets:
        * Non-zero buckets: tensor([3, 4, 5, 6, 7, 8, 9])
        * Final bucket sizes: tensor([  64, 3446, 2175,  961,  808,  887, 1659], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([0.3929, 0.4504, 0.5450, 0.6464, 0.7495, 0.8515, 0.9616])
        * Final bucket consistencies: tensor([0.0469, 0.1146, 0.3034, 0.1488, 0.1770, 0.2266, 0.3418])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([1298,  941,  797,  733,  893,  900, 1363,  815, 1300,  960],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([8681,  151,  170,  170,  120,  271,   52,  150,   81,  154],
       dtype=torch.int32)
        * Average consistency: 0.21119999885559082
    
2025-05-16 02:04:06,154 [INFO ] [calibration.py:computeMVCE:154]	 
        Computed MVCE 0.3313601613044739 for Beta Mean * 0.5 - Zero Imputation in 156.8046307032928 seconds with 10 buckets:
        * Non-zero buckets: tensor([1, 2, 3, 4, 5, 6, 7, 8, 9])
        * Final bucket sizes: tensor([   3,  277,  979, 1473, 1487, 1278, 1166, 1264, 2073],
       dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([0.1943, 0.2711, 0.3560, 0.4515, 0.5489, 0.6482, 0.7506, 0.8497, 0.9617])
        * Final bucket consistencies: tensor([0.3333, 0.1480, 0.1992, 0.2539, 0.2771, 0.3216, 0.3482, 0.3647, 0.5128])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([1298,  941,  797,  733,  893,  900, 1363,  815, 1300,  960],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([4036,  435,  904, 1539,  645, 1045,  149,  413,  344,  490],
       dtype=torch.int32)
        * Average consistency: 0.33640000224113464
    
2025-05-16 02:06:31,784 [INFO ] [calibration.py:computeMVCE:154]	 
        Computed MVCE 0.6668000221252441 for Basic Imputation - Zero Imputation in 150.35608596168458 seconds with 10 buckets:
        * Non-zero buckets: 9
        * Final bucket sizes: tensor([10000], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([1.])
        * Final bucket consistencies: tensor([0.3332])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([1298,  941,  797,  733,  893,  900, 1363,  815, 1300,  960],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([4061,  443,  908, 1639,  615, 1004,  158,  396,  340,  436],
       dtype=torch.int32)
        * Average consistency: 0.33320000767707825
    
2025-05-16 02:06:31,784 [INFO ] [calibration.py:__call__:249]	 Finished running Basic Imputation - Zero Imputation missing block-dropout in zero-one in 555.6442910637707 seconds
2025-05-16 02:06:37,139 [INFO ] [calibration.py:computeMVCE:154]	 
        Computed MVCE 0.4216172993183136 for Beta Mean * 0.99 - Zero Imputation in 152.5196886220947 seconds with 10 buckets:
        * Non-zero buckets: tensor([3, 4, 5, 6, 7, 8, 9])
        * Final bucket sizes: tensor([  81, 3417, 2180,  983,  844,  866, 1629], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([0.3925, 0.4506, 0.5447, 0.6458, 0.7485, 0.8510, 0.9619])
        * Final bucket consistencies: tensor([0.0370, 0.1030, 0.3174, 0.1394, 0.1730, 0.2067, 0.3708])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([1298,  941,  797,  733,  893,  900, 1363,  815, 1300,  960],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([8712,  164,  173,  163,  121,  233,   53,  121,   84,  176],
       dtype=torch.int32)
        * Average consistency: 0.21130000054836273
    
2025-05-16 02:06:37,140 [INFO ] [calibration.py:__call__:249]	 Finished running Beta Mean * 0.99 - Zero Imputation missing block-dropout in zero-one in 560.9968058727682 seconds
2025-05-16 02:06:37,585 [INFO ] [calibration.py:computeMVCE:154]	 
        Computed MVCE 0.33513370156288147 for Beta Mean * 0.5 - Zero Imputation in 151.42954950407147 seconds with 10 buckets:
        * Non-zero buckets: tensor([1, 2, 3, 4, 5, 6, 7, 8, 9])
        * Final bucket sizes: tensor([   4,  274,  969, 1546, 1395, 1276, 1186, 1229, 2121],
       dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([0.1963, 0.2678, 0.3540, 0.4507, 0.5492, 0.6494, 0.7507, 0.8510, 0.9623])
        * Final bucket consistencies: tensor([0.2500, 0.1861, 0.2033, 0.2426, 0.2624, 0.2994, 0.3356, 0.3995, 0.5064])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([1298,  941,  797,  733,  893,  900, 1363,  815, 1300,  960],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([3965,  448,  955, 1580,  651,  996,  168,  395,  340,  502],
       dtype=torch.int32)
        * Average consistency: 0.3334999978542328
    
2025-05-16 02:06:37,585 [INFO ] [calibration.py:__call__:249]	 Finished running Beta Mean * 0.5 - Zero Imputation missing block-dropout in zero-one in 561.4426637832075 seconds
2025-05-16 02:06:37,586 [INFO ] [mvce_dataset.py:<module>:222]	 Finished running 3/3 experiments.
2025-05-16 02:06:37,586 [INFO ] [mvce_dataset.py:<module>:227]	 Saving results to ./results/mvce-cifar10/cifar10-20250516-015710.csv
2025-05-16 02:06:37,587 [INFO ] [mvce_dataset.py:<module>:237]	 Finished saving results
