2025-05-16 01:57:06,996 [INFO ] [logger.py:dumpArgs:73]	 Starting with arguments
{"name": "starcraft", "dataset": {"path": "../../datasets/starcraftimage", "image_format": "cifar10", "image_size": 224, "sensor_size": 56}, "cache_directory": null, "output": "./results/mvce-starcraft/", "classifier": "./models/starcraft-cifar10/starcraft-20250515-150053.pklz", "classifier_feature": null, "mask": null, "drop": {"name": "block-dropout"}, "threads": 4, "generator_samples": [], "decision_samples": 1000, "cuda_index": 1, "zero_imputation": true, "zero_variance": true, "beta_variance_scales": [0.5, 0.99], "action_spaces": [{"name": "zero-one"}], "class_count": 1, "batch_size": 100, "batch_mean_imputation": [], "buckets": 10, "trials": 4, "seed": 1337, "verbose": 1}
2025-05-16 01:57:07,029 [INFO ] [util.py:selectDevice:87]	 Using cuda:1 for tensor calculations
2025-05-16 01:57:07,178 [INFO ] [mvce_dataset.py:<module>:90]	 Running on cuda:1
2025-05-16 01:57:07,500 [INFO ] [loader.py:getDatasetSplits:23]	 Loading StarCraft dataset from ../../datasets/starcraftimage
2025-05-16 01:57:07,500 [INFO ] [starcraft.py:createStarCraftDataset:97]	 Using 10 StarCraft targets: ['Acolyte LE at Beginning', 'Acolyte LE at End', 'Abyssal Reef LE at Beginning', 'Abyssal Reef LE at End', 'Ascension to Aiur LE at Beginning', 'Ascension to Aiur LE at End', 'Mech Depot LE at Beginning', 'Mech Depot LE at End', 'Odyssey LE at Beginning', 'Odyssey LE at End']
2025-05-16 01:57:07,557 [INFO ] [baselines.py:getTransform:62]	 Resizing images from 32 to 224
2025-05-16 01:57:07,557 [INFO ] [baselines.py:getTransform:78]	 Applying 0.5 normalization
2025-05-16 01:57:09,444 [INFO ] [baselines.py:splitTrainingValidation:47]	 Split training from 50000 into 35000 training images and 15000 validation images. Found 10000 testing images.
2025-05-16 01:57:09,444 [INFO ] [mvce_dataset.py:<module>:102]	 Using dataset starcraft with 10000 test samples
2025-05-16 01:57:09,444 [INFO ] [mvce_dataset.py:<module>:131]	 Setting classifier activation function to softmax for multiclass
2025-05-16 01:57:09,444 [INFO ] [mvce_dataset.py:<module>:156]	 Including baseline imputators with zero imputation.
2025-05-16 01:57:09,444 [INFO ] [mvce_dataset.py:<module>:166]	 Running all baseline imputators zero variance.
2025-05-16 01:57:09,444 [INFO ] [mvce_dataset.py:<module>:168]	 Running all baseline imputators with [0.5, 0.99] scaled beta max variance.
2025-05-16 01:57:09,445 [INFO ] [mvce_dataset.py:<module>:184]	 Adding 3 methods.
2025-05-16 01:57:09,445 [INFO ] [mvce_dataset.py:<module>:196]	 Using randon dropping with arguments {'name': 'block-dropout'}
2025-05-16 01:57:09,445 [INFO ] [mvce_dataset.py:<module>:208]	 Considering action space zero-one
2025-05-16 01:57:09,445 [INFO ] [threading_utils.py:distributeTasks:91]	 Running 3 tasks each on their own thread
2025-05-16 01:57:09,445 [INFO ] [calibration.py:__call__:223]	 Started running Basic Imputation - Zero Imputation missing block-dropout in zero-one
2025-05-16 01:57:09,445 [INFO ] [calibration.py:__call__:223]	 Started running Beta Mean * 0.5 - Zero Imputation missing block-dropout in zero-one
2025-05-16 01:57:09,446 [INFO ] [calibration.py:__call__:223]	 Started running Beta Mean * 0.99 - Zero Imputation missing block-dropout in zero-one
2025-05-16 01:59:16,661 [INFO ] [calibration.py:computeMVCE:154]	 
        Computed MVCE 0.8499000072479248 for Basic Imputation - Zero Imputation in 127.21101611852646 seconds with 10 buckets:
        * Non-zero buckets: 9
        * Final bucket sizes: tensor([10000], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([1.])
        * Final bucket consistencies: tensor([0.1501])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([1222,  778, 1190,  813, 1226,  771, 1090,  909, 1186,  815],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([ 124,    0, 5645,    0,    1,    0, 4211,    5,   12,    2],
       dtype=torch.int32)
        * Average consistency: 0.1500999927520752
    
2025-05-16 01:59:16,744 [INFO ] [calibration.py:computeMVCE:154]	 
        Computed MVCE 0.7672894597053528 for Beta Mean * 0.5 - Zero Imputation in 127.2941481443122 seconds with 10 buckets:
        * Non-zero buckets: tensor([3, 4, 5, 6, 7, 8, 9])
        * Final bucket sizes: tensor([  11,   98,  426,  483,  652, 1044, 7286], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([0.3651, 0.4682, 0.5503, 0.6507, 0.7522, 0.8541, 0.9815])
        * Final bucket consistencies: tensor([0.0909, 0.1429, 0.1362, 0.1242, 0.1181, 0.1188, 0.1544])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([1222,  778, 1190,  813, 1226,  771, 1090,  909, 1186,  815],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([ 120,    0, 5578,    0,    2,    1, 4276,    0,   19,    4],
       dtype=torch.int32)
        * Average consistency: 0.14589999616146088
    
2025-05-16 01:59:19,324 [INFO ] [calibration.py:computeMVCE:154]	 
        Computed MVCE 0.38998833298683167 for Beta Mean * 0.99 - Zero Imputation in 129.87194828037173 seconds with 10 buckets:
        * Non-zero buckets: tensor([3, 4, 5, 6, 7, 8, 9])
        * Final bucket sizes: tensor([  73, 2194, 6970,  661,   37,   24,   41], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([0.3920, 0.4476, 0.5639, 0.6101, 0.7515, 0.8506, 0.9668])
        * Final bucket consistencies: tensor([0.0274, 0.1267, 0.1532, 0.2027, 0.4865, 0.4167, 0.5366])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([1222,  778, 1190,  813, 1226,  771, 1090,  909, 1186,  815],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([1403,    1, 5064,    0,    0,    0, 3520,    0,   12,    0],
       dtype=torch.int32)
        * Average consistency: 0.15320000052452087
    
2025-05-16 02:01:21,708 [INFO ] [calibration.py:computeMVCE:154]	 
        Computed MVCE 0.8560000061988831 for Basic Imputation - Zero Imputation in 125.02809438575059 seconds with 10 buckets:
        * Non-zero buckets: 9
        * Final bucket sizes: tensor([10000], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([1.])
        * Final bucket consistencies: tensor([0.1440])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([1222,  778, 1190,  813, 1226,  771, 1090,  909, 1186,  815],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([ 103,    0, 5579,    0,    1,    0, 4303,    3,   10,    1],
       dtype=torch.int32)
        * Average consistency: 0.14399999380111694
    
2025-05-16 02:01:29,536 [INFO ] [calibration.py:computeMVCE:154]	 
        Computed MVCE 0.7712302803993225 for Beta Mean * 0.5 - Zero Imputation in 132.78406305983663 seconds with 10 buckets:
        * Non-zero buckets: tensor([3, 4, 5, 6, 7, 8, 9])
        * Final bucket sizes: tensor([  10,   59,  422,  486,  590, 1019, 7414], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([0.3696, 0.4649, 0.5485, 0.6507, 0.7525, 0.8568, 0.9809])
        * Final bucket consistencies: tensor([0.2000, 0.1695, 0.1043, 0.1379, 0.1288, 0.1384, 0.1505])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([1222,  778, 1190,  813, 1226,  771, 1090,  909, 1186,  815],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([  88,    0, 5618,    0,    1,    0, 4274,    8,   10,    1],
       dtype=torch.int32)
        * Average consistency: 0.14560000598430634
    
2025-05-16 02:01:31,048 [INFO ] [calibration.py:computeMVCE:154]	 
        Computed MVCE 0.3902793228626251 for Beta Mean * 0.99 - Zero Imputation in 131.71164130885154 seconds with 10 buckets:
        * Non-zero buckets: tensor([3, 4, 5, 6, 7, 8, 9])
        * Final bucket sizes: tensor([  64, 2200, 6990,  653,   27,   27,   39], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([0.3929, 0.4463, 0.5640, 0.6102, 0.7506, 0.8419, 0.9737])
        * Final bucket consistencies: tensor([0.0781, 0.1355, 0.1515, 0.1822, 0.5185, 0.4815, 0.4615])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([1222,  778, 1190,  813, 1226,  771, 1090,  909, 1186,  815],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([1434,    0, 5017,    0,    0,    0, 3546,    0,    3,    0],
       dtype=torch.int32)
        * Average consistency: 0.1526000052690506
    
2025-05-16 02:03:57,636 [INFO ] [calibration.py:computeMVCE:154]	 
        Computed MVCE 0.8518999814987183 for Basic Imputation - Zero Imputation in 155.9206382604316 seconds with 10 buckets:
        * Non-zero buckets: 9
        * Final bucket sizes: tensor([10000], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([1.])
        * Final bucket consistencies: tensor([0.1481])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([1222,  778, 1190,  813, 1226,  771, 1090,  909, 1186,  815],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([ 115,    0, 5629,    0,    0,    0, 4235,    4,   15,    2],
       dtype=torch.int32)
        * Average consistency: 0.14810000360012054
    
2025-05-16 02:04:08,741 [INFO ] [calibration.py:computeMVCE:154]	 
        Computed MVCE 0.7711998820304871 for Beta Mean * 0.5 - Zero Imputation in 159.1992050083354 seconds with 10 buckets:
        * Non-zero buckets: tensor([3, 4, 5, 6, 7, 8, 9])
        * Final bucket sizes: tensor([  10,   58,  418,  487,  621, 1039, 7367], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([0.3770, 0.4779, 0.5504, 0.6517, 0.7498, 0.8560, 0.9811])
        * Final bucket consistencies: tensor([0.2000, 0.1724, 0.1388, 0.1129, 0.1208, 0.1232, 0.1523])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([1222,  778, 1190,  813, 1226,  771, 1090,  909, 1186,  815],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([ 118,    0, 5653,    0,    1,    0, 4214,    3,   10,    1],
       dtype=torch.int32)
        * Average consistency: 0.14499999582767487
    
2025-05-16 02:04:13,835 [INFO ] [calibration.py:computeMVCE:154]	 
        Computed MVCE 0.38933253288269043 for Beta Mean * 0.99 - Zero Imputation in 162.7824933975935 seconds with 10 buckets:
        * Non-zero buckets: tensor([3, 4, 5, 6, 7, 8, 9])
        * Final bucket sizes: tensor([  76, 2202, 6953,  654,   36,   27,   52], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([0.3925, 0.4466, 0.5630, 0.6105, 0.7514, 0.8502, 0.9627])
        * Final bucket consistencies: tensor([0.0921, 0.1335, 0.1497, 0.1927, 0.5833, 0.5556, 0.5769])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([1222,  778, 1190,  813, 1226,  771, 1090,  909, 1186,  815],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([1451,    0, 4953,    0,    1,    0, 3591,    0,    4,    0],
       dtype=torch.int32)
        * Average consistency: 0.1534000039100647
    
2025-05-16 02:06:31,849 [INFO ] [calibration.py:computeMVCE:154]	 
        Computed MVCE 0.8560000061988831 for Basic Imputation - Zero Imputation in 154.19965017866343 seconds with 10 buckets:
        * Non-zero buckets: 9
        * Final bucket sizes: tensor([10000], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([1.])
        * Final bucket consistencies: tensor([0.1440])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([1222,  778, 1190,  813, 1226,  771, 1090,  909, 1186,  815],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([ 121,    0, 5592,    0,    0,    0, 4268,    6,   12,    1],
       dtype=torch.int32)
        * Average consistency: 0.14399999380111694
    
2025-05-16 02:06:31,850 [INFO ] [calibration.py:__call__:249]	 Finished running Basic Imputation - Zero Imputation missing block-dropout in zero-one in 562.4042822523043 seconds
2025-05-16 02:06:39,131 [INFO ] [calibration.py:computeMVCE:154]	 
        Computed MVCE 0.7669152617454529 for Beta Mean * 0.5 - Zero Imputation in 150.371905406937 seconds with 10 buckets:
        * Non-zero buckets: tensor([3, 4, 5, 6, 7, 8, 9])
        * Final bucket sizes: tensor([  10,   81,  462,  468,  603, 1075, 7301], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([0.3689, 0.4646, 0.5477, 0.6533, 0.7515, 0.8549, 0.9803])
        * Final bucket consistencies: tensor([0.0000, 0.1481, 0.0996, 0.1047, 0.1526, 0.1367, 0.1526])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([1222,  778, 1190,  813, 1226,  771, 1090,  909, 1186,  815],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([ 122,    0, 5524,    0,    1,    0, 4337,    4,   11,    1],
       dtype=torch.int32)
        * Average consistency: 0.1459999978542328
    
2025-05-16 02:06:39,132 [INFO ] [calibration.py:__call__:249]	 Finished running Beta Mean * 0.5 - Zero Imputation missing block-dropout in zero-one in 569.6863739537075 seconds
2025-05-16 02:06:40,302 [INFO ] [calibration.py:computeMVCE:154]	 
        Computed MVCE 0.3898508846759796 for Beta Mean * 0.99 - Zero Imputation in 146.4650843590498 seconds with 10 buckets:
        * Non-zero buckets: tensor([3, 4, 5, 6, 7, 8, 9])
        * Final bucket sizes: tensor([  89, 2148, 6999,  666,   32,   19,   47], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([0.3933, 0.4480, 0.5635, 0.6096, 0.7472, 0.8516, 0.9603])
        * Final bucket consistencies: tensor([0.1236, 0.1215, 0.1560, 0.1712, 0.5312, 0.5263, 0.6170])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([1222,  778, 1190,  813, 1226,  771, 1090,  909, 1186,  815],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([1395,    0, 5011,    0,    0,    0, 3591,    0,    3,    0],
       dtype=torch.int32)
        * Average consistency: 0.1534000039100647
    
2025-05-16 02:06:40,302 [INFO ] [calibration.py:__call__:249]	 Finished running Beta Mean * 0.99 - Zero Imputation missing block-dropout in zero-one in 570.8561619836837 seconds
2025-05-16 02:06:40,302 [INFO ] [mvce_dataset.py:<module>:222]	 Finished running 3/3 experiments.
2025-05-16 02:06:40,303 [INFO ] [mvce_dataset.py:<module>:227]	 Saving results to ./results/mvce-starcraft/starcraft-20250516-015706.csv
2025-05-16 02:06:40,304 [INFO ] [mvce_dataset.py:<module>:237]	 Finished saving results
