2025-05-23 04:25:36,993 [INFO ] [logger.py:dumpArgs:73]	 Starting with arguments
{"name": "starcraft", "dataset": {"path": "../../datasets/starcraftimage", "image_format": "cifar10", "image_size": 224, "sensor_size": 56}, "cache_directory": null, "output": "./results/mvce-starcraft/", "classifier": "./models/starcraft-cifar10/starcraft-20250515-150053.pklz", "classifier_feature": null, "calibration_map": "./results/calibration/starcraft/zero-one.csv", "mask": null, "drop": {"name": "block-dropout"}, "threads": 4, "generator_samples": [], "decision_samples": 1000, "cuda_index": 0, "zero_imputation": true, "zero_variance": false, "beta_variance_scales": [0.5, 0.99], "action_spaces": [{"name": "zero-one"}], "class_count": 1, "batch_size": 100, "batch_mean_imputation": [], "buckets": 10, "trials": 4, "seed": 1337, "verbose": 1}
2025-05-23 04:25:37,019 [INFO ] [util.py:selectDevice:88]	 Using cuda:0 for tensor calculations
2025-05-23 04:25:37,140 [INFO ] [mvce_dataset.py:<module>:93]	 Running on cuda:0
2025-05-23 04:25:37,386 [INFO ] [loader.py:getDatasetSplits:24]	 Loading StarCraft dataset from ../../datasets/starcraftimage
2025-05-23 04:25:37,386 [INFO ] [starcraft.py:createStarCraftDataset:96]	 Using 10 StarCraft targets: ['Acolyte LE at Beginning', 'Acolyte LE at End', 'Abyssal Reef LE at Beginning', 'Abyssal Reef LE at End', 'Ascension to Aiur LE at Beginning', 'Ascension to Aiur LE at End', 'Mech Depot LE at Beginning', 'Mech Depot LE at End', 'Odyssey LE at Beginning', 'Odyssey LE at End']
2025-05-23 04:25:37,420 [INFO ] [baselines.py:getTransform:24]	 Resizing images from 32 to 224
2025-05-23 04:25:38,493 [INFO ] [torch_utils.py:split:72]	 Split training from 50000 into 35000 training images and 15000 validation images. Found 10000 testing images.
2025-05-23 04:25:38,493 [INFO ] [mvce_dataset.py:<module>:106]	 Using dataset starcraft with 10000 test samples and 10 classes
2025-05-23 04:25:38,495 [INFO ] [mvce_dataset.py:<module>:119]	 Found calibration map at ./results/calibration/starcraft/zero-one.csv: {'Beta Mean * 0.5 - Zero Imputation': 7.5, 'Beta Mean * 0.99 - Zero Imputation': 10.0, 'Dirichlet Network': 0.5}
2025-05-23 04:25:38,495 [INFO ] [mvce_dataset.py:<module>:144]	 Setting classifier activation function to softmax for multiclass
2025-05-23 04:25:38,495 [INFO ] [mvce_dataset.py:<module>:169]	 Including baseline imputators with zero imputation.
2025-05-23 04:25:38,495 [INFO ] [mvce_dataset.py:<module>:181]	 Running all baseline imputators with [0.5, 0.99] scaled beta max variance.
2025-05-23 04:25:38,495 [INFO ] [mvce_dataset.py:<module>:197]	 Adding 2 methods.
2025-05-23 04:25:38,495 [INFO ] [mvce_dataset.py:<module>:209]	 Using randon dropping with arguments {'name': 'block-dropout'}
2025-05-23 04:25:38,495 [INFO ] [mvce_dataset.py:<module>:220]	 Considering action space zero-one
2025-05-23 04:25:38,495 [INFO ] [threading_utils.py:distributeTasks:91]	 Running 2 tasks each on their own thread
2025-05-23 04:25:38,496 [INFO ] [calibration.py:__call__:310]	 Started running Beta Mean * 0.5 - Zero Imputation missing block-dropout in zero-one
2025-05-23 04:25:38,496 [INFO ] [calibration.py:__call__:310]	 Started running Beta Mean * 0.99 - Zero Imputation missing block-dropout in zero-one
2025-05-23 04:27:20,842 [INFO ] [calibration.py:computeMVCE:153]	 
        Computed MVCE 0.0002983686572406441 for Beta Mean * 0.5 - Zero Imputation in 102.3442119108513 seconds with 10 buckets:
        * Non-zero buckets: tensor([6, 7, 8, 9])
        * Final bucket sizes: tensor([   4,    2,    4, 9990], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([0.6438, 0.7600, 0.8620, 0.9996])
        * Final bucket consistencies: tensor([0.7500, 0.5000, 0.5000, 0.9997])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([   0,    0,    9,    0,    0,    0, 9991,    0,    0,    0],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([   0,    0,    4,    0,    0,    0, 9996,    0,    0,    0],
       dtype=torch.int32)
        * Average consistency: 0.9993000030517578
    
2025-05-23 04:27:21,930 [INFO ] [calibration.py:computeMVCE:153]	 
        Computed MVCE 0.014178328216075897 for Beta Mean * 0.99 - Zero Imputation in 103.43070835527033 seconds with 10 buckets:
        * Non-zero buckets: tensor([5, 6, 7, 8, 9])
        * Final bucket sizes: tensor([   1,    8,   18,   66, 9907], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([0.5420, 0.6599, 0.7649, 0.8642, 0.9865])
        * Final bucket consistencies: tensor([1.0000, 1.0000, 0.9444, 0.9848, 0.9994])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([   0,    0,    9,    0,    0,    0, 9991,    0,    0,    0],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([   0,    0,    1,    0,    0,    0, 9999,    0,    0,    0],
       dtype=torch.int32)
        * Average consistency: 0.9991999864578247
    
2025-05-23 04:29:03,448 [INFO ] [calibration.py:computeMVCE:153]	 
        Computed MVCE 0.0009841901483014226 for Beta Mean * 0.5 - Zero Imputation in 102.60383902490139 seconds with 10 buckets:
        * Non-zero buckets: tensor([5, 6, 7, 8, 9])
        * Final bucket sizes: tensor([   4,    2,    2,    5, 9987], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([0.5330, 0.6280, 0.7650, 0.8766, 0.9997])
        * Final bucket consistencies: tensor([0.0000, 1.0000, 1.0000, 1.0000, 0.9991])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([   0,    0,    9,    0,    0,    0, 9991,    0,    0,    0],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([   0,    0,    4,    0,    0,    0, 9996,    0,    0,    0],
       dtype=torch.int32)
        * Average consistency: 0.9987000226974487
    
2025-05-23 04:29:05,583 [INFO ] [calibration.py:computeMVCE:153]	 
        Computed MVCE 0.013728189282119274 for Beta Mean * 0.99 - Zero Imputation in 103.65145364310592 seconds with 10 buckets:
        * Non-zero buckets: tensor([5, 6, 7, 8, 9])
        * Final bucket sizes: tensor([   9,    5,   10,   50, 9926], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([0.5336, 0.6598, 0.7526, 0.8590, 0.9865])
        * Final bucket consistencies: tensor([0.4444, 0.8000, 0.9000, 0.9600, 0.9995])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([   0,    0,    9,    0,    0,    0, 9991,    0,    0,    0],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([   0,    0,    7,    0,    0,    0, 9993,    0,    0,    0],
       dtype=torch.int32)
        * Average consistency: 0.9986000061035156
    
2025-05-23 04:30:45,389 [INFO ] [calibration.py:computeMVCE:153]	 
        Computed MVCE 0.0005276626325212419 for Beta Mean * 0.5 - Zero Imputation in 101.93804414384067 seconds with 10 buckets:
        * Non-zero buckets: tensor([5, 6, 7, 8, 9])
        * Final bucket sizes: tensor([   2,    3,    2,    7, 9986], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([0.5735, 0.6743, 0.7665, 0.8501, 0.9997])
        * Final bucket consistencies: tensor([1.0000, 0.3333, 0.5000, 0.5714, 0.9996])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([   0,    0,    9,    0,    0,    0, 9991,    0,    0,    0],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([   0,    0,    5,    0,    0,    0, 9995,    0,    0,    0],
       dtype=torch.int32)
        * Average consistency: 0.9990000128746033
    
2025-05-23 04:30:47,812 [INFO ] [calibration.py:computeMVCE:153]	 
        Computed MVCE 0.013885851949453354 for Beta Mean * 0.99 - Zero Imputation in 102.22676111012697 seconds with 10 buckets:
        * Non-zero buckets: tensor([5, 6, 7, 8, 9])
        * Final bucket sizes: tensor([   7,    6,   15,   58, 9914], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([0.5279, 0.6615, 0.7526, 0.8610, 0.9867])
        * Final bucket consistencies: tensor([0.5714, 0.3333, 0.8667, 1.0000, 0.9995])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([   0,    0,    9,    0,    0,    0, 9991,    0,    0,    0],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([   0,    0,    7,    0,    0,    0, 9993,    0,    0,    0],
       dtype=torch.int32)
        * Average consistency: 0.9986000061035156
    
2025-05-23 04:32:23,462 [INFO ] [calibration.py:computeMVCE:153]	 
        Computed MVCE 0.0006677687633782625 for Beta Mean * 0.5 - Zero Imputation in 98.07098112814128 seconds with 10 buckets:
        * Non-zero buckets: tensor([5, 6, 7, 8, 9])
        * Final bucket sizes: tensor([   3,    1,    2,    3, 9991], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([0.5757, 0.6470, 0.7645, 0.8697, 0.9997])
        * Final bucket consistencies: tensor([0.6667, 0.0000, 1.0000, 1.0000, 0.9992])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([   0,    0,    9,    0,    0,    0, 9991,    0,    0,    0],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([   0,    0,    1,    0,    0,    0, 9999,    0,    0,    0],
       dtype=torch.int32)
        * Average consistency: 0.9990000128746033
    
2025-05-23 04:32:23,478 [INFO ] [calibration.py:__call__:336]	 Finished running Beta Mean * 0.5 - Zero Imputation missing block-dropout in zero-one in 404.982353891246 seconds
2025-05-23 04:32:26,882 [INFO ] [calibration.py:computeMVCE:153]	 
        Computed MVCE 0.01404520869255066 for Beta Mean * 0.99 - Zero Imputation in 99.06842458155006 seconds with 10 buckets:
        * Non-zero buckets: tensor([5, 6, 7, 8, 9])
        * Final bucket sizes: tensor([   5,    8,   18,   50, 9919], dtype=torch.int32) totaling 10000 samples
        * Final bucket confidences: tensor([0.5462, 0.6484, 0.7751, 0.8611, 0.9866])
        * Final bucket consistencies: tensor([0.8000, 0.8750, 0.8889, 0.9600, 0.9997])
        * Actions: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
        * Best Action Counts: tensor([   0,    0,    9,    0,    0,    0, 9991,    0,    0,    0],
       dtype=torch.int32)
        * Prediction Action Counts: tensor([   0,    0,    4,    0,    0,    0, 9996,    0,    0,    0],
       dtype=torch.int32)
        * Average consistency: 0.9991000294685364
    
2025-05-23 04:32:26,882 [INFO ] [calibration.py:__call__:336]	 Finished running Beta Mean * 0.99 - Zero Imputation missing block-dropout in zero-one in 408.38624329306185 seconds
2025-05-23 04:32:26,882 [INFO ] [mvce_dataset.py:<module>:234]	 Finished running 2/2 experiments.
2025-05-23 04:32:26,883 [INFO ] [mvce_dataset.py:<module>:239]	 Saving results to ./results/mvce-starcraft/starcraft-20250523-042536.csv
2025-05-23 04:32:26,883 [INFO ] [mvce_dataset.py:<module>:249]	 Finished saving results
