
=== Start adding workers ===
{'Type': 'Setup', 'Dataset': 'cifar10', 'data_dir': '/home/he/dero/datasets/', 'train': True, 'download': True, 'batch_size': 64, 'shuffle': None, 'sampler': 'DistributedSampler(num_replicas=10,rank=0,shuffle=True)'}
=> Add worker SGDMWorker(index=0, momentum=0.9)
{'Type': 'Setup', 'Dataset': 'cifar10', 'data_dir': '/home/he/dero/datasets/', 'train': True, 'download': True, 'batch_size': 64, 'shuffle': None, 'sampler': 'DistributedSampler(num_replicas=10,rank=1,shuffle=True)'}
=> Add worker SGDMWorker(index=1, momentum=0.9)
{'Type': 'Setup', 'Dataset': 'cifar10', 'data_dir': '/home/he/dero/datasets/', 'train': True, 'download': True, 'batch_size': 64, 'shuffle': None, 'sampler': 'DistributedSampler(num_replicas=10,rank=2,shuffle=True)'}
=> Add worker SGDMWorker(index=2, momentum=0.9)
{'Type': 'Setup', 'Dataset': 'cifar10', 'data_dir': '/home/he/dero/datasets/', 'train': True, 'download': True, 'batch_size': 64, 'shuffle': None, 'sampler': 'DistributedSampler(num_replicas=10,rank=3,shuffle=True)'}
=> Add worker SGDMWorker(index=3, momentum=0.9)
{'Type': 'Setup', 'Dataset': 'cifar10', 'data_dir': '/home/he/dero/datasets/', 'train': True, 'download': True, 'batch_size': 64, 'shuffle': None, 'sampler': 'DistributedSampler(num_replicas=10,rank=4,shuffle=True)'}
=> Add worker SGDMWorker(index=4, momentum=0.9)
{'Type': 'Setup', 'Dataset': 'cifar10', 'data_dir': '/home/he/dero/datasets/', 'train': True, 'download': True, 'batch_size': 64, 'shuffle': None, 'sampler': 'DistributedSampler(num_replicas=10,rank=5,shuffle=True)'}
=> Add worker SGDMWorker(index=5, momentum=0.9)
{'Type': 'Setup', 'Dataset': 'cifar10', 'data_dir': '/home/he/dero/datasets/', 'train': True, 'download': True, 'batch_size': 64, 'shuffle': None, 'sampler': 'DistributedSampler(num_replicas=10,rank=6,shuffle=True)'}
=> Add worker SGDMWorker(index=6, momentum=0.9)
{'Type': 'Setup', 'Dataset': 'cifar10', 'data_dir': '/home/he/dero/datasets/', 'train': True, 'download': True, 'batch_size': 64, 'shuffle': None, 'sampler': 'DistributedSampler(num_replicas=10,rank=7,shuffle=True)'}
=> Add worker SGDMWorker(index=7, momentum=0.9)
{'Type': 'Setup', 'Dataset': 'cifar10', 'data_dir': '/home/he/dero/datasets/', 'train': True, 'download': True, 'batch_size': 64, 'shuffle': None, 'sampler': 'DistributedSampler(num_replicas=10,rank=8,shuffle=True)'}
=> Add worker SGDMWorker(index=8, momentum=0.9)
{'Type': 'Setup', 'Dataset': 'cifar10', 'data_dir': '/home/he/dero/datasets/', 'train': True, 'download': True, 'batch_size': 64, 'shuffle': None, 'sampler': 'DistributedSampler(num_replicas=10,rank=9,shuffle=True)'}
=> Add worker SGDMWorker(index=9, momentum=0.9)

=== Start adding graph ===
<codes.graph_utils.Dumbbell object at 0x7fa913346730>

{'Type': 'Setup', 'Dataset': 'cifar10', 'data_dir': '/home/he/dero/datasets/', 'train': False, 'download': True, 'batch_size': 128, 'shuffle': False, 'sampler': None}
Train epoch 1
[E 1B0  |    640/50000 (  1%) ] Loss: 2.3038 top1=  9.3750

=== Peeking data label distribution E1B0 ===
Worker 0 has targets: tensor([1, 4, 1, 1, 1], device='cuda:0')
Worker 1 has targets: tensor([1, 4, 2, 2, 3], device='cuda:0')
Worker 2 has targets: tensor([1, 2, 4, 2, 3], device='cuda:0')
Worker 3 has targets: tensor([1, 0, 4, 3, 0], device='cuda:0')
Worker 4 has targets: tensor([4, 0, 0, 4, 0], device='cuda:0')
Worker 5 has targets: tensor([9, 6, 8, 9, 9], device='cuda:0')
Worker 6 has targets: tensor([6, 9, 5, 9, 9], device='cuda:0')
Worker 7 has targets: tensor([8, 5, 8, 9, 5], device='cuda:0')
Worker 8 has targets: tensor([9, 7, 5, 5, 8], device='cuda:0')
Worker 9 has targets: tensor([8, 9, 7, 5, 7], device='cuda:0')



=== Log mixing matrix @ E1B0 ===
[[0.167 0.167 0.167 0.167 0.167 0.    0.    0.    0.    0.167]
 [0.167 0.333 0.167 0.167 0.167 0.    0.    0.    0.    0.   ]
 [0.167 0.167 0.333 0.167 0.167 0.    0.    0.    0.    0.   ]
 [0.167 0.167 0.167 0.333 0.167 0.    0.    0.    0.    0.   ]
 [0.167 0.167 0.167 0.167 0.333 0.    0.    0.    0.    0.   ]
 [0.    0.    0.    0.    0.    0.333 0.167 0.167 0.167 0.167]
 [0.    0.    0.    0.    0.    0.167 0.333 0.167 0.167 0.167]
 [0.    0.    0.    0.    0.    0.167 0.167 0.333 0.167 0.167]
 [0.    0.    0.    0.    0.    0.167 0.167 0.167 0.333 0.167]
 [0.167 0.    0.    0.    0.    0.167 0.167 0.167 0.167 0.167]]


[E 1B10 |   7040/50000 ( 14%) ] Loss: 1.9887 top1= 19.3750
[E 1B20 |  13440/50000 ( 27%) ] Loss: 1.8007 top1= 20.6250
[E 1B30 |  19840/50000 ( 40%) ] Loss: 1.6460 top1= 22.0312
[E 1B40 |  26240/50000 ( 52%) ] Loss: 1.6626 top1= 22.3438
[E 1B50 |  32640/50000 ( 65%) ] Loss: 1.5935 top1= 26.0938
[E 1B60 |  39040/50000 ( 78%) ] Loss: 1.5521 top1= 27.8125
[E 1B70 |  45440/50000 ( 91%) ] Loss: 1.5185 top1= 31.4062

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3039 top1= 10.1462


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.3194 top1= 15.8353


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.1777 top1= 18.6599

Train epoch 2
[E 2B0  |    640/50000 (  1%) ] Loss: 1.4924 top1= 34.3750
[E 2B10 |   7040/50000 ( 14%) ] Loss: 1.4538 top1= 33.7500
[E 2B20 |  13440/50000 ( 27%) ] Loss: 1.6049 top1= 29.2188
[E 2B30 |  19840/50000 ( 40%) ] Loss: 1.4645 top1= 32.1875
[E 2B40 |  26240/50000 ( 52%) ] Loss: 1.4006 top1= 40.0000
[E 2B50 |  32640/50000 ( 65%) ] Loss: 1.3876 top1= 34.5312
[E 2B60 |  39040/50000 ( 78%) ] Loss: 1.4788 top1= 32.9688
[E 2B70 |  45440/50000 ( 91%) ] Loss: 1.6627 top1= 26.8750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3059 top1= 10.0160


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.0690 top1= 14.0925


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=3.6189 top1= 16.9772

Train epoch 3
[E 3B0  |    640/50000 (  1%) ] Loss: 1.5619 top1= 31.8750
[E 3B10 |   7040/50000 ( 14%) ] Loss: 1.5363 top1= 31.4062
[E 3B20 |  13440/50000 ( 27%) ] Loss: 1.4980 top1= 30.1562
[E 3B30 |  19840/50000 ( 40%) ] Loss: 1.4775 top1= 35.1562
[E 3B40 |  26240/50000 ( 52%) ] Loss: 1.5195 top1= 33.2812
[E 3B50 |  32640/50000 ( 65%) ] Loss: 1.4384 top1= 34.5312
[E 3B60 |  39040/50000 ( 78%) ] Loss: 1.5471 top1= 32.5000
[E 3B70 |  45440/50000 ( 91%) ] Loss: 1.4737 top1= 30.4688

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3248 top1= 10.0160


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=3.8184 top1= 12.9507


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.2371 top1= 21.0337

Train epoch 4
[E 4B0  |    640/50000 (  1%) ] Loss: 1.5091 top1= 31.8750
[E 4B10 |   7040/50000 ( 14%) ] Loss: 1.5035 top1= 30.4688
[E 4B20 |  13440/50000 ( 27%) ] Loss: 1.4676 top1= 32.0312
[E 4B30 |  19840/50000 ( 40%) ] Loss: 1.4158 top1= 33.2812
[E 4B40 |  26240/50000 ( 52%) ] Loss: 1.4392 top1= 33.9062
[E 4B50 |  32640/50000 ( 65%) ] Loss: 1.4384 top1= 34.2188
[E 4B60 |  39040/50000 ( 78%) ] Loss: 1.4037 top1= 40.6250
[E 4B70 |  45440/50000 ( 91%) ] Loss: 1.3393 top1= 37.9688

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3274 top1=  9.9359


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.7852 top1= 15.4347


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.4680 top1= 26.9832

Train epoch 5
[E 5B0  |    640/50000 (  1%) ] Loss: 1.3926 top1= 41.0938
[E 5B10 |   7040/50000 ( 14%) ] Loss: 1.3244 top1= 43.2812
[E 5B20 |  13440/50000 ( 27%) ] Loss: 1.3237 top1= 44.8438
[E 5B30 |  19840/50000 ( 40%) ] Loss: 1.2881 top1= 43.9062
[E 5B40 |  26240/50000 ( 52%) ] Loss: 1.2597 top1= 47.6562
[E 5B50 |  32640/50000 ( 65%) ] Loss: 1.3240 top1= 45.7812
[E 5B60 |  39040/50000 ( 78%) ] Loss: 1.2740 top1= 46.8750
[E 5B70 |  45440/50000 ( 91%) ] Loss: 1.1792 top1= 50.9375

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3233 top1= 10.0060


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.8532 top1= 20.4127


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.2271 top1= 30.3185

Train epoch 6
[E 6B0  |    640/50000 (  1%) ] Loss: 1.2392 top1= 50.1562
[E 6B10 |   7040/50000 ( 14%) ] Loss: 1.1706 top1= 54.0625
[E 6B20 |  13440/50000 ( 27%) ] Loss: 1.1920 top1= 52.1875
[E 6B30 |  19840/50000 ( 40%) ] Loss: 1.1781 top1= 51.5625
[E 6B40 |  26240/50000 ( 52%) ] Loss: 1.2251 top1= 47.9688
[E 6B50 |  32640/50000 ( 65%) ] Loss: 1.2467 top1= 48.4375
[E 6B60 |  39040/50000 ( 78%) ] Loss: 1.2500 top1= 49.0625
[E 6B70 |  45440/50000 ( 91%) ] Loss: 1.1516 top1= 52.6562

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3187 top1= 10.0160


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.4771 top1= 19.0505


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.1358 top1= 32.2817

Train epoch 7
[E 7B0  |    640/50000 (  1%) ] Loss: 1.1980 top1= 48.1250
[E 7B10 |   7040/50000 ( 14%) ] Loss: 1.0981 top1= 55.3125
[E 7B20 |  13440/50000 ( 27%) ] Loss: 1.1538 top1= 54.5312
[E 7B30 |  19840/50000 ( 40%) ] Loss: 1.0844 top1= 57.1875
[E 7B40 |  26240/50000 ( 52%) ] Loss: 1.0710 top1= 56.8750
[E 7B50 |  32640/50000 ( 65%) ] Loss: 1.1386 top1= 55.7812
[E 7B60 |  39040/50000 ( 78%) ] Loss: 1.0637 top1= 59.3750
[E 7B70 |  45440/50000 ( 91%) ] Loss: 1.0405 top1= 56.0938

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3249 top1= 10.0160


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.8446 top1= 24.2788


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.7609 top1= 33.3834

Train epoch 8
[E 8B0  |    640/50000 (  1%) ] Loss: 1.1255 top1= 55.7812
[E 8B10 |   7040/50000 ( 14%) ] Loss: 1.1160 top1= 58.2812
[E 8B20 |  13440/50000 ( 27%) ] Loss: 1.0565 top1= 56.2500
[E 8B30 |  19840/50000 ( 40%) ] Loss: 1.0225 top1= 57.8125
[E 8B40 |  26240/50000 ( 52%) ] Loss: 1.0467 top1= 57.5000
[E 8B50 |  32640/50000 ( 65%) ] Loss: 1.0213 top1= 58.5938
[E 8B60 |  39040/50000 ( 78%) ] Loss: 1.0203 top1= 59.0625
[E 8B70 |  45440/50000 ( 91%) ] Loss: 0.9845 top1= 59.0625

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3277 top1= 10.0160


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.9849 top1= 27.2236


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.9857 top1= 34.9559

Train epoch 9
[E 9B0  |    640/50000 (  1%) ] Loss: 1.0076 top1= 61.4062
[E 9B10 |   7040/50000 ( 14%) ] Loss: 1.0132 top1= 60.4688
[E 9B20 |  13440/50000 ( 27%) ] Loss: 0.9986 top1= 59.5312
[E 9B30 |  19840/50000 ( 40%) ] Loss: 0.9377 top1= 62.0312
[E 9B40 |  26240/50000 ( 52%) ] Loss: 0.9269 top1= 62.3438
[E 9B50 |  32640/50000 ( 65%) ] Loss: 0.9022 top1= 63.2812
[E 9B60 |  39040/50000 ( 78%) ] Loss: 1.0032 top1= 59.2188
[E 9B70 |  45440/50000 ( 91%) ] Loss: 0.9283 top1= 61.8750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3339 top1= 10.0160


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.8938 top1= 30.2384


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.7697 top1= 34.5753

Train epoch 10
[E10B0  |    640/50000 (  1%) ] Loss: 0.9451 top1= 62.6562
[E10B10 |   7040/50000 ( 14%) ] Loss: 0.9961 top1= 62.1875
[E10B20 |  13440/50000 ( 27%) ] Loss: 0.9169 top1= 64.0625
[E10B30 |  19840/50000 ( 40%) ] Loss: 0.8831 top1= 66.2500
[E10B40 |  26240/50000 ( 52%) ] Loss: 0.8503 top1= 66.8750
[E10B50 |  32640/50000 ( 65%) ] Loss: 0.8905 top1= 64.5312
[E10B60 |  39040/50000 ( 78%) ] Loss: 0.8681 top1= 65.9375
[E10B70 |  45440/50000 ( 91%) ] Loss: 0.8706 top1= 64.8438

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3356 top1= 10.0160


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.1128 top1= 29.8878


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.5260 top1= 33.9844

Train epoch 11
[E11B0  |    640/50000 (  1%) ] Loss: 0.9772 top1= 60.7812
[E11B10 |   7040/50000 ( 14%) ] Loss: 0.9003 top1= 66.5625
[E11B20 |  13440/50000 ( 27%) ] Loss: 0.8471 top1= 67.6562
[E11B30 |  19840/50000 ( 40%) ] Loss: 0.8036 top1= 67.9688
[E11B40 |  26240/50000 ( 52%) ] Loss: 0.8415 top1= 65.7812
[E11B50 |  32640/50000 ( 65%) ] Loss: 0.8293 top1= 68.7500
[E11B60 |  39040/50000 ( 78%) ] Loss: 0.8460 top1= 68.9062
[E11B70 |  45440/50000 ( 91%) ] Loss: 0.8172 top1= 67.3438

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3448 top1= 10.0160


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.4138 top1= 30.6390


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.7608 top1= 37.0893

Train epoch 12
[E12B0  |    640/50000 (  1%) ] Loss: 0.9073 top1= 64.0625
[E12B10 |   7040/50000 ( 14%) ] Loss: 0.8533 top1= 68.1250
[E12B20 |  13440/50000 ( 27%) ] Loss: 0.7772 top1= 68.2812
[E12B30 |  19840/50000 ( 40%) ] Loss: 0.8146 top1= 70.0000
[E12B40 |  26240/50000 ( 52%) ] Loss: 0.7517 top1= 71.2500
[E12B50 |  32640/50000 ( 65%) ] Loss: 0.8055 top1= 69.5312
[E12B60 |  39040/50000 ( 78%) ] Loss: 0.8242 top1= 68.4375
[E12B70 |  45440/50000 ( 91%) ] Loss: 0.7756 top1= 69.6875

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3561 top1= 10.0160


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.2559 top1= 31.7208


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.9542 top1= 38.6819

Train epoch 13
[E13B0  |    640/50000 (  1%) ] Loss: 0.7870 top1= 68.2812
[E13B10 |   7040/50000 ( 14%) ] Loss: 0.7634 top1= 71.2500
[E13B20 |  13440/50000 ( 27%) ] Loss: 0.7591 top1= 68.5938
[E13B30 |  19840/50000 ( 40%) ] Loss: 0.7571 top1= 71.4062
[E13B40 |  26240/50000 ( 52%) ] Loss: 0.7500 top1= 72.3438
[E13B50 |  32640/50000 ( 65%) ] Loss: 0.8104 top1= 69.8438
[E13B60 |  39040/50000 ( 78%) ] Loss: 0.7199 top1= 70.9375
[E13B70 |  45440/50000 ( 91%) ] Loss: 0.7132 top1= 73.1250

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3584 top1= 10.0160


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.5467 top1= 32.2817


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.9892 top1= 38.7620

Train epoch 14
[E14B0  |    640/50000 (  1%) ] Loss: 0.8079 top1= 67.9688
[E14B10 |   7040/50000 ( 14%) ] Loss: 0.8129 top1= 68.2812
[E14B20 |  13440/50000 ( 27%) ] Loss: 0.7492 top1= 70.0000
[E14B30 |  19840/50000 ( 40%) ] Loss: 0.7738 top1= 69.6875
[E14B40 |  26240/50000 ( 52%) ] Loss: 0.7119 top1= 73.4375
[E14B50 |  32640/50000 ( 65%) ] Loss: 0.8007 top1= 70.3125
[E14B60 |  39040/50000 ( 78%) ] Loss: 0.7627 top1= 70.4688
[E14B70 |  45440/50000 ( 91%) ] Loss: 0.7078 top1= 70.4688

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3711 top1= 10.0160


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.9932 top1= 34.1947


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.8742 top1= 38.8221

Train epoch 15
[E15B0  |    640/50000 (  1%) ] Loss: 0.7127 top1= 74.6875
[E15B10 |   7040/50000 ( 14%) ] Loss: 0.6990 top1= 75.1562
[E15B20 |  13440/50000 ( 27%) ] Loss: 0.7150 top1= 73.2812
[E15B30 |  19840/50000 ( 40%) ] Loss: 0.7194 top1= 74.6875
[E15B40 |  26240/50000 ( 52%) ] Loss: 0.6697 top1= 75.9375
[E15B50 |  32640/50000 ( 65%) ] Loss: 0.7474 top1= 70.7812
[E15B60 |  39040/50000 ( 78%) ] Loss: 0.6871 top1= 75.7812
[E15B70 |  45440/50000 ( 91%) ] Loss: 0.6481 top1= 75.0000

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3667 top1= 10.0160


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.0776 top1= 34.6154


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.7574 top1= 39.9139

Train epoch 16
[E16B0  |    640/50000 (  1%) ] Loss: 0.6981 top1= 71.7188
[E16B10 |   7040/50000 ( 14%) ] Loss: 0.7199 top1= 74.6875
[E16B20 |  13440/50000 ( 27%) ] Loss: 0.6658 top1= 74.8438
[E16B30 |  19840/50000 ( 40%) ] Loss: 0.6739 top1= 75.4688
[E16B40 |  26240/50000 ( 52%) ] Loss: 0.6927 top1= 74.3750
[E16B50 |  32640/50000 ( 65%) ] Loss: 0.6714 top1= 75.9375
[E16B60 |  39040/50000 ( 78%) ] Loss: 0.6389 top1= 77.5000
[E16B70 |  45440/50000 ( 91%) ] Loss: 0.6935 top1= 74.2188

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3588 top1= 10.0160


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.9679 top1= 35.3566


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.8731 top1= 39.4030

Train epoch 17
[E17B0  |    640/50000 (  1%) ] Loss: 0.6759 top1= 73.9062
[E17B10 |   7040/50000 ( 14%) ] Loss: 0.6966 top1= 72.5000
[E17B20 |  13440/50000 ( 27%) ] Loss: 0.5944 top1= 77.5000
[E17B30 |  19840/50000 ( 40%) ] Loss: 0.6256 top1= 76.8750
[E17B40 |  26240/50000 ( 52%) ] Loss: 0.6453 top1= 74.5312
[E17B50 |  32640/50000 ( 65%) ] Loss: 0.6443 top1= 76.4062
[E17B60 |  39040/50000 ( 78%) ] Loss: 0.6274 top1= 75.9375
[E17B70 |  45440/50000 ( 91%) ] Loss: 0.7057 top1= 73.1250

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3646 top1= 10.0160


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.9346 top1= 36.1779


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.9354 top1= 38.7720

Train epoch 18
[E18B0  |    640/50000 (  1%) ] Loss: 0.7110 top1= 75.0000
[E18B10 |   7040/50000 ( 14%) ] Loss: 0.6370 top1= 76.0938
[E18B20 |  13440/50000 ( 27%) ] Loss: 0.6339 top1= 77.9688
[E18B30 |  19840/50000 ( 40%) ] Loss: 0.5798 top1= 77.3438
[E18B40 |  26240/50000 ( 52%) ] Loss: 0.5777 top1= 78.2812
[E18B50 |  32640/50000 ( 65%) ] Loss: 0.6180 top1= 77.3438
[E18B60 |  39040/50000 ( 78%) ] Loss: 0.6143 top1= 77.5000
[E18B70 |  45440/50000 ( 91%) ] Loss: 0.5927 top1= 77.6562

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3715 top1= 10.0160


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.0985 top1= 36.1478


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.9062 top1= 41.0156

Train epoch 19
[E19B0  |    640/50000 (  1%) ] Loss: 0.5905 top1= 78.9062
[E19B10 |   7040/50000 ( 14%) ] Loss: 0.6223 top1= 78.2812
[E19B20 |  13440/50000 ( 27%) ] Loss: 0.5705 top1= 78.9062
[E19B30 |  19840/50000 ( 40%) ] Loss: 0.5431 top1= 80.4688
[E19B40 |  26240/50000 ( 52%) ] Loss: 0.5639 top1= 77.1875
[E19B50 |  32640/50000 ( 65%) ] Loss: 0.6015 top1= 77.3438
[E19B60 |  39040/50000 ( 78%) ] Loss: 0.5849 top1= 78.2812
[E19B70 |  45440/50000 ( 91%) ] Loss: 0.5241 top1= 80.1562

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3716 top1= 10.0160


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.1990 top1= 37.0292


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.9876 top1= 40.9355

Train epoch 20
[E20B0  |    640/50000 (  1%) ] Loss: 0.5631 top1= 77.8125
[E20B10 |   7040/50000 ( 14%) ] Loss: 0.6147 top1= 77.9688
[E20B20 |  13440/50000 ( 27%) ] Loss: 0.5173 top1= 82.0312
[E20B30 |  19840/50000 ( 40%) ] Loss: 0.5680 top1= 79.0625
[E20B40 |  26240/50000 ( 52%) ] Loss: 0.5196 top1= 80.0000
[E20B50 |  32640/50000 ( 65%) ] Loss: 0.4995 top1= 81.4062
[E20B60 |  39040/50000 ( 78%) ] Loss: 0.5750 top1= 79.6875
[E20B70 |  45440/50000 ( 91%) ] Loss: 0.5014 top1= 80.3125

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3701 top1= 10.0160


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.9838 top1= 37.3598


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.7818 top1= 42.1174

Train epoch 21
[E21B0  |    640/50000 (  1%) ] Loss: 0.5416 top1= 78.9062
[E21B10 |   7040/50000 ( 14%) ] Loss: 0.6083 top1= 78.5938
[E21B20 |  13440/50000 ( 27%) ] Loss: 0.4846 top1= 82.3438
[E21B30 |  19840/50000 ( 40%) ] Loss: 0.4959 top1= 80.4688
[E21B40 |  26240/50000 ( 52%) ] Loss: 0.5041 top1= 82.1875
[E21B50 |  32640/50000 ( 65%) ] Loss: 0.5606 top1= 79.3750
[E21B60 |  39040/50000 ( 78%) ] Loss: 0.5588 top1= 80.4688
[E21B70 |  45440/50000 ( 91%) ] Loss: 0.4856 top1= 81.8750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3590 top1= 10.0160


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.8861 top1= 37.1695


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.6513 top1= 41.0657

Train epoch 22
[E22B0  |    640/50000 (  1%) ] Loss: 0.5917 top1= 77.5000
[E22B10 |   7040/50000 ( 14%) ] Loss: 0.5886 top1= 78.1250
[E22B20 |  13440/50000 ( 27%) ] Loss: 0.4900 top1= 80.7812
[E22B30 |  19840/50000 ( 40%) ] Loss: 0.5074 top1= 81.7188
[E22B40 |  26240/50000 ( 52%) ] Loss: 0.5446 top1= 79.3750
[E22B50 |  32640/50000 ( 65%) ] Loss: 0.4708 top1= 83.2812
[E22B60 |  39040/50000 ( 78%) ] Loss: 0.5451 top1= 81.2500
[E22B70 |  45440/50000 ( 91%) ] Loss: 0.5695 top1= 78.7500

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3635 top1= 10.0160


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.9777 top1= 38.0909


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.9812 top1= 41.6366

Train epoch 23
[E23B0  |    640/50000 (  1%) ] Loss: 0.5060 top1= 81.7188
[E23B10 |   7040/50000 ( 14%) ] Loss: 0.5416 top1= 78.2812
[E23B20 |  13440/50000 ( 27%) ] Loss: 0.5069 top1= 81.7188
[E23B30 |  19840/50000 ( 40%) ] Loss: 0.4491 top1= 82.6562
[E23B40 |  26240/50000 ( 52%) ] Loss: 0.4999 top1= 81.2500
[E23B50 |  32640/50000 ( 65%) ] Loss: 0.5323 top1= 80.0000
[E23B60 |  39040/50000 ( 78%) ] Loss: 0.5127 top1= 81.2500
[E23B70 |  45440/50000 ( 91%) ] Loss: 0.4845 top1= 81.7188

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3626 top1= 10.0160


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.1500 top1= 38.1911


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.1190 top1= 41.9071

Train epoch 24
[E24B0  |    640/50000 (  1%) ] Loss: 0.4802 top1= 81.5625
[E24B10 |   7040/50000 ( 14%) ] Loss: 0.5300 top1= 81.4062
[E24B20 |  13440/50000 ( 27%) ] Loss: 0.4356 top1= 84.0625
[E24B30 |  19840/50000 ( 40%) ] Loss: 0.4767 top1= 81.4062
[E24B40 |  26240/50000 ( 52%) ] Loss: 0.5050 top1= 81.8750
[E24B50 |  32640/50000 ( 65%) ] Loss: 0.5313 top1= 79.3750
[E24B60 |  39040/50000 ( 78%) ] Loss: 0.5201 top1= 80.3125
[E24B70 |  45440/50000 ( 91%) ] Loss: 0.4151 top1= 84.8438

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3594 top1= 10.0160


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.1079 top1= 38.5417


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.0001 top1= 43.1991

Train epoch 25
[E25B0  |    640/50000 (  1%) ] Loss: 0.4846 top1= 80.6250
[E25B10 |   7040/50000 ( 14%) ] Loss: 0.4784 top1= 82.6562
[E25B20 |  13440/50000 ( 27%) ] Loss: 0.4346 top1= 83.2812
[E25B30 |  19840/50000 ( 40%) ] Loss: 0.4121 top1= 85.6250
[E25B40 |  26240/50000 ( 52%) ] Loss: 0.4610 top1= 82.1875
[E25B50 |  32640/50000 ( 65%) ] Loss: 0.5087 top1= 81.4062
[E25B60 |  39040/50000 ( 78%) ] Loss: 0.4476 top1= 83.1250
[E25B70 |  45440/50000 ( 91%) ] Loss: 0.4662 top1= 82.3438

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3452 top1= 10.0160


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.1217 top1= 38.9824


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.0349 top1= 43.6999

Train epoch 26
[E26B0  |    640/50000 (  1%) ] Loss: 0.4416 top1= 83.7500
[E26B10 |   7040/50000 ( 14%) ] Loss: 0.4690 top1= 83.2812
[E26B20 |  13440/50000 ( 27%) ] Loss: 0.4196 top1= 85.3125
[E26B30 |  19840/50000 ( 40%) ] Loss: 0.4200 top1= 83.7500
[E26B40 |  26240/50000 ( 52%) ] Loss: 0.4470 top1= 82.8125
[E26B50 |  32640/50000 ( 65%) ] Loss: 0.5225 top1= 80.9375
[E26B60 |  39040/50000 ( 78%) ] Loss: 0.5304 top1= 80.6250
[E26B70 |  45440/50000 ( 91%) ] Loss: 0.4191 top1= 85.0000

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3360 top1= 10.0160


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.3908 top1= 37.3698


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.7203 top1= 43.4295

Train epoch 27
[E27B0  |    640/50000 (  1%) ] Loss: 0.4804 top1= 81.0938
[E27B10 |   7040/50000 ( 14%) ] Loss: 0.4654 top1= 83.1250
[E27B20 |  13440/50000 ( 27%) ] Loss: 0.4123 top1= 82.5000
[E27B30 |  19840/50000 ( 40%) ] Loss: 0.4424 top1= 83.9062
[E27B40 |  26240/50000 ( 52%) ] Loss: 0.4668 top1= 83.4375
[E27B50 |  32640/50000 ( 65%) ] Loss: 0.4867 top1= 80.3125
[E27B60 |  39040/50000 ( 78%) ] Loss: 0.4755 top1= 84.0625
[E27B70 |  45440/50000 ( 91%) ] Loss: 0.4028 top1= 85.3125

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3235 top1= 10.0260


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.6859 top1= 39.2829


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.7297 top1= 43.8602

Train epoch 28
[E28B0  |    640/50000 (  1%) ] Loss: 0.4125 top1= 84.0625
[E28B10 |   7040/50000 ( 14%) ] Loss: 0.4189 top1= 85.3125
[E28B20 |  13440/50000 ( 27%) ] Loss: 0.3732 top1= 85.0000
[E28B30 |  19840/50000 ( 40%) ] Loss: 0.3968 top1= 85.1562
[E28B40 |  26240/50000 ( 52%) ] Loss: 0.4043 top1= 85.9375
[E28B50 |  32640/50000 ( 65%) ] Loss: 0.3808 top1= 85.9375
[E28B60 |  39040/50000 ( 78%) ] Loss: 0.4290 top1= 85.1562
[E28B70 |  45440/50000 ( 91%) ] Loss: 0.4450 top1= 82.0312

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3090 top1= 10.0461


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.2035 top1= 39.4131


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.2835 top1= 43.3093

Train epoch 29
[E29B0  |    640/50000 (  1%) ] Loss: 0.4455 top1= 84.0625
[E29B10 |   7040/50000 ( 14%) ] Loss: 0.4068 top1= 85.0000
[E29B20 |  13440/50000 ( 27%) ] Loss: 0.4220 top1= 84.5312
[E29B30 |  19840/50000 ( 40%) ] Loss: 0.4187 top1= 85.3125
[E29B40 |  26240/50000 ( 52%) ] Loss: 0.4252 top1= 85.7812
[E29B50 |  32640/50000 ( 65%) ] Loss: 0.4340 top1= 84.3750
[E29B60 |  39040/50000 ( 78%) ] Loss: 0.4417 top1= 84.3750
[E29B70 |  45440/50000 ( 91%) ] Loss: 0.4113 top1= 86.2500

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3085 top1= 10.0761


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.3065 top1= 40.1242


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.0675 top1= 43.0389

Train epoch 30
[E30B0  |    640/50000 (  1%) ] Loss: 0.4285 top1= 84.8438
[E30B10 |   7040/50000 ( 14%) ] Loss: 0.4501 top1= 84.2188
[E30B20 |  13440/50000 ( 27%) ] Loss: 0.3617 top1= 87.5000
[E30B30 |  19840/50000 ( 40%) ] Loss: 0.3628 top1= 87.6562
[E30B40 |  26240/50000 ( 52%) ] Loss: 0.3882 top1= 86.7188
[E30B50 |  32640/50000 ( 65%) ] Loss: 0.4024 top1= 85.3125
[E30B60 |  39040/50000 ( 78%) ] Loss: 0.4264 top1= 85.1562
[E30B70 |  45440/50000 ( 91%) ] Loss: 0.3775 top1= 85.1562

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3061 top1= 10.0561


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.3719 top1= 40.3145


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.9894 top1= 44.2308

Train epoch 31
[E31B0  |    640/50000 (  1%) ] Loss: 0.3712 top1= 84.8438
[E31B10 |   7040/50000 ( 14%) ] Loss: 0.3992 top1= 84.8438
[E31B20 |  13440/50000 ( 27%) ] Loss: 0.3322 top1= 88.7500
[E31B30 |  19840/50000 ( 40%) ] Loss: 0.3738 top1= 85.1562
[E31B40 |  26240/50000 ( 52%) ] Loss: 0.3820 top1= 86.0938
[E31B50 |  32640/50000 ( 65%) ] Loss: 0.4540 top1= 82.8125
[E31B60 |  39040/50000 ( 78%) ] Loss: 0.4250 top1= 84.8438
[E31B70 |  45440/50000 ( 91%) ] Loss: 0.4181 top1= 83.1250

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.3071 top1= 10.0661


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.1329 top1= 39.8738


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.0062 top1= 43.2091

Train epoch 32
[E32B0  |    640/50000 (  1%) ] Loss: 0.3817 top1= 85.6250
[E32B10 |   7040/50000 ( 14%) ] Loss: 0.4423 top1= 84.2188
[E32B20 |  13440/50000 ( 27%) ] Loss: 0.3806 top1= 86.0938
[E32B30 |  19840/50000 ( 40%) ] Loss: 0.3691 top1= 87.5000
[E32B40 |  26240/50000 ( 52%) ] Loss: 0.4082 top1= 86.0938
[E32B50 |  32640/50000 ( 65%) ] Loss: 0.4061 top1= 84.6875
[E32B60 |  39040/50000 ( 78%) ] Loss: 0.4482 top1= 83.9062
[E32B70 |  45440/50000 ( 91%) ] Loss: 0.3556 top1= 87.5000

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.2824 top1= 10.5068


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.4966 top1= 40.2544


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.7792 top1= 42.8686

Train epoch 33
[E33B0  |    640/50000 (  1%) ] Loss: 0.4579 top1= 82.5000
[E33B10 |   7040/50000 ( 14%) ] Loss: 0.4114 top1= 86.4062
[E33B20 |  13440/50000 ( 27%) ] Loss: 0.3512 top1= 86.7188
[E33B30 |  19840/50000 ( 40%) ] Loss: 0.3537 top1= 88.1250
[E33B40 |  26240/50000 ( 52%) ] Loss: 0.3235 top1= 90.3125
[E33B50 |  32640/50000 ( 65%) ] Loss: 0.3648 top1= 85.3125
[E33B60 |  39040/50000 ( 78%) ] Loss: 0.4186 top1= 86.4062
[E33B70 |  45440/50000 ( 91%) ] Loss: 0.3459 top1= 86.8750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.2989 top1= 10.1462


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.1271 top1= 40.0240


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.9032 top1= 43.9303

Train epoch 34
[E34B0  |    640/50000 (  1%) ] Loss: 0.4073 top1= 84.8438
[E34B10 |   7040/50000 ( 14%) ] Loss: 0.3691 top1= 86.7188
[E34B20 |  13440/50000 ( 27%) ] Loss: 0.3340 top1= 87.6562
[E34B30 |  19840/50000 ( 40%) ] Loss: 0.3421 top1= 87.6562
[E34B40 |  26240/50000 ( 52%) ] Loss: 0.3721 top1= 86.0938
[E34B50 |  32640/50000 ( 65%) ] Loss: 0.3903 top1= 84.6875
[E34B60 |  39040/50000 ( 78%) ] Loss: 0.3329 top1= 87.5000
[E34B70 |  45440/50000 ( 91%) ] Loss: 0.3386 top1= 88.2812

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.2899 top1= 10.1663


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.1936 top1= 40.8854


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.9565 top1= 43.6298

Train epoch 35
[E35B0  |    640/50000 (  1%) ] Loss: 0.3625 top1= 85.9375
[E35B10 |   7040/50000 ( 14%) ] Loss: 0.3793 top1= 85.4688
[E35B20 |  13440/50000 ( 27%) ] Loss: 0.3294 top1= 89.0625
[E35B30 |  19840/50000 ( 40%) ] Loss: 0.3473 top1= 87.5000
[E35B40 |  26240/50000 ( 52%) ] Loss: 0.3734 top1= 86.7188
[E35B50 |  32640/50000 ( 65%) ] Loss: 0.3637 top1= 87.0312
[E35B60 |  39040/50000 ( 78%) ] Loss: 0.3326 top1= 87.3438
[E35B70 |  45440/50000 ( 91%) ] Loss: 0.3092 top1= 88.2812

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.2770 top1= 10.4968


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.4529 top1= 41.2760


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.2027 top1= 43.7300

Train epoch 36
[E36B0  |    640/50000 (  1%) ] Loss: 0.3288 top1= 87.6562
[E36B10 |   7040/50000 ( 14%) ] Loss: 0.3535 top1= 86.2500
[E36B20 |  13440/50000 ( 27%) ] Loss: 0.3079 top1= 89.2188
[E36B30 |  19840/50000 ( 40%) ] Loss: 0.3214 top1= 87.6562
[E36B40 |  26240/50000 ( 52%) ] Loss: 0.3667 top1= 86.5625
[E36B50 |  32640/50000 ( 65%) ] Loss: 0.3916 top1= 84.0625
[E36B60 |  39040/50000 ( 78%) ] Loss: 0.3766 top1= 86.2500
[E36B70 |  45440/50000 ( 91%) ] Loss: 0.2922 top1= 90.1562

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.2644 top1= 10.8874


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.8461 top1= 41.1959


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.9710 top1= 43.5797

Train epoch 37
[E37B0  |    640/50000 (  1%) ] Loss: 0.3654 top1= 86.0938
[E37B10 |   7040/50000 ( 14%) ] Loss: 0.3787 top1= 86.8750
[E37B20 |  13440/50000 ( 27%) ] Loss: 0.3219 top1= 87.0312
[E37B30 |  19840/50000 ( 40%) ] Loss: 0.3116 top1= 88.9062
[E37B40 |  26240/50000 ( 52%) ] Loss: 0.3837 top1= 85.7812
[E37B50 |  32640/50000 ( 65%) ] Loss: 0.3801 top1= 85.9375
[E37B60 |  39040/50000 ( 78%) ] Loss: 0.3299 top1= 88.1250
[E37B70 |  45440/50000 ( 91%) ] Loss: 0.3205 top1= 86.8750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.2750 top1= 10.6370


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.5600 top1= 41.0457


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.3294 top1= 43.7901

Train epoch 38
[E38B0  |    640/50000 (  1%) ] Loss: 0.3559 top1= 87.5000
[E38B10 |   7040/50000 ( 14%) ] Loss: 0.3810 top1= 86.0938
[E38B20 |  13440/50000 ( 27%) ] Loss: 0.3469 top1= 87.5000
[E38B30 |  19840/50000 ( 40%) ] Loss: 0.3024 top1= 88.4375
[E38B40 |  26240/50000 ( 52%) ] Loss: 0.3139 top1= 88.4375
[E38B50 |  32640/50000 ( 65%) ] Loss: 0.3667 top1= 85.1562
[E38B60 |  39040/50000 ( 78%) ] Loss: 0.3211 top1= 89.2188
[E38B70 |  45440/50000 ( 91%) ] Loss: 0.3080 top1= 88.2812

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.2701 top1= 11.6787


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.4755 top1= 40.6651


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.8375 top1= 44.0405

Train epoch 39
[E39B0  |    640/50000 (  1%) ] Loss: 0.3423 top1= 86.5625
[E39B10 |   7040/50000 ( 14%) ] Loss: 0.3635 top1= 85.4688
[E39B20 |  13440/50000 ( 27%) ] Loss: 0.2711 top1= 90.7812
[E39B30 |  19840/50000 ( 40%) ] Loss: 0.2934 top1= 89.0625
[E39B40 |  26240/50000 ( 52%) ] Loss: 0.3531 top1= 86.7188
[E39B50 |  32640/50000 ( 65%) ] Loss: 0.3040 top1= 88.7500
[E39B60 |  39040/50000 ( 78%) ] Loss: 0.3891 top1= 86.2500
[E39B70 |  45440/50000 ( 91%) ] Loss: 0.3340 top1= 87.6562

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.2126 top1= 12.7704


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=6.1095 top1= 41.4163


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.0743 top1= 44.3309

Train epoch 40
[E40B0  |    640/50000 (  1%) ] Loss: 0.3367 top1= 87.5000
[E40B10 |   7040/50000 ( 14%) ] Loss: 0.3515 top1= 86.8750
[E40B20 |  13440/50000 ( 27%) ] Loss: 0.3245 top1= 87.9688
[E40B30 |  19840/50000 ( 40%) ] Loss: 0.3060 top1= 88.5938
[E40B40 |  26240/50000 ( 52%) ] Loss: 0.3343 top1= 89.3750
[E40B50 |  32640/50000 ( 65%) ] Loss: 0.3416 top1= 87.1875
[E40B60 |  39040/50000 ( 78%) ] Loss: 0.3492 top1= 87.1875
[E40B70 |  45440/50000 ( 91%) ] Loss: 0.3319 top1= 87.8125

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.2225 top1= 11.9091


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.4893 top1= 41.4463


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.8664 top1= 44.0104

Train epoch 41
[E41B0  |    640/50000 (  1%) ] Loss: 0.3137 top1= 87.1875
[E41B10 |   7040/50000 ( 14%) ] Loss: 0.3256 top1= 87.8125
[E41B20 |  13440/50000 ( 27%) ] Loss: 0.2756 top1= 89.5312
[E41B30 |  19840/50000 ( 40%) ] Loss: 0.2690 top1= 90.3125
[E41B40 |  26240/50000 ( 52%) ] Loss: 0.3226 top1= 89.8438
[E41B50 |  32640/50000 ( 65%) ] Loss: 0.3412 top1= 89.2188
[E41B60 |  39040/50000 ( 78%) ] Loss: 0.3314 top1= 88.5938
[E41B70 |  45440/50000 ( 91%) ] Loss: 0.2713 top1= 89.6875

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.1718 top1= 15.0741


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.6142 top1= 41.1258


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.5043 top1= 44.9319

Train epoch 42
[E42B0  |    640/50000 (  1%) ] Loss: 0.3376 top1= 86.8750
[E42B10 |   7040/50000 ( 14%) ] Loss: 0.2993 top1= 89.0625
[E42B20 |  13440/50000 ( 27%) ] Loss: 0.2961 top1= 89.5312
[E42B30 |  19840/50000 ( 40%) ] Loss: 0.2767 top1= 89.0625
[E42B40 |  26240/50000 ( 52%) ] Loss: 0.2823 top1= 88.7500
[E42B50 |  32640/50000 ( 65%) ] Loss: 0.3821 top1= 85.6250
[E42B60 |  39040/50000 ( 78%) ] Loss: 0.3642 top1= 86.0938
[E42B70 |  45440/50000 ( 91%) ] Loss: 0.3044 top1= 88.9062

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.1592 top1= 15.7151


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.6838 top1= 41.2961


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.5961 top1= 43.8001

Train epoch 43
[E43B0  |    640/50000 (  1%) ] Loss: 0.3074 top1= 86.8750
[E43B10 |   7040/50000 ( 14%) ] Loss: 0.2973 top1= 89.3750
[E43B20 |  13440/50000 ( 27%) ] Loss: 0.2485 top1= 91.8750
[E43B30 |  19840/50000 ( 40%) ] Loss: 0.2566 top1= 90.9375
[E43B40 |  26240/50000 ( 52%) ] Loss: 0.3225 top1= 87.0312
[E43B50 |  32640/50000 ( 65%) ] Loss: 0.2612 top1= 89.6875
[E43B60 |  39040/50000 ( 78%) ] Loss: 0.2892 top1= 89.5312
[E43B70 |  45440/50000 ( 91%) ] Loss: 0.2722 top1= 89.2188

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.1690 top1= 15.0040


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.4741 top1= 42.0573


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.7713 top1= 44.9219

Train epoch 44
[E44B0  |    640/50000 (  1%) ] Loss: 0.2537 top1= 90.9375
[E44B10 |   7040/50000 ( 14%) ] Loss: 0.3203 top1= 87.8125
[E44B20 |  13440/50000 ( 27%) ] Loss: 0.2492 top1= 90.3125
[E44B30 |  19840/50000 ( 40%) ] Loss: 0.2619 top1= 90.1562
[E44B40 |  26240/50000 ( 52%) ] Loss: 0.2993 top1= 90.1562
[E44B50 |  32640/50000 ( 65%) ] Loss: 0.3048 top1= 87.9688
[E44B60 |  39040/50000 ( 78%) ] Loss: 0.2802 top1= 90.4688
[E44B70 |  45440/50000 ( 91%) ] Loss: 0.2807 top1= 89.0625

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.1558 top1= 15.6651


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=6.0238 top1= 42.0373


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.2521 top1= 44.4111

Train epoch 45
[E45B0  |    640/50000 (  1%) ] Loss: 0.3090 top1= 87.5000
[E45B10 |   7040/50000 ( 14%) ] Loss: 0.3462 top1= 87.5000
[E45B20 |  13440/50000 ( 27%) ] Loss: 0.3103 top1= 87.9688
[E45B30 |  19840/50000 ( 40%) ] Loss: 0.2728 top1= 89.8438
[E45B40 |  26240/50000 ( 52%) ] Loss: 0.2493 top1= 92.5000
[E45B50 |  32640/50000 ( 65%) ] Loss: 0.2905 top1= 89.3750
[E45B60 |  39040/50000 ( 78%) ] Loss: 0.3191 top1= 88.1250
[E45B70 |  45440/50000 ( 91%) ] Loss: 0.2638 top1= 90.7812

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.1465 top1= 15.7151


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.7246 top1= 41.5966


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.7623 top1= 44.7416

Train epoch 46
[E46B0  |    640/50000 (  1%) ] Loss: 0.3387 top1= 88.7500
[E46B10 |   7040/50000 ( 14%) ] Loss: 0.3461 top1= 86.2500
[E46B20 |  13440/50000 ( 27%) ] Loss: 0.2870 top1= 90.1562
[E46B30 |  19840/50000 ( 40%) ] Loss: 0.3064 top1= 88.4375
[E46B40 |  26240/50000 ( 52%) ] Loss: 0.2890 top1= 90.9375
[E46B50 |  32640/50000 ( 65%) ] Loss: 0.3023 top1= 88.5938
[E46B60 |  39040/50000 ( 78%) ] Loss: 0.2752 top1= 90.4688
[E46B70 |  45440/50000 ( 91%) ] Loss: 0.2740 top1= 90.7812

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.1454 top1= 15.5349


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.7869 top1= 42.0172


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.3354 top1= 44.8017

Train epoch 47
[E47B0  |    640/50000 (  1%) ] Loss: 0.2629 top1= 91.4062
[E47B10 |   7040/50000 ( 14%) ] Loss: 0.2972 top1= 90.0000
[E47B20 |  13440/50000 ( 27%) ] Loss: 0.2389 top1= 90.6250
[E47B30 |  19840/50000 ( 40%) ] Loss: 0.2774 top1= 90.1562
[E47B40 |  26240/50000 ( 52%) ] Loss: 0.2379 top1= 91.2500
[E47B50 |  32640/50000 ( 65%) ] Loss: 0.2656 top1= 90.3125
[E47B60 |  39040/50000 ( 78%) ] Loss: 0.2698 top1= 90.4688
[E47B70 |  45440/50000 ( 91%) ] Loss: 0.2946 top1= 90.4688

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.1401 top1= 15.3446


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.6504 top1= 41.8369


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.3486 top1= 44.9319

Train epoch 48
[E48B0  |    640/50000 (  1%) ] Loss: 0.2894 top1= 89.8438
[E48B10 |   7040/50000 ( 14%) ] Loss: 0.2971 top1= 89.5312
[E48B20 |  13440/50000 ( 27%) ] Loss: 0.2519 top1= 90.1562
[E48B30 |  19840/50000 ( 40%) ] Loss: 0.2214 top1= 92.3438
[E48B40 |  26240/50000 ( 52%) ] Loss: 0.2532 top1= 90.6250
[E48B50 |  32640/50000 ( 65%) ] Loss: 0.2630 top1= 90.9375
[E48B60 |  39040/50000 ( 78%) ] Loss: 0.3101 top1= 88.2812
[E48B70 |  45440/50000 ( 91%) ] Loss: 0.2642 top1= 90.7812

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.1433 top1= 15.3846


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.5124 top1= 42.1575


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.6760 top1= 44.5112

Train epoch 49
[E49B0  |    640/50000 (  1%) ] Loss: 0.2767 top1= 89.3750
[E49B10 |   7040/50000 ( 14%) ] Loss: 0.3107 top1= 88.2812
[E49B20 |  13440/50000 ( 27%) ] Loss: 0.2481 top1= 90.7812
[E49B30 |  19840/50000 ( 40%) ] Loss: 0.2456 top1= 91.2500
[E49B40 |  26240/50000 ( 52%) ] Loss: 0.2681 top1= 91.5625
[E49B50 |  32640/50000 ( 65%) ] Loss: 0.2623 top1= 90.6250
[E49B60 |  39040/50000 ( 78%) ] Loss: 0.2976 top1= 89.0625
[E49B70 |  45440/50000 ( 91%) ] Loss: 0.2050 top1= 92.1875

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.1013 top1= 20.5228


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.6670 top1= 41.7167


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.4873 top1= 45.6430

Train epoch 50
[E50B0  |    640/50000 (  1%) ] Loss: 0.2380 top1= 89.3750
[E50B10 |   7040/50000 ( 14%) ] Loss: 0.2770 top1= 89.5312
[E50B20 |  13440/50000 ( 27%) ] Loss: 0.2693 top1= 91.2500
[E50B30 |  19840/50000 ( 40%) ] Loss: 0.2496 top1= 90.4688
[E50B40 |  26240/50000 ( 52%) ] Loss: 0.2280 top1= 92.8125
[E50B50 |  32640/50000 ( 65%) ] Loss: 0.2612 top1= 89.0625
[E50B60 |  39040/50000 ( 78%) ] Loss: 0.2522 top1= 91.4062
[E50B70 |  45440/50000 ( 91%) ] Loss: 0.2369 top1= 90.4688

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.0928 top1= 21.8750


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.6074 top1= 41.6466


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.3807 top1= 45.3225

Train epoch 51
[E51B0  |    640/50000 (  1%) ] Loss: 0.2711 top1= 89.8438
[E51B10 |   7040/50000 ( 14%) ] Loss: 0.2987 top1= 87.9688
[E51B20 |  13440/50000 ( 27%) ] Loss: 0.2157 top1= 92.0312
[E51B30 |  19840/50000 ( 40%) ] Loss: 0.2429 top1= 91.0938
[E51B40 |  26240/50000 ( 52%) ] Loss: 0.2712 top1= 90.4688
[E51B50 |  32640/50000 ( 65%) ] Loss: 0.3013 top1= 90.6250
[E51B60 |  39040/50000 ( 78%) ] Loss: 0.2134 top1= 92.0312
[E51B70 |  45440/50000 ( 91%) ] Loss: 0.2464 top1= 91.5625

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.0999 top1= 22.1554


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.3631 top1= 41.8670


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.2393 top1= 44.8017

Train epoch 52
[E52B0  |    640/50000 (  1%) ] Loss: 0.2637 top1= 90.7812
[E52B10 |   7040/50000 ( 14%) ] Loss: 0.2487 top1= 91.0938
[E52B20 |  13440/50000 ( 27%) ] Loss: 0.2366 top1= 90.7812
[E52B30 |  19840/50000 ( 40%) ] Loss: 0.2512 top1= 89.5312
[E52B40 |  26240/50000 ( 52%) ] Loss: 0.2633 top1= 91.0938
[E52B50 |  32640/50000 ( 65%) ] Loss: 0.2944 top1= 87.8125
[E52B60 |  39040/50000 ( 78%) ] Loss: 0.2262 top1= 92.3438
[E52B70 |  45440/50000 ( 91%) ] Loss: 0.2738 top1= 90.0000

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.0872 top1= 22.7163


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.9337 top1= 41.9872


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.2732 top1= 44.8718

Train epoch 53
[E53B0  |    640/50000 (  1%) ] Loss: 0.2643 top1= 89.3750
[E53B10 |   7040/50000 ( 14%) ] Loss: 0.2769 top1= 89.3750
[E53B20 |  13440/50000 ( 27%) ] Loss: 0.2628 top1= 90.4688
[E53B30 |  19840/50000 ( 40%) ] Loss: 0.2425 top1= 91.5625
[E53B40 |  26240/50000 ( 52%) ] Loss: 0.2318 top1= 92.5000
[E53B50 |  32640/50000 ( 65%) ] Loss: 0.3065 top1= 88.9062
[E53B60 |  39040/50000 ( 78%) ] Loss: 0.2623 top1= 90.0000
[E53B70 |  45440/50000 ( 91%) ] Loss: 0.2949 top1= 89.5312

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.0576 top1= 26.5525


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.3339 top1= 41.8570


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.7695 top1= 44.9119

Train epoch 54
[E54B0  |    640/50000 (  1%) ] Loss: 0.2717 top1= 90.6250
[E54B10 |   7040/50000 ( 14%) ] Loss: 0.2488 top1= 90.4688
[E54B20 |  13440/50000 ( 27%) ] Loss: 0.1971 top1= 93.2812
[E54B30 |  19840/50000 ( 40%) ] Loss: 0.2399 top1= 91.0938
[E54B40 |  26240/50000 ( 52%) ] Loss: 0.2280 top1= 92.3438
[E54B50 |  32640/50000 ( 65%) ] Loss: 0.2674 top1= 91.0938
[E54B60 |  39040/50000 ( 78%) ] Loss: 0.2375 top1= 91.0938
[E54B70 |  45440/50000 ( 91%) ] Loss: 0.2515 top1= 90.9375

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.0481 top1= 27.0733


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.7175 top1= 42.1174


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.7708 top1= 44.8618

Train epoch 55
[E55B0  |    640/50000 (  1%) ] Loss: 0.2807 top1= 89.3750
[E55B10 |   7040/50000 ( 14%) ] Loss: 0.2543 top1= 91.8750
[E55B20 |  13440/50000 ( 27%) ] Loss: 0.2229 top1= 92.8125
[E55B30 |  19840/50000 ( 40%) ] Loss: 0.2642 top1= 91.2500
[E55B40 |  26240/50000 ( 52%) ] Loss: 0.2223 top1= 91.7188
[E55B50 |  32640/50000 ( 65%) ] Loss: 0.3023 top1= 89.2188
[E55B60 |  39040/50000 ( 78%) ] Loss: 0.2614 top1= 89.8438
[E55B70 |  45440/50000 ( 91%) ] Loss: 0.2090 top1= 92.1875

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.0164 top1= 29.1466


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.8900 top1= 42.2276


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.9905 top1= 45.1823

Train epoch 56
[E56B0  |    640/50000 (  1%) ] Loss: 0.2416 top1= 89.3750
[E56B10 |   7040/50000 ( 14%) ] Loss: 0.2275 top1= 90.1562
[E56B20 |  13440/50000 ( 27%) ] Loss: 0.2303 top1= 91.5625
[E56B30 |  19840/50000 ( 40%) ] Loss: 0.2396 top1= 91.5625
[E56B40 |  26240/50000 ( 52%) ] Loss: 0.2559 top1= 90.9375
[E56B50 |  32640/50000 ( 65%) ] Loss: 0.2820 top1= 89.6875
[E56B60 |  39040/50000 ( 78%) ] Loss: 0.2342 top1= 91.8750
[E56B70 |  45440/50000 ( 91%) ] Loss: 0.2317 top1= 91.5625

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.0336 top1= 26.8329


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.6763 top1= 42.5982


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.8552 top1= 45.5128

Train epoch 57
[E57B0  |    640/50000 (  1%) ] Loss: 0.2713 top1= 89.6875
[E57B10 |   7040/50000 ( 14%) ] Loss: 0.2873 top1= 89.2188
[E57B20 |  13440/50000 ( 27%) ] Loss: 0.2226 top1= 91.7188
[E57B30 |  19840/50000 ( 40%) ] Loss: 0.2574 top1= 91.0938
[E57B40 |  26240/50000 ( 52%) ] Loss: 0.2473 top1= 90.1562
[E57B50 |  32640/50000 ( 65%) ] Loss: 0.2910 top1= 90.3125
[E57B60 |  39040/50000 ( 78%) ] Loss: 0.2356 top1= 90.9375
[E57B70 |  45440/50000 ( 91%) ] Loss: 0.2348 top1= 92.1875

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.0277 top1= 24.0084


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.7065 top1= 42.4479


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=6.2078 top1= 45.2324

Train epoch 58
[E58B0  |    640/50000 (  1%) ] Loss: 0.2300 top1= 91.2500
[E58B10 |   7040/50000 ( 14%) ] Loss: 0.2395 top1= 90.6250
[E58B20 |  13440/50000 ( 27%) ] Loss: 0.1833 top1= 92.9688
[E58B30 |  19840/50000 ( 40%) ] Loss: 0.2009 top1= 92.3438
[E58B40 |  26240/50000 ( 52%) ] Loss: 0.2233 top1= 91.7188
[E58B50 |  32640/50000 ( 65%) ] Loss: 0.2087 top1= 92.6562
[E58B60 |  39040/50000 ( 78%) ] Loss: 0.2387 top1= 92.5000
[E58B70 |  45440/50000 ( 91%) ] Loss: 0.2090 top1= 92.5000

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.9935 top1= 28.2752


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=6.3240 top1= 42.3978


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=6.4209 top1= 44.9018

Train epoch 59
[E59B0  |    640/50000 (  1%) ] Loss: 0.2468 top1= 90.7812
[E59B10 |   7040/50000 ( 14%) ] Loss: 0.3157 top1= 88.2812
[E59B20 |  13440/50000 ( 27%) ] Loss: 0.2013 top1= 92.6562
[E59B30 |  19840/50000 ( 40%) ] Loss: 0.2244 top1= 92.5000
[E59B40 |  26240/50000 ( 52%) ] Loss: 0.1958 top1= 92.8125
[E59B50 |  32640/50000 ( 65%) ] Loss: 0.2204 top1= 92.8125
[E59B60 |  39040/50000 ( 78%) ] Loss: 0.2035 top1= 92.5000
[E59B70 |  45440/50000 ( 91%) ] Loss: 0.2032 top1= 92.1875

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.9857 top1= 30.7993


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=6.2081 top1= 42.2977


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.9993 top1= 45.2524

Train epoch 60
[E60B0  |    640/50000 (  1%) ] Loss: 0.2426 top1= 91.4062
[E60B10 |   7040/50000 ( 14%) ] Loss: 0.2339 top1= 92.0312
[E60B20 |  13440/50000 ( 27%) ] Loss: 0.1880 top1= 93.1250
[E60B30 |  19840/50000 ( 40%) ] Loss: 0.2148 top1= 90.9375
[E60B40 |  26240/50000 ( 52%) ] Loss: 0.2064 top1= 93.7500
[E60B50 |  32640/50000 ( 65%) ] Loss: 0.2378 top1= 92.0312
[E60B60 |  39040/50000 ( 78%) ] Loss: 0.2305 top1= 91.7188
[E60B70 |  45440/50000 ( 91%) ] Loss: 0.2461 top1= 91.4062

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.0138 top1= 28.4355


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.7656 top1= 42.9287


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.5783 top1= 45.2023

Train epoch 61
[E61B0  |    640/50000 (  1%) ] Loss: 0.2223 top1= 91.7188
[E61B10 |   7040/50000 ( 14%) ] Loss: 0.2894 top1= 88.4375
[E61B20 |  13440/50000 ( 27%) ] Loss: 0.1522 top1= 95.1562
[E61B30 |  19840/50000 ( 40%) ] Loss: 0.2226 top1= 91.7188
[E61B40 |  26240/50000 ( 52%) ] Loss: 0.1962 top1= 93.5938
[E61B50 |  32640/50000 ( 65%) ] Loss: 0.2575 top1= 90.9375
[E61B60 |  39040/50000 ( 78%) ] Loss: 0.2267 top1= 90.9375
[E61B70 |  45440/50000 ( 91%) ] Loss: 0.2347 top1= 91.0938

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.9664 top1= 32.6522


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.6129 top1= 42.3277


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=6.0893 top1= 44.9519

Train epoch 62
[E62B0  |    640/50000 (  1%) ] Loss: 0.2372 top1= 91.7188
[E62B10 |   7040/50000 ( 14%) ] Loss: 0.2434 top1= 91.7188
[E62B20 |  13440/50000 ( 27%) ] Loss: 0.2021 top1= 92.9688
[E62B30 |  19840/50000 ( 40%) ] Loss: 0.2150 top1= 91.5625
[E62B40 |  26240/50000 ( 52%) ] Loss: 0.2443 top1= 91.7188
[E62B50 |  32640/50000 ( 65%) ] Loss: 0.2509 top1= 90.4688
[E62B60 |  39040/50000 ( 78%) ] Loss: 0.2132 top1= 92.1875
[E62B70 |  45440/50000 ( 91%) ] Loss: 0.2270 top1= 91.0938

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.9667 top1= 30.4087


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=6.1132 top1= 41.9471


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.7238 top1= 45.3425

Train epoch 63
[E63B0  |    640/50000 (  1%) ] Loss: 0.2590 top1= 91.5625
[E63B10 |   7040/50000 ( 14%) ] Loss: 0.2633 top1= 89.8438
[E63B20 |  13440/50000 ( 27%) ] Loss: 0.2366 top1= 91.7188
[E63B30 |  19840/50000 ( 40%) ] Loss: 0.1823 top1= 93.4375
[E63B40 |  26240/50000 ( 52%) ] Loss: 0.1699 top1= 93.4375
[E63B50 |  32640/50000 ( 65%) ] Loss: 0.2033 top1= 91.4062
[E63B60 |  39040/50000 ( 78%) ] Loss: 0.1961 top1= 91.8750
[E63B70 |  45440/50000 ( 91%) ] Loss: 0.1611 top1= 94.2188

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.9168 top1= 37.3798


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=6.1064 top1= 42.3277


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.7115 top1= 45.0120

Train epoch 64
[E64B0  |    640/50000 (  1%) ] Loss: 0.2276 top1= 90.7812
[E64B10 |   7040/50000 ( 14%) ] Loss: 0.2262 top1= 92.3438
[E64B20 |  13440/50000 ( 27%) ] Loss: 0.1942 top1= 94.3750
[E64B30 |  19840/50000 ( 40%) ] Loss: 0.1801 top1= 93.1250
[E64B40 |  26240/50000 ( 52%) ] Loss: 0.2184 top1= 91.5625
[E64B50 |  32640/50000 ( 65%) ] Loss: 0.2242 top1= 91.5625
[E64B60 |  39040/50000 ( 78%) ] Loss: 0.1835 top1= 93.4375
[E64B70 |  45440/50000 ( 91%) ] Loss: 0.2100 top1= 92.1875

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.9332 top1= 32.1715


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=6.0486 top1= 42.7684


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.9999 top1= 45.5329

Train epoch 65
[E65B0  |    640/50000 (  1%) ] Loss: 0.1974 top1= 92.6562
[E65B10 |   7040/50000 ( 14%) ] Loss: 0.2411 top1= 91.8750
[E65B20 |  13440/50000 ( 27%) ] Loss: 0.1930 top1= 91.7188
[E65B30 |  19840/50000 ( 40%) ] Loss: 0.2529 top1= 91.2500
[E65B40 |  26240/50000 ( 52%) ] Loss: 0.2222 top1= 93.2812
[E65B50 |  32640/50000 ( 65%) ] Loss: 0.2565 top1= 91.2500
[E65B60 |  39040/50000 ( 78%) ] Loss: 0.2246 top1= 92.3438
[E65B70 |  45440/50000 ( 91%) ] Loss: 0.2490 top1= 90.7812

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.8918 top1= 39.3630


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=6.0681 top1= 41.9772


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=6.0599 top1= 45.9034

Train epoch 66
[E66B0  |    640/50000 (  1%) ] Loss: 0.2057 top1= 92.3438
[E66B10 |   7040/50000 ( 14%) ] Loss: 0.2055 top1= 92.8125
[E66B20 |  13440/50000 ( 27%) ] Loss: 0.2175 top1= 91.7188
[E66B30 |  19840/50000 ( 40%) ] Loss: 0.1939 top1= 94.0625
[E66B40 |  26240/50000 ( 52%) ] Loss: 0.2049 top1= 91.7188
[E66B50 |  32640/50000 ( 65%) ] Loss: 0.2129 top1= 91.2500
[E66B60 |  39040/50000 ( 78%) ] Loss: 0.2142 top1= 92.6562
[E66B70 |  45440/50000 ( 91%) ] Loss: 0.2313 top1= 91.7188

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.9429 top1= 32.8125


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.6971 top1= 42.3778


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=6.0485 top1= 46.1138

Train epoch 67
[E67B0  |    640/50000 (  1%) ] Loss: 0.1694 top1= 94.2188
[E67B10 |   7040/50000 ( 14%) ] Loss: 0.1821 top1= 92.9688
[E67B20 |  13440/50000 ( 27%) ] Loss: 0.1920 top1= 93.4375
[E67B30 |  19840/50000 ( 40%) ] Loss: 0.2204 top1= 91.2500
[E67B40 |  26240/50000 ( 52%) ] Loss: 0.1953 top1= 93.2812
[E67B50 |  32640/50000 ( 65%) ] Loss: 0.2482 top1= 92.0312
[E67B60 |  39040/50000 ( 78%) ] Loss: 0.1850 top1= 93.1250
[E67B70 |  45440/50000 ( 91%) ] Loss: 0.1799 top1= 93.4375

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.8838 top1= 38.9523


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.7724 top1= 41.9571


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=6.4059 top1= 45.8834

Train epoch 68
[E68B0  |    640/50000 (  1%) ] Loss: 0.2198 top1= 92.3438
[E68B10 |   7040/50000 ( 14%) ] Loss: 0.2974 top1= 90.9375
[E68B20 |  13440/50000 ( 27%) ] Loss: 0.2362 top1= 91.5625
[E68B30 |  19840/50000 ( 40%) ] Loss: 0.2097 top1= 93.5938
[E68B40 |  26240/50000 ( 52%) ] Loss: 0.2699 top1= 90.9375
[E68B50 |  32640/50000 ( 65%) ] Loss: 0.2582 top1= 91.8750
[E68B60 |  39040/50000 ( 78%) ] Loss: 0.1885 top1= 94.0625
[E68B70 |  45440/50000 ( 91%) ] Loss: 0.2306 top1= 91.0938

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.9101 top1= 39.2228


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.5406 top1= 42.3778


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=6.2917 top1= 46.0437

Train epoch 69
[E69B0  |    640/50000 (  1%) ] Loss: 0.1932 top1= 93.5938
[E69B10 |   7040/50000 ( 14%) ] Loss: 0.2246 top1= 90.9375
[E69B20 |  13440/50000 ( 27%) ] Loss: 0.2321 top1= 91.0938
[E69B30 |  19840/50000 ( 40%) ] Loss: 0.2136 top1= 91.5625
[E69B40 |  26240/50000 ( 52%) ] Loss: 0.1930 top1= 92.8125
[E69B50 |  32640/50000 ( 65%) ] Loss: 0.2011 top1= 93.4375
[E69B60 |  39040/50000 ( 78%) ] Loss: 0.2004 top1= 93.2812
[E69B70 |  45440/50000 ( 91%) ] Loss: 0.1722 top1= 93.4375

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.8771 top1= 39.3129


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=6.2196 top1= 43.2492


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.7753 top1= 45.4427

Train epoch 70
[E70B0  |    640/50000 (  1%) ] Loss: 0.1839 top1= 93.7500
[E70B10 |   7040/50000 ( 14%) ] Loss: 0.2159 top1= 91.5625
[E70B20 |  13440/50000 ( 27%) ] Loss: 0.1978 top1= 92.9688
[E70B30 |  19840/50000 ( 40%) ] Loss: 0.2225 top1= 91.7188
[E70B40 |  26240/50000 ( 52%) ] Loss: 0.2284 top1= 91.8750
[E70B50 |  32640/50000 ( 65%) ] Loss: 0.2338 top1= 90.9375
[E70B60 |  39040/50000 ( 78%) ] Loss: 0.1935 top1= 93.4375
[E70B70 |  45440/50000 ( 91%) ] Loss: 0.2317 top1= 92.5000

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.8662 top1= 36.8890


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=6.0585 top1= 42.7183


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=6.2920 top1= 45.4227

Train epoch 71
[E71B0  |    640/50000 (  1%) ] Loss: 0.2105 top1= 92.1875
[E71B10 |   7040/50000 ( 14%) ] Loss: 0.2019 top1= 91.7188
[E71B20 |  13440/50000 ( 27%) ] Loss: 0.1853 top1= 93.5938
[E71B30 |  19840/50000 ( 40%) ] Loss: 0.2423 top1= 91.5625
[E71B40 |  26240/50000 ( 52%) ] Loss: 0.2125 top1= 92.3438
[E71B50 |  32640/50000 ( 65%) ] Loss: 0.2224 top1= 91.5625
[E71B60 |  39040/50000 ( 78%) ] Loss: 0.2436 top1= 92.0312
[E71B70 |  45440/50000 ( 91%) ] Loss: 0.1827 top1= 93.5938

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.8372 top1= 42.1174


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=6.0182 top1= 42.6282


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.9918 top1= 45.4828

Train epoch 72
[E72B0  |    640/50000 (  1%) ] Loss: 0.2147 top1= 92.0312
[E72B10 |   7040/50000 ( 14%) ] Loss: 0.2485 top1= 92.0312
[E72B20 |  13440/50000 ( 27%) ] Loss: 0.1658 top1= 93.2812
[E72B30 |  19840/50000 ( 40%) ] Loss: 0.2095 top1= 92.5000
[E72B40 |  26240/50000 ( 52%) ] Loss: 0.2053 top1= 92.8125
[E72B50 |  32640/50000 ( 65%) ] Loss: 0.2668 top1= 90.3125
[E72B60 |  39040/50000 ( 78%) ] Loss: 0.2107 top1= 92.9688
[E72B70 |  45440/50000 ( 91%) ] Loss: 0.1717 top1= 93.5938

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.8354 top1= 44.6915


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.8195 top1= 42.3578


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=6.0754 top1= 45.3225

Train epoch 73
[E73B0  |    640/50000 (  1%) ] Loss: 0.1999 top1= 93.7500
[E73B10 |   7040/50000 ( 14%) ] Loss: 0.2153 top1= 91.5625
[E73B20 |  13440/50000 ( 27%) ] Loss: 0.2291 top1= 91.4062
[E73B30 |  19840/50000 ( 40%) ] Loss: 0.1785 top1= 93.1250
[E73B40 |  26240/50000 ( 52%) ] Loss: 0.1883 top1= 93.2812
[E73B50 |  32640/50000 ( 65%) ] Loss: 0.2227 top1= 91.0938
[E73B60 |  39040/50000 ( 78%) ] Loss: 0.2149 top1= 92.3438
[E73B70 |  45440/50000 ( 91%) ] Loss: 0.2136 top1= 92.9688

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.8419 top1= 41.6066


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.8408 top1= 42.2276


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=6.5780 top1= 45.3626

Train epoch 74
[E74B0  |    640/50000 (  1%) ] Loss: 0.1942 top1= 93.5938
[E74B10 |   7040/50000 ( 14%) ] Loss: 0.2009 top1= 92.1875
[E74B20 |  13440/50000 ( 27%) ] Loss: 0.1522 top1= 94.3750
[E74B30 |  19840/50000 ( 40%) ] Loss: 0.1982 top1= 92.3438
[E74B40 |  26240/50000 ( 52%) ] Loss: 0.2077 top1= 91.8750
[E74B50 |  32640/50000 ( 65%) ] Loss: 0.1946 top1= 92.0312
[E74B60 |  39040/50000 ( 78%) ] Loss: 0.1987 top1= 91.8750
[E74B70 |  45440/50000 ( 91%) ] Loss: 0.2236 top1= 91.4062

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.8375 top1= 39.7837


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=6.3825 top1= 42.9587


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=6.0613 top1= 45.6731

Train epoch 75
[E75B0  |    640/50000 (  1%) ] Loss: 0.2239 top1= 92.5000
[E75B10 |   7040/50000 ( 14%) ] Loss: 0.2079 top1= 93.1250
[E75B20 |  13440/50000 ( 27%) ] Loss: 0.2026 top1= 91.5625
[E75B30 |  19840/50000 ( 40%) ] Loss: 0.2164 top1= 92.3438
[E75B40 |  26240/50000 ( 52%) ] Loss: 0.1956 top1= 93.5938
[E75B50 |  32640/50000 ( 65%) ] Loss: 0.2271 top1= 90.3125
[E75B60 |  39040/50000 ( 78%) ] Loss: 0.1513 top1= 94.5312
[E75B70 |  45440/50000 ( 91%) ] Loss: 0.1704 top1= 93.9062

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.7521 top1= 49.2188


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=6.0823 top1= 41.2360


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=7.2748 top1= 45.6731

Train epoch 76
[E76B0  |    640/50000 (  1%) ] Loss: 0.1968 top1= 92.5000
[E76B10 |   7040/50000 ( 14%) ] Loss: 0.2680 top1= 90.4688
[E76B20 |  13440/50000 ( 27%) ] Loss: 0.1818 top1= 92.9688
[E76B30 |  19840/50000 ( 40%) ] Loss: 0.2180 top1= 92.6562
[E76B40 |  26240/50000 ( 52%) ] Loss: 0.1570 top1= 94.8438
[E76B50 |  32640/50000 ( 65%) ] Loss: 0.1672 top1= 94.2188
[E76B60 |  39040/50000 ( 78%) ] Loss: 0.1608 top1= 93.9062
[E76B70 |  45440/50000 ( 91%) ] Loss: 0.1754 top1= 92.9688

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.7712 top1= 47.5861


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=6.4443 top1= 42.9988


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=6.5736 top1= 45.5429

Train epoch 77
[E77B0  |    640/50000 (  1%) ] Loss: 0.1873 top1= 93.2812
[E77B10 |   7040/50000 ( 14%) ] Loss: 0.2104 top1= 91.5625
[E77B20 |  13440/50000 ( 27%) ] Loss: 0.1870 top1= 93.5938
[E77B30 |  19840/50000 ( 40%) ] Loss: 0.2063 top1= 92.5000
[E77B40 |  26240/50000 ( 52%) ] Loss: 0.1713 top1= 93.9062
[E77B50 |  32640/50000 ( 65%) ] Loss: 0.2470 top1= 91.8750
[E77B60 |  39040/50000 ( 78%) ] Loss: 0.1618 top1= 94.2188
[E77B70 |  45440/50000 ( 91%) ] Loss: 0.1798 top1= 93.9062

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.7885 top1= 45.3125


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=7.1034 top1= 42.4479


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.8883 top1= 45.4527

Train epoch 78
[E78B0  |    640/50000 (  1%) ] Loss: 0.2360 top1= 91.5625
[E78B10 |   7040/50000 ( 14%) ] Loss: 0.2197 top1= 92.0312
[E78B20 |  13440/50000 ( 27%) ] Loss: 0.1647 top1= 93.1250
[E78B30 |  19840/50000 ( 40%) ] Loss: 0.2434 top1= 91.8750
[E78B40 |  26240/50000 ( 52%) ] Loss: 0.2352 top1= 92.0312
[E78B50 |  32640/50000 ( 65%) ] Loss: 0.2157 top1= 93.2812
[E78B60 |  39040/50000 ( 78%) ] Loss: 0.1496 top1= 94.3750
[E78B70 |  45440/50000 ( 91%) ] Loss: 0.1736 top1= 93.2812

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.7949 top1= 44.3510


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=7.0304 top1= 43.2893


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.6014 top1= 44.7015

Train epoch 79
[E79B0  |    640/50000 (  1%) ] Loss: 0.2405 top1= 92.5000
[E79B10 |   7040/50000 ( 14%) ] Loss: 0.2013 top1= 92.1875
[E79B20 |  13440/50000 ( 27%) ] Loss: 0.1971 top1= 92.6562
[E79B30 |  19840/50000 ( 40%) ] Loss: 0.1628 top1= 94.0625
[E79B40 |  26240/50000 ( 52%) ] Loss: 0.1946 top1= 93.7500
[E79B50 |  32640/50000 ( 65%) ] Loss: 0.2026 top1= 92.9688
[E79B60 |  39040/50000 ( 78%) ] Loss: 0.1824 top1= 93.9062
[E79B70 |  45440/50000 ( 91%) ] Loss: 0.2470 top1= 91.7188

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.7309 top1= 51.0116


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=6.2372 top1= 42.5681


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=7.0451 top1= 45.7031

Train epoch 80
[E80B0  |    640/50000 (  1%) ] Loss: 0.1841 top1= 92.1875
[E80B10 |   7040/50000 ( 14%) ] Loss: 0.2095 top1= 92.1875
[E80B20 |  13440/50000 ( 27%) ] Loss: 0.1563 top1= 93.9062
[E80B30 |  19840/50000 ( 40%) ] Loss: 0.1957 top1= 94.5312
[E80B40 |  26240/50000 ( 52%) ] Loss: 0.1707 top1= 93.9062
[E80B50 |  32640/50000 ( 65%) ] Loss: 0.1905 top1= 92.5000
[E80B60 |  39040/50000 ( 78%) ] Loss: 0.1919 top1= 91.7188
[E80B70 |  45440/50000 ( 91%) ] Loss: 0.1683 top1= 93.9062

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.7596 top1= 45.3526


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=7.2147 top1= 42.8085


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=6.6809 top1= 45.4928

Train epoch 81
[E81B0  |    640/50000 (  1%) ] Loss: 0.1703 top1= 94.5312
[E81B10 |   7040/50000 ( 14%) ] Loss: 0.1829 top1= 92.8125
[E81B20 |  13440/50000 ( 27%) ] Loss: 0.1468 top1= 95.7812
[E81B30 |  19840/50000 ( 40%) ] Loss: 0.1460 top1= 94.8438
[E81B40 |  26240/50000 ( 52%) ] Loss: 0.1116 top1= 95.9375
[E81B50 |  32640/50000 ( 65%) ] Loss: 0.1004 top1= 96.8750
[E81B60 |  39040/50000 ( 78%) ] Loss: 0.1001 top1= 97.1875
[E81B70 |  45440/50000 ( 91%) ] Loss: 0.1260 top1= 95.0000

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.7221 top1= 46.8149


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=6.9691 top1= 43.9002


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=7.3728 top1= 46.3642

Train epoch 82
[E82B0  |    640/50000 (  1%) ] Loss: 0.1352 top1= 95.4688
[E82B10 |   7040/50000 ( 14%) ] Loss: 0.1155 top1= 95.4688
[E82B20 |  13440/50000 ( 27%) ] Loss: 0.0791 top1= 97.0312
[E82B30 |  19840/50000 ( 40%) ] Loss: 0.1234 top1= 95.3125
[E82B40 |  26240/50000 ( 52%) ] Loss: 0.0955 top1= 96.7188
[E82B50 |  32640/50000 ( 65%) ] Loss: 0.0828 top1= 96.7188
