
=== Start adding workers ===
=> Add worker SGDMWorker(index=0, momentum=0.9)
=> Add worker SGDMWorker(index=1, momentum=0.9)
=> Add worker SGDMWorker(index=2, momentum=0.9)
=> Add worker SGDMWorker(index=3, momentum=0.9)
=> Add worker SGDMWorker(index=4, momentum=0.9)
=> Add worker SGDMWorker(index=5, momentum=0.9)
=> Add worker SGDMWorker(index=6, momentum=0.9)
=> Add worker SGDMWorker(index=7, momentum=0.9)
=> Add worker SGDMWorker(index=8, momentum=0.9)
=> Add worker SGDMWorker(index=9, momentum=0.9)
=> Add worker ByzantineWorker(index=10)
=> Add worker ByzantineWorker(index=11)

=== Start adding graph ===
<codes.graph_utils.Dumbbell object at 0x7febd99d8730>

Train epoch 1
[E 1B0  |    384/60000 (  1%) ] Loss: 2.3054 top1= 10.0000

=== Peeking data label distribution E1B0 ===
Worker 0 has targets: tensor([0, 0, 0, 0, 0], device='cuda:0')
Worker 1 has targets: tensor([1, 1, 1, 1, 1], device='cuda:0')
Worker 2 has targets: tensor([1, 2, 2, 2, 2], device='cuda:0')
Worker 3 has targets: tensor([2, 3, 3, 3, 3], device='cuda:0')
Worker 4 has targets: tensor([3, 4, 4, 4, 4], device='cuda:0')
Worker 5 has targets: tensor([4, 5, 5, 5, 5], device='cuda:0')
Worker 6 has targets: tensor([6, 6, 6, 6, 6], device='cuda:0')
Worker 7 has targets: tensor([7, 7, 7, 7, 7], device='cuda:0')
Worker 8 has targets: tensor([7, 8, 8, 8, 8], device='cuda:0')
Worker 9 has targets: tensor([8, 9, 9, 9, 9], device='cuda:0')
Worker 10 has targets: tensor([0, 0, 0, 0, 0], device='cuda:0')
Worker 11 has targets: tensor([1, 1, 1, 1, 1], device='cuda:0')


[E 1B10 |   4224/60000 (  7%) ] Loss: 0.2755 top1= 92.8125
[E 1B20 |   8064/60000 ( 13%) ] Loss: 0.1101 top1= 98.1250

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=3.0327 top1= 36.2680


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=8.5083 top1= 46.4643


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.3177 top1= 39.8237

Train epoch 2
[E 2B0  |    384/60000 (  1%) ] Loss: 0.2293 top1= 96.2500
[E 2B10 |   4224/60000 (  7%) ] Loss: 0.0646 top1= 97.8125
[E 2B20 |   8064/60000 ( 13%) ] Loss: 0.0700 top1= 98.7500

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.7601 top1= 50.1903


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=5.7014 top1= 47.1955


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=2.7273 top1= 42.1274

Train epoch 3
[E 3B0  |    384/60000 (  1%) ] Loss: 0.1229 top1= 97.5000
[E 3B10 |   4224/60000 (  7%) ] Loss: 0.0482 top1= 98.4375
[E 3B20 |   8064/60000 ( 13%) ] Loss: 0.0380 top1= 98.4375

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3776 top1= 51.9732


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.8913 top1= 47.4960


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=2.1572 top1= 51.1118

Train epoch 4
[E 4B0  |    384/60000 (  1%) ] Loss: 0.0800 top1= 98.1250
[E 4B10 |   4224/60000 (  7%) ] Loss: 0.0365 top1= 99.0625
[E 4B20 |   8064/60000 ( 13%) ] Loss: 0.0290 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.4263 top1= 51.3922


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.7262 top1= 47.3758


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.9436 top1= 50.8714

Train epoch 5
[E 5B0  |    384/60000 (  1%) ] Loss: 0.0621 top1= 98.7500
[E 5B10 |   4224/60000 (  7%) ] Loss: 0.0326 top1= 99.0625
[E 5B20 |   8064/60000 ( 13%) ] Loss: 0.0268 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.4417 top1= 51.7628


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.6822 top1= 47.4058


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.9179 top1= 52.2336

Train epoch 6
[E 6B0  |    384/60000 (  1%) ] Loss: 0.0600 top1= 98.7500
[E 6B10 |   4224/60000 (  7%) ] Loss: 0.0319 top1= 99.0625
[E 6B20 |   8064/60000 ( 13%) ] Loss: 0.0265 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.4341 top1= 51.8429


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.6488 top1= 47.4760


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8125 top1= 55.6290

Train epoch 7
[E 7B0  |    384/60000 (  1%) ] Loss: 0.0716 top1= 98.4375
[E 7B10 |   4224/60000 (  7%) ] Loss: 0.0347 top1= 98.7500
[E 7B20 |   8064/60000 ( 13%) ] Loss: 0.0255 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.4777 top1= 51.6727


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.6240 top1= 47.4960


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.7313 top1= 56.8910

Train epoch 8
[E 8B0  |    384/60000 (  1%) ] Loss: 0.0554 top1= 99.0625
[E 8B10 |   4224/60000 (  7%) ] Loss: 0.0282 top1= 99.0625
[E 8B20 |   8064/60000 ( 13%) ] Loss: 0.0257 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.5208 top1= 51.6727


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.6112 top1= 47.5661


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.5821 top1= 58.4936

Train epoch 9
[E 9B0  |    384/60000 (  1%) ] Loss: 0.0926 top1= 98.1250
[E 9B10 |   4224/60000 (  7%) ] Loss: 0.0277 top1= 99.0625
[E 9B20 |   8064/60000 ( 13%) ] Loss: 0.0321 top1= 99.0625

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3905 top1= 53.7059


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5623 top1= 47.7464


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.6900 top1= 53.2252

Train epoch 10
[E10B0  |    384/60000 (  1%) ] Loss: 0.0862 top1= 98.4375
[E10B10 |   4224/60000 (  7%) ] Loss: 0.0280 top1= 99.0625
[E10B20 |   8064/60000 ( 13%) ] Loss: 0.0247 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.2661 top1= 55.6691


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5162 top1= 47.7865


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.9145 top1= 53.0549

Train epoch 11
[E11B0  |    384/60000 (  1%) ] Loss: 0.0579 top1= 99.0625
[E11B10 |   4224/60000 (  7%) ] Loss: 0.0246 top1= 99.0625
[E11B20 |   8064/60000 ( 13%) ] Loss: 0.0220 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3730 top1= 53.9062


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5357 top1= 47.5761


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.6934 top1= 54.8277

Train epoch 12
[E12B0  |    384/60000 (  1%) ] Loss: 0.0575 top1= 98.7500
[E12B10 |   4224/60000 (  7%) ] Loss: 0.0237 top1= 99.0625
[E12B20 |   8064/60000 ( 13%) ] Loss: 0.0224 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3249 top1= 54.5773


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5230 top1= 47.5962


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.7811 top1= 53.4255

Train epoch 13
[E13B0  |    384/60000 (  1%) ] Loss: 0.0559 top1= 99.0625
[E13B10 |   4224/60000 (  7%) ] Loss: 0.0232 top1= 99.0625
[E13B20 |   8064/60000 ( 13%) ] Loss: 0.0216 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3375 top1= 54.4872


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5301 top1= 47.5361


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.7703 top1= 53.0749

Train epoch 14
[E14B0  |    384/60000 (  1%) ] Loss: 0.0563 top1= 98.7500
[E14B10 |   4224/60000 (  7%) ] Loss: 0.0229 top1= 99.0625
[E14B20 |   8064/60000 ( 13%) ] Loss: 0.0216 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3378 top1= 54.4972


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5319 top1= 47.5661


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.7753 top1= 52.8345

Train epoch 15
[E15B0  |    384/60000 (  1%) ] Loss: 0.0557 top1= 98.7500
[E15B10 |   4224/60000 (  7%) ] Loss: 0.0226 top1= 99.0625
[E15B20 |   8064/60000 ( 13%) ] Loss: 0.0213 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3386 top1= 54.5873


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5365 top1= 47.5361


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.7805 top1= 52.6142

Train epoch 16
[E16B0  |    384/60000 (  1%) ] Loss: 0.0558 top1= 98.7500
[E16B10 |   4224/60000 (  7%) ] Loss: 0.0227 top1= 99.0625
[E16B20 |   8064/60000 ( 13%) ] Loss: 0.0213 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3374 top1= 54.6374


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5397 top1= 47.5361


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.7860 top1= 52.4539

Train epoch 17
[E17B0  |    384/60000 (  1%) ] Loss: 0.0559 top1= 98.7500
[E17B10 |   4224/60000 (  7%) ] Loss: 0.0223 top1= 99.0625
[E17B20 |   8064/60000 ( 13%) ] Loss: 0.0208 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3405 top1= 54.7075


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5445 top1= 47.5060


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.7892 top1= 52.2937

Train epoch 18
[E18B0  |    384/60000 (  1%) ] Loss: 0.0557 top1= 98.7500
[E18B10 |   4224/60000 (  7%) ] Loss: 0.0225 top1= 99.0625
[E18B20 |   8064/60000 ( 13%) ] Loss: 0.0208 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3378 top1= 54.7576


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5466 top1= 47.5160


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.7935 top1= 52.0933

Train epoch 19
[E19B0  |    384/60000 (  1%) ] Loss: 0.0557 top1= 98.7500
[E19B10 |   4224/60000 (  7%) ] Loss: 0.0222 top1= 99.0625
[E19B20 |   8064/60000 ( 13%) ] Loss: 0.0206 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3421 top1= 54.6575


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5541 top1= 47.4659


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.7952 top1= 52.1034

Train epoch 20
[E20B0  |    384/60000 (  1%) ] Loss: 0.0556 top1= 98.7500
[E20B10 |   4224/60000 (  7%) ] Loss: 0.0223 top1= 99.0625
[E20B20 |   8064/60000 ( 13%) ] Loss: 0.0205 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3399 top1= 54.7877


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5565 top1= 47.4659


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.7989 top1= 51.9932

Train epoch 21
[E21B0  |    384/60000 (  1%) ] Loss: 0.0557 top1= 98.7500
[E21B10 |   4224/60000 (  7%) ] Loss: 0.0222 top1= 99.0625
[E21B20 |   8064/60000 ( 13%) ] Loss: 0.0203 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3419 top1= 54.7576


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5610 top1= 47.4559


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8041 top1= 51.9231

Train epoch 22
[E22B0  |    384/60000 (  1%) ] Loss: 0.0555 top1= 98.7500
[E22B10 |   4224/60000 (  7%) ] Loss: 0.0224 top1= 99.0625
[E22B20 |   8064/60000 ( 13%) ] Loss: 0.0203 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3404 top1= 54.7776


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5617 top1= 47.4559


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8080 top1= 51.7929

Train epoch 23
[E23B0  |    384/60000 (  1%) ] Loss: 0.0556 top1= 98.7500
[E23B10 |   4224/60000 (  7%) ] Loss: 0.0220 top1= 99.0625
[E23B20 |   8064/60000 ( 13%) ] Loss: 0.0200 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3400 top1= 54.7877


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5667 top1= 47.4259


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8143 top1= 51.6627

Train epoch 24
[E24B0  |    384/60000 (  1%) ] Loss: 0.0548 top1= 98.7500
[E24B10 |   4224/60000 (  7%) ] Loss: 0.0225 top1= 99.0625
[E24B20 |   8064/60000 ( 13%) ] Loss: 0.0202 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3397 top1= 54.7676


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5686 top1= 47.4459


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8180 top1= 51.5625

Train epoch 25
[E25B0  |    384/60000 (  1%) ] Loss: 0.0547 top1= 98.7500
[E25B10 |   4224/60000 (  7%) ] Loss: 0.0220 top1= 99.0625
[E25B20 |   8064/60000 ( 13%) ] Loss: 0.0197 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3404 top1= 54.7776


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5702 top1= 47.4058


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8207 top1= 51.4623

Train epoch 26
[E26B0  |    384/60000 (  1%) ] Loss: 0.0543 top1= 98.7500
[E26B10 |   4224/60000 (  7%) ] Loss: 0.0224 top1= 99.0625
[E26B20 |   8064/60000 ( 13%) ] Loss: 0.0200 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3405 top1= 54.8377


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5701 top1= 47.4659


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8221 top1= 51.4022

Train epoch 27
[E27B0  |    384/60000 (  1%) ] Loss: 0.0542 top1= 98.7500
[E27B10 |   4224/60000 (  7%) ] Loss: 0.0218 top1= 99.0625
[E27B20 |   8064/60000 ( 13%) ] Loss: 0.0198 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3406 top1= 54.8678


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5742 top1= 47.4259


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8261 top1= 51.3722

Train epoch 28
[E28B0  |    384/60000 (  1%) ] Loss: 0.0535 top1= 98.7500
[E28B10 |   4224/60000 (  7%) ] Loss: 0.0223 top1= 99.0625
[E28B20 |   8064/60000 ( 13%) ] Loss: 0.0198 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3394 top1= 54.9179


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5716 top1= 47.4259


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8280 top1= 51.3121

Train epoch 29
[E29B0  |    384/60000 (  1%) ] Loss: 0.0534 top1= 98.7500
[E29B10 |   4224/60000 (  7%) ] Loss: 0.0219 top1= 99.0625
[E29B20 |   8064/60000 ( 13%) ] Loss: 0.0197 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3378 top1= 54.9980


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5743 top1= 47.4259


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8327 top1= 51.2420

Train epoch 30
[E30B0  |    384/60000 (  1%) ] Loss: 0.0526 top1= 98.7500
[E30B10 |   4224/60000 (  7%) ] Loss: 0.0221 top1= 99.0625
[E30B20 |   8064/60000 ( 13%) ] Loss: 0.0195 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3370 top1= 55.0381


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5715 top1= 47.4359


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8353 top1= 51.0817

Train epoch 31
[E31B0  |    384/60000 (  1%) ] Loss: 0.0527 top1= 98.7500
[E31B10 |   4224/60000 (  7%) ] Loss: 0.0220 top1= 99.0625
[E31B20 |   8064/60000 ( 13%) ] Loss: 0.0197 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3355 top1= 55.1282


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5722 top1= 47.4459


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8379 top1= 51.0717

Train epoch 32
[E32B0  |    384/60000 (  1%) ] Loss: 0.0516 top1= 98.7500
[E32B10 |   4224/60000 (  7%) ] Loss: 0.0222 top1= 99.0625
[E32B20 |   8064/60000 ( 13%) ] Loss: 0.0194 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3340 top1= 55.1883


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5690 top1= 47.4359


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8417 top1= 51.0216

Train epoch 33
[E33B0  |    384/60000 (  1%) ] Loss: 0.0512 top1= 98.7500
[E33B10 |   4224/60000 (  7%) ] Loss: 0.0216 top1= 99.0625
[E33B20 |   8064/60000 ( 13%) ] Loss: 0.0192 top1= 99.6875

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3327 top1= 55.2484


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5717 top1= 47.4359


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8452 top1= 51.0116

Train epoch 34
[E34B0  |    384/60000 (  1%) ] Loss: 0.0507 top1= 98.7500
[E34B10 |   4224/60000 (  7%) ] Loss: 0.0225 top1= 99.0625
[E34B20 |   8064/60000 ( 13%) ] Loss: 0.0198 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3305 top1= 55.2885


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5653 top1= 47.5260


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8479 top1= 50.9615

Train epoch 35
[E35B0  |    384/60000 (  1%) ] Loss: 0.0501 top1= 98.7500
[E35B10 |   4224/60000 (  7%) ] Loss: 0.0213 top1= 99.0625
[E35B20 |   8064/60000 ( 13%) ] Loss: 0.0189 top1= 99.6875

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3303 top1= 55.2985


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5710 top1= 47.4359


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8493 top1= 51.0116

Train epoch 36
[E36B0  |    384/60000 (  1%) ] Loss: 0.0499 top1= 98.7500
[E36B10 |   4224/60000 (  7%) ] Loss: 0.0225 top1= 99.0625
[E36B20 |   8064/60000 ( 13%) ] Loss: 0.0197 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3280 top1= 55.4187


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5630 top1= 47.5661


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8526 top1= 50.8614

Train epoch 37
[E37B0  |    384/60000 (  1%) ] Loss: 0.0492 top1= 98.7500
[E37B10 |   4224/60000 (  7%) ] Loss: 0.0212 top1= 99.0625
[E37B20 |   8064/60000 ( 13%) ] Loss: 0.0189 top1= 99.6875

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3267 top1= 55.4087


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5693 top1= 47.4760


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8576 top1= 50.8413

Train epoch 38
[E38B0  |    384/60000 (  1%) ] Loss: 0.0492 top1= 98.7500
[E38B10 |   4224/60000 (  7%) ] Loss: 0.0221 top1= 99.0625
[E38B20 |   8064/60000 ( 13%) ] Loss: 0.0196 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3248 top1= 55.5288


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5620 top1= 47.5761


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8614 top1= 50.7312

Train epoch 39
[E39B0  |    384/60000 (  1%) ] Loss: 0.0483 top1= 99.0625
[E39B10 |   4224/60000 (  7%) ] Loss: 0.0213 top1= 99.0625
[E39B20 |   8064/60000 ( 13%) ] Loss: 0.0189 top1= 99.6875

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3227 top1= 55.5489


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5640 top1= 47.4860


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8613 top1= 50.7812

Train epoch 40
[E40B0  |    384/60000 (  1%) ] Loss: 0.0483 top1= 99.0625
[E40B10 |   4224/60000 (  7%) ] Loss: 0.0216 top1= 99.0625
[E40B20 |   8064/60000 ( 13%) ] Loss: 0.0195 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3236 top1= 55.6190


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5631 top1= 47.5661


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8654 top1= 50.6911

Train epoch 41
[E41B0  |    384/60000 (  1%) ] Loss: 0.0478 top1= 99.0625
[E41B10 |   4224/60000 (  7%) ] Loss: 0.0212 top1= 99.0625
[E41B20 |   8064/60000 ( 13%) ] Loss: 0.0188 top1= 99.6875

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3197 top1= 55.6290


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5631 top1= 47.4960


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8672 top1= 50.7612

Train epoch 42
[E42B0  |    384/60000 (  1%) ] Loss: 0.0478 top1= 99.0625
[E42B10 |   4224/60000 (  7%) ] Loss: 0.0214 top1= 99.0625
[E42B20 |   8064/60000 ( 13%) ] Loss: 0.0194 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3191 top1= 55.6891


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5609 top1= 47.5962


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8717 top1= 50.6410

Train epoch 43
[E43B0  |    384/60000 (  1%) ] Loss: 0.0471 top1= 99.0625
[E43B10 |   4224/60000 (  7%) ] Loss: 0.0210 top1= 99.0625
[E43B20 |   8064/60000 ( 13%) ] Loss: 0.0187 top1= 99.6875

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3171 top1= 55.7492


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5600 top1= 47.5260


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8723 top1= 50.7011

Train epoch 44
[E44B0  |    384/60000 (  1%) ] Loss: 0.0476 top1= 99.0625
[E44B10 |   4224/60000 (  7%) ] Loss: 0.0213 top1= 99.0625
[E44B20 |   8064/60000 ( 13%) ] Loss: 0.0193 top1= 99.6875

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3161 top1= 55.7792


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5587 top1= 47.6162


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8762 top1= 50.6210

Train epoch 45
[E45B0  |    384/60000 (  1%) ] Loss: 0.0468 top1= 99.0625
[E45B10 |   4224/60000 (  7%) ] Loss: 0.0208 top1= 99.0625
[E45B20 |   8064/60000 ( 13%) ] Loss: 0.0186 top1= 99.6875

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3145 top1= 55.8093


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5607 top1= 47.5561


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8748 top1= 50.7212

Train epoch 46
[E46B0  |    384/60000 (  1%) ] Loss: 0.0474 top1= 99.0625
[E46B10 |   4224/60000 (  7%) ] Loss: 0.0212 top1= 99.0625
[E46B20 |   8064/60000 ( 13%) ] Loss: 0.0193 top1= 99.6875

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3141 top1= 55.8393


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5575 top1= 47.6462


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8774 top1= 50.6711

Train epoch 47
[E47B0  |    384/60000 (  1%) ] Loss: 0.0467 top1= 99.0625
[E47B10 |   4224/60000 (  7%) ] Loss: 0.0207 top1= 99.0625
[E47B20 |   8064/60000 ( 13%) ] Loss: 0.0186 top1= 99.3750

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3110 top1= 55.8794


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5593 top1= 47.5661


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8786 top1= 50.7212

Train epoch 48
[E48B0  |    384/60000 (  1%) ] Loss: 0.0471 top1= 99.0625
[E48B10 |   4224/60000 (  7%) ] Loss: 0.0209 top1= 99.0625
[E48B20 |   8064/60000 ( 13%) ] Loss: 0.0190 top1= 99.6875

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3094 top1= 55.8594


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5572 top1= 47.6562


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8846 top1= 50.6110

Train epoch 49
[E49B0  |    384/60000 (  1%) ] Loss: 0.0465 top1= 99.0625
[E49B10 |   4224/60000 (  7%) ] Loss: 0.0207 top1= 99.0625
[E49B20 |   8064/60000 ( 13%) ] Loss: 0.0186 top1= 99.6875

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3094 top1= 55.8894


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5610 top1= 47.6162


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8825 top1= 50.6510

Train epoch 50
[E50B0  |    384/60000 (  1%) ] Loss: 0.0471 top1= 99.0625
[E50B10 |   4224/60000 (  7%) ] Loss: 0.0208 top1= 99.0625
[E50B20 |   8064/60000 ( 13%) ] Loss: 0.0190 top1= 99.6875

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.3079 top1= 55.9495


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=4.5596 top1= 47.6863


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=1.8863 top1= 50.5709

