
=== Start adding workers ===
=> Add worker SGDMWorker(index=0, momentum=0.9)
=> Add worker SGDMWorker(index=1, momentum=0.9)
=> Add worker SGDMWorker(index=2, momentum=0.9)
=> Add worker SGDMWorker(index=3, momentum=0.9)
=> Add worker SGDMWorker(index=4, momentum=0.9)
=> Add worker SGDMWorker(index=5, momentum=0.9)
=> Add worker SGDMWorker(index=6, momentum=0.9)
=> Add worker SGDMWorker(index=7, momentum=0.9)
=> Add worker SGDMWorker(index=8, momentum=0.9)
=> Add worker SGDMWorker(index=9, momentum=0.9)
=> Add worker BitFlippingWorker
=> Add worker BitFlippingWorker

=== Start adding graph ===
<codes.graph_utils.Dumbbell object at 0x7fb2db2596d0>

Train epoch 1
[E 1B0  |    384/60000 (  1%) ] Loss: 2.3052 top1= 10.3125

=== Peeking data label distribution E1B0 ===
Worker 0 has targets: tensor([3, 1, 0, 0, 2], device='cuda:0')
Worker 1 has targets: tensor([1, 4, 3, 3, 4], device='cuda:0')
Worker 2 has targets: tensor([2, 3, 0, 4, 4], device='cuda:0')
Worker 3 has targets: tensor([4, 4, 3, 4, 1], device='cuda:0')
Worker 4 has targets: tensor([0, 2, 2, 3, 4], device='cuda:0')
Worker 5 has targets: tensor([9, 8, 5, 9, 9], device='cuda:0')
Worker 6 has targets: tensor([5, 9, 8, 9, 5], device='cuda:0')
Worker 7 has targets: tensor([5, 7, 6, 6, 8], device='cuda:0')
Worker 8 has targets: tensor([8, 9, 8, 9, 5], device='cuda:0')
Worker 9 has targets: tensor([8, 5, 9, 6, 6], device='cuda:0')
Worker 10 has targets: tensor([3, 1, 0, 0, 2], device='cuda:0')
Worker 11 has targets: tensor([9, 8, 5, 9, 9], device='cuda:0')


[E 1B10 |   4224/60000 (  7%) ] Loss: 1.5017 top1= 50.9375
[E 1B20 |   8064/60000 ( 13%) ] Loss: 0.7892 top1= 78.1250
[E 1B30 |  11904/60000 ( 20%) ] Loss: 0.4090 top1= 90.9375
[E 1B40 |  15744/60000 ( 26%) ] Loss: 0.4566 top1= 87.8125
[E 1B50 |  19584/60000 ( 33%) ] Loss: 0.4149 top1= 87.5000
[E 1B60 |  23424/60000 ( 39%) ] Loss: 0.3743 top1= 90.0000
[E 1B70 |  27264/60000 ( 45%) ] Loss: 0.3040 top1= 93.1250
[E 1B80 |  31104/60000 ( 52%) ] Loss: 0.3804 top1= 88.1250
[E 1B90 |  34944/60000 ( 58%) ] Loss: 0.3269 top1= 92.1875
[E 1B100|  38784/60000 ( 65%) ] Loss: 0.3349 top1= 90.6250
[E 1B110|  42624/60000 ( 71%) ] Loss: 0.3500 top1= 90.6250
[E 1B120|  46464/60000 ( 77%) ] Loss: 0.3494 top1= 89.6875
[E 1B130|  50304/60000 ( 84%) ] Loss: 0.3367 top1= 91.5625
[E 1B140|  54144/60000 ( 90%) ] Loss: 0.3494 top1= 91.8750
[E 1B150|  57984/60000 ( 97%) ] Loss: 0.3133 top1= 92.5000
[E 1B160|  61824/60000 (103%) ] Loss: 0.3338 top1= 91.5625
[E 1B170|  65664/60000 (109%) ] Loss: 0.3366 top1= 92.1875
[E 1B180|  69504/60000 (116%) ] Loss: 0.3210 top1= 92.8125

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.0593 top1= 79.6775


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=2.5219 top1= 49.6695


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=2.8372 top1= 44.7516

Train epoch 2
[E 2B0  |    384/60000 (  1%) ] Loss: 0.2688 top1= 94.3750
[E 2B10 |   4224/60000 (  7%) ] Loss: 0.3203 top1= 93.7500
[E 2B20 |   8064/60000 ( 13%) ] Loss: 0.3628 top1= 89.0625
[E 2B30 |  11904/60000 ( 20%) ] Loss: 0.3314 top1= 93.7500
[E 2B40 |  15744/60000 ( 26%) ] Loss: 0.3623 top1= 92.1875
[E 2B50 |  19584/60000 ( 33%) ] Loss: 0.3680 top1= 90.6250
[E 2B60 |  23424/60000 ( 39%) ] Loss: 0.3417 top1= 89.6875
[E 2B70 |  27264/60000 ( 45%) ] Loss: 0.2715 top1= 94.0625
[E 2B80 |  31104/60000 ( 52%) ] Loss: 0.3387 top1= 90.9375
[E 2B90 |  34944/60000 ( 58%) ] Loss: 0.3105 top1= 92.1875
[E 2B100|  38784/60000 ( 65%) ] Loss: 0.2956 top1= 91.8750
[E 2B110|  42624/60000 ( 71%) ] Loss: 0.3037 top1= 91.8750
[E 2B120|  46464/60000 ( 77%) ] Loss: 0.3251 top1= 90.6250
[E 2B130|  50304/60000 ( 84%) ] Loss: 0.3216 top1= 91.5625
[E 2B140|  54144/60000 ( 90%) ] Loss: 0.3286 top1= 91.8750
[E 2B150|  57984/60000 ( 97%) ] Loss: 0.2866 top1= 93.7500
[E 2B160|  61824/60000 (103%) ] Loss: 0.3113 top1= 92.1875
[E 2B170|  65664/60000 (109%) ] Loss: 0.3130 top1= 92.1875
[E 2B180|  69504/60000 (116%) ] Loss: 0.3026 top1= 94.6875

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.0983 top1= 77.6342


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=2.3788 top1= 49.6995


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=2.7448 top1= 45.0421

Train epoch 3
[E 3B0  |    384/60000 (  1%) ] Loss: 0.2643 top1= 94.0625
[E 3B10 |   4224/60000 (  7%) ] Loss: 0.3055 top1= 92.8125
[E 3B20 |   8064/60000 ( 13%) ] Loss: 0.3597 top1= 89.3750
[E 3B30 |  11904/60000 ( 20%) ] Loss: 0.3195 top1= 94.0625
[E 3B40 |  15744/60000 ( 26%) ] Loss: 0.3562 top1= 93.1250
[E 3B50 |  19584/60000 ( 33%) ] Loss: 0.3709 top1= 90.3125
[E 3B60 |  23424/60000 ( 39%) ] Loss: 0.3392 top1= 89.6875
[E 3B70 |  27264/60000 ( 45%) ] Loss: 0.2733 top1= 93.4375
[E 3B80 |  31104/60000 ( 52%) ] Loss: 0.3249 top1= 91.8750
[E 3B90 |  34944/60000 ( 58%) ] Loss: 0.3047 top1= 92.5000
[E 3B100|  38784/60000 ( 65%) ] Loss: 0.2882 top1= 92.1875
[E 3B110|  42624/60000 ( 71%) ] Loss: 0.2979 top1= 93.4375
[E 3B120|  46464/60000 ( 77%) ] Loss: 0.3180 top1= 90.9375
[E 3B130|  50304/60000 ( 84%) ] Loss: 0.3118 top1= 91.8750
[E 3B140|  54144/60000 ( 90%) ] Loss: 0.3164 top1= 92.1875
[E 3B150|  57984/60000 ( 97%) ] Loss: 0.2837 top1= 93.7500
[E 3B160|  61824/60000 (103%) ] Loss: 0.2989 top1= 92.1875
[E 3B170|  65664/60000 (109%) ] Loss: 0.3093 top1= 92.1875
[E 3B180|  69504/60000 (116%) ] Loss: 0.2983 top1= 94.6875

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.1075 top1= 77.3438


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=2.3472 top1= 49.7196


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=2.7313 top1= 44.9720

Train epoch 4
[E 4B0  |    384/60000 (  1%) ] Loss: 0.2702 top1= 93.7500
[E 4B10 |   4224/60000 (  7%) ] Loss: 0.3071 top1= 93.1250
[E 4B20 |   8064/60000 ( 13%) ] Loss: 0.3456 top1= 89.3750
[E 4B30 |  11904/60000 ( 20%) ] Loss: 0.3095 top1= 93.7500
[E 4B40 |  15744/60000 ( 26%) ] Loss: 0.3640 top1= 92.8125
[E 4B50 |  19584/60000 ( 33%) ] Loss: 0.3692 top1= 90.6250
[E 4B60 |  23424/60000 ( 39%) ] Loss: 0.3226 top1= 90.9375
[E 4B70 |  27264/60000 ( 45%) ] Loss: 0.2702 top1= 94.0625
[E 4B80 |  31104/60000 ( 52%) ] Loss: 0.3198 top1= 90.9375
[E 4B90 |  34944/60000 ( 58%) ] Loss: 0.3065 top1= 93.1250
[E 4B100|  38784/60000 ( 65%) ] Loss: 0.2830 top1= 92.5000
[E 4B110|  42624/60000 ( 71%) ] Loss: 0.2889 top1= 92.5000
[E 4B120|  46464/60000 ( 77%) ] Loss: 0.3177 top1= 89.3750
[E 4B130|  50304/60000 ( 84%) ] Loss: 0.3077 top1= 92.5000
[E 4B140|  54144/60000 ( 90%) ] Loss: 0.3131 top1= 93.1250
[E 4B150|  57984/60000 ( 97%) ] Loss: 0.2840 top1= 94.3750
[E 4B160|  61824/60000 (103%) ] Loss: 0.2991 top1= 92.5000
[E 4B170|  65664/60000 (109%) ] Loss: 0.3063 top1= 91.2500
[E 4B180|  69504/60000 (116%) ] Loss: 0.3002 top1= 95.0000

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.1156 top1= 77.7845


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=2.3003 top1= 49.6995


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=2.6542 top1= 44.9619

Train epoch 5
[E 5B0  |    384/60000 (  1%) ] Loss: 0.2680 top1= 94.6875
[E 5B10 |   4224/60000 (  7%) ] Loss: 0.3015 top1= 93.4375
[E 5B20 |   8064/60000 ( 13%) ] Loss: 0.3502 top1= 89.6875
[E 5B30 |  11904/60000 ( 20%) ] Loss: 0.3149 top1= 93.4375
[E 5B40 |  15744/60000 ( 26%) ] Loss: 0.3665 top1= 92.8125
[E 5B50 |  19584/60000 ( 33%) ] Loss: 0.3701 top1= 90.0000
[E 5B60 |  23424/60000 ( 39%) ] Loss: 0.3204 top1= 90.3125
[E 5B70 |  27264/60000 ( 45%) ] Loss: 0.2699 top1= 94.6875
[E 5B80 |  31104/60000 ( 52%) ] Loss: 0.3243 top1= 90.3125
[E 5B90 |  34944/60000 ( 58%) ] Loss: 0.3057 top1= 92.8125
[E 5B100|  38784/60000 ( 65%) ] Loss: 0.2815 top1= 92.8125
[E 5B110|  42624/60000 ( 71%) ] Loss: 0.2874 top1= 92.8125
[E 5B120|  46464/60000 ( 77%) ] Loss: 0.3175 top1= 88.7500
[E 5B130|  50304/60000 ( 84%) ] Loss: 0.3052 top1= 92.5000
[E 5B140|  54144/60000 ( 90%) ] Loss: 0.3180 top1= 93.4375
[E 5B150|  57984/60000 ( 97%) ] Loss: 0.2800 top1= 94.3750
[E 5B160|  61824/60000 (103%) ] Loss: 0.3074 top1= 91.5625
[E 5B170|  65664/60000 (109%) ] Loss: 0.3061 top1= 91.2500
[E 5B180|  69504/60000 (116%) ] Loss: 0.2996 top1= 94.6875

=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.1160 top1= 77.9547


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=2.2978 top1= 49.7396


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=2.6282 top1= 44.9920

Train epoch 6
[E 6B0  |    384/60000 (  1%) ] Loss: 0.2674 top1= 95.3125
[E 6B10 |   4224/60000 (  7%) ] Loss: 0.2975 top1= 93.7500
[E 6B20 |   8064/60000 ( 13%) ] Loss: 0.3500 top1= 90.3125
[E 6B30 |  11904/60000 ( 20%) ] Loss: 0.3164 top1= 94.0625
[E 6B40 |  15744/60000 ( 26%) ] Loss: 0.3740 top1= 92.8125
[E 6B50 |  19584/60000 ( 33%) ] Loss: 0.3775 top1= 88.7500
[E 6B60 |  23424/60000 ( 39%) ] Loss: 0.3138 top1= 90.6250
[E 6B70 |  27264/60000 ( 45%) ] Loss: 0.2655 top1= 94.6875
[E 6B80 |  31104/60000 ( 52%) ] Loss: 0.3264 top1= 90.6250
[E 6B90 |  34944/60000 ( 58%) ] Loss: 0.3051 top1= 93.1250
[E 6B100|  38784/60000 ( 65%) ] Loss: 0.2733 top1= 94.3750
[E 6B110|  42624/60000 ( 71%) ] Loss: 0.2920 top1= 93.1250
[E 6B120|  46464/60000 ( 77%) ] Loss: 0.3139 top1= 90.0000
[E 6B130|  50304/60000 ( 84%) ] Loss: 0.3083 top1= 93.1250
[E 6B140|  54144/60000 ( 90%) ] Loss: 0.3230 top1= 93.1250
[E 6B150|  57984/60000 ( 97%) ] Loss: 0.2785 top1= 94.3750
[E 6B160|  61824/60000 (103%) ] Loss: 0.3144 top1= 91.2500
[E 6B170|  65664/60000 (109%) ] Loss: 0.3097 top1= 91.5625
