
=== Start adding workers ===
=> Add worker SGDMWorker(index=0, momentum=0.9)
=> Add worker SGDMWorker(index=1, momentum=0.9)
=> Add worker SGDMWorker(index=2, momentum=0.9)
=> Add worker SGDMWorker(index=3, momentum=0.9)
=> Add worker SGDMWorker(index=4, momentum=0.9)
=> Add worker SGDMWorker(index=5, momentum=0.9)
=> Add worker SGDMWorker(index=6, momentum=0.9)
=> Add worker SGDMWorker(index=7, momentum=0.9)
=> Add worker SGDMWorker(index=8, momentum=0.9)
=> Add worker SGDMWorker(index=9, momentum=0.9)
=> Add worker SGDMWorker(index=10, momentum=0.9)
=> Add worker ByzantineWorker(index=11)

=== Start adding graph ===
TwoCliquesWithByzantine(m=5,b=1)

Train epoch 1
[E 1B0  |    384/60000 (  1%) ] Loss: 2.3142 top1=  7.6705

=== Peeking data label distribution E1B0 ===
Worker 0 has targets: tensor([0, 0, 0, 0, 0], device='cuda:0')
Worker 1 has targets: tensor([1, 1, 1, 0, 1], device='cuda:0')
Worker 2 has targets: tensor([1, 1, 2, 1, 1], device='cuda:0')
Worker 3 has targets: tensor([2, 2, 3, 2, 2], device='cuda:0')
Worker 4 has targets: tensor([3, 3, 3, 3, 3], device='cuda:0')
Worker 5 has targets: tensor([4, 4, 4, 4, 4], device='cuda:0')
Worker 6 has targets: tensor([5, 5, 5, 5, 5], device='cuda:0')
Worker 7 has targets: tensor([6, 6, 6, 5, 6], device='cuda:0')
Worker 8 has targets: tensor([6, 7, 7, 6, 6], device='cuda:0')
Worker 9 has targets: tensor([7, 7, 8, 7, 7], device='cuda:0')
Worker 10 has targets: tensor([8, 8, 9, 8, 8], device='cuda:0')
Worker 11 has targets: tensor([9, 9, 9, 9, 9], device='cuda:0')



=== Log global consensus distance @ E1B0 ===
consensus_distance=0.000



=== Log clique consensus distance @ E1B0 ===
clique1_consensus_distance=0.000
clique2_consensus_distance=0.000



=== Log mixing matrix @ E1B0 ===
[[0.167 0.167 0.167 0.167 0.167 0.    0.    0.    0.    0.    0.167 0.   ]
 [0.167 0.233 0.2   0.2   0.2   0.    0.    0.    0.    0.    0.    0.   ]
 [0.167 0.2   0.233 0.2   0.2   0.    0.    0.    0.    0.    0.    0.   ]
 [0.167 0.2   0.2   0.233 0.2   0.    0.    0.    0.    0.    0.    0.   ]
 [0.167 0.2   0.2   0.2   0.233 0.    0.    0.    0.    0.    0.    0.   ]
 [0.    0.    0.    0.    0.    0.233 0.2   0.2   0.2   0.167 0.    0.   ]
 [0.    0.    0.    0.    0.    0.2   0.233 0.2   0.2   0.167 0.    0.   ]
 [0.    0.    0.    0.    0.    0.2   0.2   0.233 0.2   0.167 0.    0.   ]
 [0.    0.    0.    0.    0.    0.2   0.2   0.2   0.233 0.167 0.    0.   ]
 [0.    0.    0.    0.    0.    0.167 0.167 0.167 0.167 0.167 0.167 0.   ]
 [0.167 0.    0.    0.    0.    0.    0.    0.    0.    0.167 0.417 0.25 ]
 [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.25  0.75 ]]


[E 1B10 |   4224/60000 (  7%) ] Loss: 2.0693 top1= 41.4773

=== Log global consensus distance @ E1B10 ===
consensus_distance=0.058



=== Log clique consensus distance @ E1B10 ===
clique1_consensus_distance=0.003
clique2_consensus_distance=0.031


[E 1B20 |   8064/60000 ( 13%) ] Loss: 1.8221 top1= 46.8750

=== Log global consensus distance @ E1B20 ===
consensus_distance=0.089



=== Log clique consensus distance @ E1B20 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.134



=> Averaged model (Global Average Validation Accuracy) | Eval Loss=2.2433 top1=  9.7456


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=2.3878 top1= 28.8762


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=2.3424 top1= 24.3089

Train epoch 2
[E 2B0  |    384/60000 (  1%) ] Loss: 1.5301 top1= 55.6818

=== Log global consensus distance @ E2B0 ===
consensus_distance=0.124



=== Log clique consensus distance @ E2B0 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.290


[E 2B10 |   4224/60000 (  7%) ] Loss: 1.2417 top1= 62.2159

=== Log global consensus distance @ E2B10 ===
consensus_distance=0.156



=== Log clique consensus distance @ E2B10 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.434


[E 2B20 |   8064/60000 ( 13%) ] Loss: 1.0225 top1= 73.2955

=== Log global consensus distance @ E2B20 ===
consensus_distance=0.173



=== Log clique consensus distance @ E2B20 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.513



=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.9501 top1= 13.8221


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=2.6977 top1= 38.4215


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=2.8022 top1= 35.0962

Train epoch 3
[E 3B0  |    384/60000 (  1%) ] Loss: 0.8076 top1= 79.2614

=== Log global consensus distance @ E3B0 ===
consensus_distance=0.180



=== Log clique consensus distance @ E3B0 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.545


[E 3B10 |   4224/60000 (  7%) ] Loss: 0.6866 top1= 86.9318

=== Log global consensus distance @ E3B10 ===
consensus_distance=0.181



=== Log clique consensus distance @ E3B10 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.550


[E 3B20 |   8064/60000 ( 13%) ] Loss: 0.5980 top1= 85.5114

=== Log global consensus distance @ E3B20 ===
consensus_distance=0.178



=== Log clique consensus distance @ E3B20 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.539



=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.5166 top1= 56.3301


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=2.5191 top1= 39.0124


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=2.7636 top1= 37.7704

Train epoch 4
[E 4B0  |    384/60000 (  1%) ] Loss: 0.4355 top1= 92.0455

=== Log global consensus distance @ E4B0 ===
consensus_distance=0.173



=== Log clique consensus distance @ E4B0 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.518


[E 4B10 |   4224/60000 (  7%) ] Loss: 0.4274 top1= 90.0568

=== Log global consensus distance @ E4B10 ===
consensus_distance=0.165



=== Log clique consensus distance @ E4B10 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.482


[E 4B20 |   8064/60000 ( 13%) ] Loss: 0.4350 top1= 89.4886

=== Log global consensus distance @ E4B20 ===
consensus_distance=0.156



=== Log clique consensus distance @ E4B20 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.442



=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.2045 top1= 66.1859


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=2.2936 top1= 39.5232


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=2.7185 top1= 41.2460

Train epoch 5
[E 5B0  |    384/60000 (  1%) ] Loss: 0.3207 top1= 93.7500

=== Log global consensus distance @ E5B0 ===
consensus_distance=0.148



=== Log clique consensus distance @ E5B0 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.411


[E 5B10 |   4224/60000 (  7%) ] Loss: 0.3431 top1= 92.0455

=== Log global consensus distance @ E5B10 ===
consensus_distance=0.141



=== Log clique consensus distance @ E5B10 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.381


[E 5B20 |   8064/60000 ( 13%) ] Loss: 0.3797 top1= 90.0568

=== Log global consensus distance @ E5B20 ===
consensus_distance=0.135



=== Log clique consensus distance @ E5B20 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.357



=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.0855 top1= 64.9439


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=2.2269 top1= 42.6583


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=2.7939 top1= 42.5481

Train epoch 6
[E 6B0  |    384/60000 (  1%) ] Loss: 0.2960 top1= 93.1818

=== Log global consensus distance @ E6B0 ===
consensus_distance=0.132



=== Log clique consensus distance @ E6B0 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.345


[E 6B10 |   4224/60000 (  7%) ] Loss: 0.3114 top1= 92.0455

=== Log global consensus distance @ E6B10 ===
consensus_distance=0.130



=== Log clique consensus distance @ E6B10 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.337


[E 6B20 |   8064/60000 ( 13%) ] Loss: 0.3561 top1= 90.9091

=== Log global consensus distance @ E6B20 ===
consensus_distance=0.129



=== Log clique consensus distance @ E6B20 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.332



=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.1164 top1= 63.4115


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=2.2782 top1= 44.8317


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=3.0442 top1= 42.8486

Train epoch 7
[E 7B0  |    384/60000 (  1%) ] Loss: 0.2994 top1= 93.4659

=== Log global consensus distance @ E7B0 ===
consensus_distance=0.129



=== Log clique consensus distance @ E7B0 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.338


[E 7B10 |   4224/60000 (  7%) ] Loss: 0.2952 top1= 92.3295

=== Log global consensus distance @ E7B10 ===
consensus_distance=0.131



=== Log clique consensus distance @ E7B10 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.349


[E 7B20 |   8064/60000 ( 13%) ] Loss: 0.3426 top1= 91.7614

=== Log global consensus distance @ E7B20 ===
consensus_distance=0.134



=== Log clique consensus distance @ E7B20 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.359



=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.2460 top1= 62.0994


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=2.3636 top1= 45.8634


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=3.4965 top1= 42.9287

Train epoch 8
[E 8B0  |    384/60000 (  1%) ] Loss: 0.3082 top1= 92.3295

=== Log global consensus distance @ E8B0 ===
consensus_distance=0.137



=== Log clique consensus distance @ E8B0 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.375


[E 8B10 |   4224/60000 (  7%) ] Loss: 0.2790 top1= 91.4773

=== Log global consensus distance @ E8B10 ===
consensus_distance=0.141



=== Log clique consensus distance @ E8B10 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.397


[E 8B20 |   8064/60000 ( 13%) ] Loss: 0.3325 top1= 92.0455

=== Log global consensus distance @ E8B20 ===
consensus_distance=0.145



=== Log clique consensus distance @ E8B20 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.413



=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.4188 top1= 61.2280


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=2.4177 top1= 46.1438


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.0867 top1= 43.0589

Train epoch 9
[E 9B0  |    384/60000 (  1%) ] Loss: 0.3141 top1= 93.1818

=== Log global consensus distance @ E9B0 ===
consensus_distance=0.148



=== Log clique consensus distance @ E9B0 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.430


[E 9B10 |   4224/60000 (  7%) ] Loss: 0.2617 top1= 92.6136

=== Log global consensus distance @ E9B10 ===
consensus_distance=0.154



=== Log clique consensus distance @ E9B10 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.455


[E 9B20 |   8064/60000 ( 13%) ] Loss: 0.3283 top1= 92.8977

=== Log global consensus distance @ E9B20 ===
consensus_distance=0.158



=== Log clique consensus distance @ E9B20 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.475



=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.6112 top1= 60.1863


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=2.4851 top1= 46.2540


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=4.7511 top1= 43.1490

Train epoch 10
[E10B0  |    384/60000 (  1%) ] Loss: 0.3195 top1= 94.6023

=== Log global consensus distance @ E10B0 ===
consensus_distance=0.161



=== Log clique consensus distance @ E10B0 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.489


[E10B10 |   4224/60000 (  7%) ] Loss: 0.2520 top1= 92.8977

=== Log global consensus distance @ E10B10 ===
consensus_distance=0.166



=== Log clique consensus distance @ E10B10 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.514


[E10B20 |   8064/60000 ( 13%) ] Loss: 0.3334 top1= 92.3295

=== Log global consensus distance @ E10B20 ===
consensus_distance=0.170



=== Log clique consensus distance @ E10B20 ===
clique1_consensus_distance=0.004
clique2_consensus_distance=0.531



=> Averaged model (Global Average Validation Accuracy) | Eval Loss=1.7922 top1= 60.3165


=> Averaged model (Clique1 Average Validation Accuracy) | Eval Loss=2.5903 top1= 45.9736


=> Averaged model (Clique2 Average Validation Accuracy) | Eval Loss=5.4103 top1= 43.2292

